How Accurate Can Block Matches Be in Stereo Vision?
Abstract
This article explores the subpixel accuracy attainable for the disparity computed from a rectified stereo pair of images with small baseline. In this framework we consider translations as the local deformation model between patches in the images. A mathematical study first shows how discrete block-matching can be performed with arbitrary precision under Shannon–Whittaker conditions. This study leads to the specification of a block-matching algorithm which is able to refine disparities with subpixel accuracy. Moreover, a formula for the variance of the disparity error caused by the noise is introduced and proved. Several simulated and real experiments show a decent agreement between this theoretical error variance and the observed root mean squared error in stereo pairs with good signal-to-noise ratio and low baseline. A practical consequence is that under realistic sampling and noise conditions in optical imaging, the disparity map in stereo-rectified images can be computed for the majority of pixels (but only for those pixels with meaningful matches) with a $1/20$ pixel precision.
MSC codes
Keywords
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
M. Balci and H. Foroosh, Subpixel estimation of shifts directly in the Fourier domain, IEEE Trans. Image Process., 15 (2006), pp. 1965–1972.
2.
A. Banno and K. Ikeuchi, Disparity map refinement and 3D surface smoothing via directed anisotropic diffusion, in Proceedings of the 12th IEEE International Conference on Computer Vision Workshops, 2009, pp. 1870–1877.
3.
S. Birchfield and C. Tomasi, A pixel dissimilarity measure that is insensitive to image sampling, IEEE Trans. Pattern Anal. Mach. Intell., 20 (1998), pp. 401–406.
4.
G. Blanchet, A. Buades, B. Coll, J.-M. Morel, and B. Rougé, Fattening Free Correlation Algorithms, preprint, 2010; available online from http://hal.archives-ouvertes.fr/hal-00466406/en/.
5.
Y. Boykov and V. Kolmogorov, An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision, IEEE Trans. Pattern Anal. Mach. Intell., 26 (2004), pp. 1124–1137.
6.
M. Z. Brown, D. Burschka, and G. D. Hager, Advances in computational stereo, IEEE Trans. Pattern Anal. Mach. Intell., 25 (2003), pp. 993–1008.
7.
T. Buades, Y. Lou, J.-M. Morel, and Z. Tang, A note on multi-image denoising, in Proceedings of the International Workshop on Local and Non-Local Approximation in Image Processing, 2009, pp. 1–15.
8.
A. Chambolle, Total variation minimization and a class of binary MRF models, in Energy Minimization Methods in Computer Vision and Pattern Recognition, Lecture Notes in Comput. Sci. 3757, A. Rangarajan, B. Vemuri, and A. L. Yuille, eds., Springer-Verlag, Berlin, Heidelberg, 2005, pp. 136–152.
9.
L. Chen and K.-H. Yap, An effective technique for subpixel image registration under noisy conditions, IEEE Trans. Syst. Man Cybernet. A, Syst. Humans, 38 (2008), pp. 881–887.
10.
W. Chen, M. Zhang, and Z. Xiong, Segmentation-based stereo matching with occlusion handling via region border constraints, Comput. Vis. Image Underst., submitted.
11.
J. Darbon and M. Sigelle, A fast and exact algorithm for total variation minimization, in Pattern Recognition and Image Analysis, Springer-Verlag, Berlin, Heidelberg, 2005, pp. 351–359.
12.
J. Delon and B. Rougé, Small baseline stereovision, J. Math. Imaging Vis., 28 (2007), pp. 209–223.
13.
G. Facciolo, Variational Adhesion Correction with Image Based Regularization for Digital Elevation Models, Master's thesis, Universidad de la República, Montevideo, Uruguay, 2005.
14.
G. Facciolo, F. Lecumberry, A. Almansa, A. Pardo, V. Caselles, and B. Rougé, Constrained anisotropic diffusion and some applications, in Proceedings of the British Machine Vision Conference (BMVC 2006), Edinburgh, 2006.
15.
S. Forstmann, Y. Kanou, J. Ohya, S. Thuering, and A. Schmitt, Real-time stereo by using dynamic programming, in Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop, Vol. 3, IEEE Computer Society, Washington, DC, 2004, pp. 29–36.
16.
S. Gehrig and U. Franke, Improving stereo sub-pixel accuracy for long range stereo, in Proceedings of the 11th IEEE International Conference on Computer Vision, 2007, pp. 1–7.
17.
A. Giros, B. Rougé, and H. Vadon, Appariement fin d'images stéréoscopiques et instrument dédié avec un faible coefficient stéréoscopique, French Patent N.0403143, 2004.
18.
M. Gong, R. Yang, L. Wang, and M. Gong, A performance study on different cost aggregation approaches used in real-time stereo matching, Int. J. Comput. Vision, 75 (2007), pp. 283–296.
19.
H. Hirschmuller, Stereo vision in structured environments by consistent semi-global matching, in Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2006, pp. 2386–2393.
20.
H. Ishikawa, Exact optimization for Markov random fields with convex priors, IEEE Trans. Pattern Anal. Mach. Intell., 25 (2003), pp. 1333–1336.
21.
T. Kanade and M. Okutomi, A stereo matching algorithm with an adaptive window: Theory and experiment, IEEE Trans. Pattern Anal. Mach. Intell., 16 (1994), pp. 920–932.
22.
S. B. Kang, R. Szeliski, and J. Chai, Handling occlusions in dense multi-view stereo, in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2001, pp. 103–110.
23.
M. Kasser and Y. Egels, Digital Photogrammetry, Taylor & Francis, London, 2002.
24.
A. Klaus, M. Sormann, and K. Karner, Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure, in Proceedings of the 18th International Conference on Pattern Recognition (ICPR), IEEE Computer Society, Washington, DC, 2006, pp. 15–18.
25.
V. Kolmogorov and R. Zabih, Graph cut algorithms for binocular stereo with occlusions, in Handbook of Mathematical Models in Computer Vision, Springer-Verlag, New York, 2006, pp. 423–437.
26.
J. Kybic, Bootstrap resampling for image registration uncertainty estimation without ground truth, IEEE Trans. Image Process., 19 (2010), pp. 64–73.
27.
H.J. Landau and H.O. Pollak, Prolate spheroidal wave functions, Fourier analysis and uncertainty. III. The dimension of the space of essentially time- and band-limited signals, Bell System Tech. J., 41 (1962), pp. 1295–1336.
28.
D. Lowe, Distinctive image features from scale-invariant keypoints, Int. J. Comput. Vision, 60 (2004), pp. 91–110.
29.
B. D. Lucas and T. Kanade, An iterative image registration technique with an application to stereo vision, in Proceedings of the Seventh International Joint Conference on Artificial Intelligence(IJCAI'81), Morgan Kaufmann, San Francisco, 1981, pp. 674–679.
30.
L. Moisan, Simulation of High-Precision Stereo Pairs Using Geometric Integration, Technical report, CNES, Toulouse, France, 2010.
31.
P. Musé, F. Sur, F. Cao, J.-L. Lisani, and J.-M. Morel, A Theory of Shape Identification, Lecture Notes in Math. 1948, Springer-Verlag, Berlin, Heidelberg, 2008.
32.
D. Nehab, S. Rusinkiewicz, and J. Davis, Improved sub-pixel stereo correspondences through symmetric refinement, in Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05), Vol. 1, IEEE Computer Society, Washington, DC, 2005, pp. 557–563.
33.
Y. Ohta and T. Kanade, Stereo by intra- and inter-scanline search using dynamic programming, IEEE Trans. Pattern Anal. Mach. Intell., 7 (1985), pp. 139–154.
34.
M. Okutomi and T. Kanade, A multiple-baseline stereo, IEEE Trans. Pattern Anal. Mach. Intell., 15 (1993), pp. 353–363.
35.
E. Z. Psarakis and G. D. Evangelidis, An enhanced correlation-based method for stereo correspondence with sub-pixel accuracy, in Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05), Vol. 1, IEEE Computer Society, Washington, DC, 2005, pp. 907–912.
36.
J. Ren, J. Jiang, and T. Vlachos, High-accuracy sub-pixel motion estimation from noisy images in Fourier domain, IEEE Trans. Image Process., 19 (2010), pp. 1379–1384.
37.
D. Robinson and P. Milanfar, Fundamental performance limits in image registration, IEEE Trans. Image Process., 13 (2004), pp. 1185–1199.
38.
D. Robinson and P. Milanfar, Bias minimizing filter design for gradient-based image registration, Signal Process. Image Commun., 20 (2005), pp. 554–568.
39.
G. K. Rohde, A. Aldroubi, and D. M. Healy, Interpolation artifacts in sub-pixel image registration, IEEE Trans. Image Process., 18 (2009), pp. 333–345.
40.
N. Sabater, L. Moisan, G. Blanchet, A. Almansa, and J.-M. Morel, Review of low-baseline stereo algorithms and benchmarks, in Image and Signal Processing for Remote Sensing XVI, Proceedings of SPIE, Vol. 7830, 2010, 783005.
41.
N. Sabater, J.-M. Morel, and A. Almansa, Rejecting Wrong Matches in Stereovision, CMLA Preprint, 2008-28, ENS Cachan, Cachan, France, 2008.
42.
N. Sabater, J.-M. Morel, A. Almansa, and G. Blanchet, Discarding moving objects in quasi-simultaneous stereovision, in Proceedings of the 17th IEEE International Conference on Image Processing (ICIP), 2010, pp. 2957–2960.
43.
D. Scharstein and R. Szeliski, A taxonomy and evaluation of dense two-frame stereo correspondence algorithms, Int. J. Comput. Vision, 47 (2002), pp. 7–42.
44.
C. Schmid and A. Zisserman, The geometry and matching of lines and curves over multiple views, Int. J. Comput. Vision, 40 (2000), pp. 199–234.
45.
M. Shimizu and M. Okutomi, Precise sub-pixel estimation on area-based matching, in Proceedings of the Eighth IEEE International Conference on Computer Vision, Vol. 1, 2001, pp. 90–97.
46.
R. Szeliski and D. Scharstein, Sampling the disparity space image, IEEE Trans. Pattern Anal. Mach. Intell., 26 (2004), pp. 419–425.
47.
Q. Tian and M. N. Huhns, Algorithms for subpixel registration, Comput. Vision Graph. Image Process., 35 (1986), pp. 220–233.
48.
O. Veksler, Fast variable window for stereo correspondence using integral images, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 1, 2003, pp. 556–561.
49.
Z.-F. Wang and Z.-G. Zheng, A region based stereo matching algorithm using cooperative optimization, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–8.
50.
M. Xu, H. Chen, and P. K. Varshney, Ziv–Zakai bounds on image registration, IEEE Trans. Signal Process., 57 (2009), pp. 1745–1755.
51.
Q. Yang, R. Yang, J. Davis, and D. Nister, Spatial-depth super resolution for range images, in Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition (CVPR '07), 2007, pp. 1–8.
52.
I. S. Yetik and A. Nehorai, Performance bounds on image registration, IEEE Trans. Signal Process., 54 (2006), pp. 1737–1749.
53.
S. Yoon and K.-J. Kweon, Adaptive support-weight approach for correspondence search, IEEE Trans. Pattern Anal. Mach. Intell., 28 (2006), pp. 650–656.
Information & Authors
Information
Published In
Copyright
Copyright © 2011 Society for Industrial and Applied Mathematics.
History
Submitted: 7 June 2010
Accepted: 10 January 2011
Published online: 31 March 2011
MSC codes
Keywords
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- StaSiS-Net: A stacked and siamese disparity estimation network for depth reconstruction in modern 3D laparoscopyMedical Image Analysis, Vol. 77 | 1 Apr 2022
- On Biases in Displacement Estimation for Image Registration, with a Focus on PhotomechanicsJournal of Mathematical Imaging and Vision, Vol. 63, No. 7 | 4 May 2021
- A low‐cost underwater particle tracking velocimetry system for measuring in situ particle flux and sedimentation rate in low‐turbulence environmentsLimnology and Oceanography: Methods, Vol. 17, No. 12 | 21 November 2019
- Single-shot 3D shape reconstruction using multi-wavelength pattern projectionDimensional Optical Metrology and Inspection for Practical Applications VIII | 13 May 2019
- Single-frame three-dimensional imaging using spectral-coded patterns and multispectral snapshot camerasOptical Engineering, Vol. 57, No. 12 | 1 Dec 2018
- Dense matching for multi-scale images by propagationMultimedia Tools and Applications, Vol. 77, No. 17 | 1 February 2018
- Depth Estimation Based on Pyramid Normalized Cross-Correlation Algorithm for Vergence ControlIEEE Access, Vol. 6 | 1 Jan 2018
- Robust and Dense Depth Estimation for Light Field ImagesIEEE Transactions on Image Processing, Vol. 26, No. 4 | 1 Apr 2017
- A Subpixel Matching Method for Stereovision of Narrow Baseline Remotely Sensed ImageryMathematical Problems in Engineering, Vol. 2017, No. 1 | 23 February 2017
- Reliable light field multiwindow disparity estimation2016 IEEE International Conference on Image Processing (ICIP) | 1 Sep 2016
- Rectification for fronto-parallel camera movement in 3D reconstruction of printed circuit boards2016 IEEE International Instrumentation and Measurement Technology Conference Proceedings | 1 May 2016
- Reliable Multiscale and Multiwindow Stereo MatchingSIAM Journal on Imaging Sciences, Vol. 8, No. 2 | 21 April 2015
- Linear Multiscale Analysis of Similarities between Images on Riemannian Manifolds: Practical Formula and Affine Covariant MetricsSIAM Journal on Imaging Sciences, Vol. 8, No. 3 | 22 September 2015
- Accurate Disparity Estimation for Plenoptic ImagesComputer Vision - ECCV 2014 Workshops | 20 March 2015
- Improving the matching precision of SIFT2014 IEEE International Conference on Image Processing (ICIP) | 1 Oct 2014
- A tight multiframe registration problem with application to Earth observation satellite design2014 IEEE International Conference on Imaging Systems and Techniques (IST) Proceedings | 1 Oct 2014
- Discriminability Limits in Spatio-Temporal Stereo Block MatchingIEEE Transactions on Image Processing, Vol. 23, No. 5 | 1 May 2014
- Analysis of Image Registration with Tangent DistanceSIAM Journal on Imaging Sciences, Vol. 7, No. 4 | 23 December 2014
- Know Your Limits: Accuracy of Long Range Stereoscopic Object Measurements in PracticeComputer Vision – ECCV 2014 | 1 Jan 2014
- Single-lens low-disparity stereo using microlensesJournal of Systems Architecture, Vol. 59, No. 10 | 1 Nov 2013
- Analysis of Descent-Based Image RegistrationSIAM Journal on Imaging Sciences, Vol. 6, No. 4 | 21 November 2013
- A precise real-time stereo algorithmProceedings of the 27th Conference on Image and Vision Computing New Zealand | 26 November 2012
- Contrast Invariant and Affine sub-pixel Optical Flow2012 19th IEEE International Conference on Image Processing | 1 Sep 2012
- Robust outliers detection in image point matching2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) | 1 Nov 2011
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].