Detection of Intensity and Motion Edges within Optical Flow via Multidimensional Control
Abstract
In this paper, we propose a new optimization approach for the simultaneous computation of optical flow and edge detection therein. Instead of using an Ambrosio–Tortorelli type energy functional, we reformulate the optical flow problem as a multidimensional control problem. The optimal control problem is solved by discretization methods and large-scale optimization techniques. The edge detector can be immediately built from the control variables. We provide three series of numerical examples. The first shows that the mere presence of a gradient restriction has a regularizing effect, while the second demonstrates how to balance the regularizing effects of a term within the objective and the control restriction. The third series of numerical results is concerned with the direct evaluation of a TV-regularization term by introduction of control variables with sign restrictions.
MSC codes
Keywords
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
L. Ambrosio and V. M. Tortorelli, Approximation of functionals depending on jumps by elliptic functionals via $\Gamma$-convergence, Comm. Pure Appl. Math., 43 (1990), pp. 999–1036.
2.
L. Ambrosio and V. M. Tortorelli, On the approximation of free discontinuity problems, Boll. Un. Mat. Ital. B (7), 6 (1992), pp. 105–123.
3.
G. Aubert, R. Deriche, and P. Kornprobst, Computing optical flow via variational techniques, SIAM J. Appl. Math., 60 (1999), pp. 156–182.
4.
G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, 2nd ed., Springer, New York, 2006.
5.
G. Bellettini and A. Coscia, Discrete approximation of a free discontinuity problem, Numer. Funct. Anal. Optim., 15 (1994), pp. 201–224.
6.
B. Bourdin, Image segmentation with a finite element method, M2AN Math. Model. Numer. Anal., 33 (1999), pp. 229–244.
7.
T. Brox, A. Bruhn, and J. Weickert, Variational motion segmentation with level sets, in Computer Vision—ECCV 2006, Lecture Notes in Comput. Sci. 3951, H. Bischof, A. Leonardis, and A. Pinz, eds., Springer, Berlin, Heidelberg, 2006, pp. 471–483.
8.
A. Bruhn, J. Weickert, T. Kohlberger, and C. Schnörr, A multigrid platform for real-time motion computation with discontinuity-preserving variational methods, Int. J. Comput. Vision, 70 (2006), pp. 257–277.
9.
A. Bruhn, J. Weickert, and C. Schnörr, Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods, Int. J. Comput. Vision, 61 (2005), pp. 211–231.
10.
C. Brune, Berechnung des Optischen Flusses und Kantenerkennung mit Optimierungsmethoden, Diploma thesis, Institut für Numerische und Angewandte Mathematik, Westfälische Wilhelms-Universität Münster, Münster, Germany, 2007.
11.
A. Chambolle, An algorithm for total variation minimization and applications, J. Math. Imaging Vision, 20 (2004), pp. 89–97.
12.
B. Dacorogna, Direct Methods in the Calculus of Variations, 2nd ed., Springer, New York, 2008.
13.
G. Deweß and P. Helbig, Einschätzen und Optimieren von Verkehrsflüssen, in Mathematik in der Praxis. Fallstudien aus Industrie, Wirtschaft, Naturwissenschaften und Medizin, A. Bachem, M. Jünger, and R. Schrader, eds., Springer, Berlin, 1995, pp. 473–492.
14.
W. Enkelmann, Investigations of multigrid algorithms for the estimation of optical flow fields in image sequences, Comput. Vision Graph. Image Process., 43 (1988), pp. 150–177.
15.
L. C. Evans and R. F. Gariepy, Measure Theory and Fine Properties of Functions, CRC Press, Boca Raton, FL, 1992.
16.
R. Fourer, D. M. Gay, and B. W. Kernighan, AMPL: A Modeling Language for Mathematical Programming, 2nd ed., Brooks/Cole—Thomson Learning, Pacific Grove, CA, 2003.
17.
L. Franek, M. Franek, H. Maurer, and M. Wagner, Image Restoration and Simultaneous Edge Detection by Optimal Control Methods, Preprint-Reihe Mathematik, Preprint Nr. M-05/2008, BTU Cottbus, Cottbus, Germany; Optimal Control Appl. Methods, to appear.
18.
T. Gandhi, S. Devadiga, R. Kasturi, and O. Camps, Detection of obstacles on runways using ego-motion compensation and tracking of significant features, Image Vision Comput., 18 (2000), pp. 805–815.
19.
H. Grossauer, Inpainting of movies using optical flow, in Mathematical Models for Registration and Applications to Medical Imaging, Math. Ind. 10, O. Scherzer, ed., Springer, Berlin, 2006, pp. 151–162.
20.
L. He, M. Burger, and S. J. Osher, Iterative total variation regularization with non-quadratic fidelity, J. Math. Imaging Vision, 26 (2006), pp. 167–184.
21.
W. Hinterberger and O. Scherzer, Models for image interpolation based on the optical flow, Computing, 66 (2001), pp. 231–247.
22.
W. Hinterberger, O. Scherzer, C. Schnörr, and J. Weickert, Analysis of optical flow models in the framework of the calculus of variations, Numer. Funct. Anal. Optim., 23 (2002), pp. 69–89.
23.
J.-B. Hiriart-Urruty and C. Lemaréchal, Convex Analysis and Minimization Algorithms. I. Fundamentals, Grundlehren Math. Wiss. 305, Springer, Berlin, 1993.
24.
B. K. P. Horn and B. G. Schunck, Determining optical flow, Artificial Intelligence, 17 (1981), pp. 185–203.
25.
B. Jansen, Interior Point Techniques in Optimization, Kluwer Academic Publishers, Dordrecht, The Netherlands, 1997.
26.
B. Jung and G. S. Sukhatme, Real-Time Motion Tracking from a Mobile Robot, Technical Report CRES-05-008, Center for Robotics and Embedded Systems, University of Southern California, Los Angeles.
27.
P. Kornprobst, R. Deriche, and G. Aubert, Image sequence analysis via partial differential equations, J. Math. Imaging Vision, 11 (1999), pp. 5–26.
28.
A. Kumar, S. Haker, A. Stillman, C. W. Curry, D. P. Giddens, A. R. Tannenbaum, and A. Yezzi, Total-variational-based optical flow for cardiac-wall motion tracking, in Medical Imaging 2001: Image Processing, Proc. SPIE 4322, M. Sonka and K. M. Hanson, eds., SPIE—International Society for Optical Engineering, Bellingham, WA, 2001, pp. 1550–1553.
29.
C. Laird and A. Wächter, Introduction to IPOPT: A Tutorial for Downloading, Installing, and Using IPOPT. Revision No. 1024, http://www.coin-or.org/Ipopt/documentation/ (last access: 29.02.2008).
30.
H. Maurer and H. D. Mittelmann, Optimization techniques for solving elliptic control problems with control and state constraints. I. Boundary control, Comput. Optim. Appl., 16 (2000), pp. 29–55.
31.
H. Maurer and H. D. Mittelmann, Optimization techniques for solving elliptic control problems with control and state constraints. II. Distributed control, Comput. Optim. Appl., 18 (2001), pp. 141–160.
32.
B. McCane, K. Novins, D. Crannitch, and B. Galvin, On benchmarking optical flow, Comput. Vis. Image Underst., 84 (2001), pp. 126–143.
33.
P. Nesi, Variational approach to optical flow estimation managing discontinuities, Image Vision Comput., 11 (1993), pp. 419–439.
34.
N. Papenberg, A. Bruhn, T. Brox, S. Didas, and J. Weickert, Highly accurate optic flow computation with theoretically justified warping, Int. J. Comput. Vision, 67 (2006), pp. 141–158.
35.
S. Pickenhain and M. Wagner, Critical points in relaxed deposit problems, in Calculus of Variations and Optimal Control, Chapman & Hall/CRC Res. Notes Math. 411, A. Ioffe, S. Reich, and I. Shafrir, eds., Chapman & Hall/CRC Press, Boca Raton, FL, 2000, pp. 217–236.
36.
K. N. Plataniotis and A. N. Venetsanopoulos, Color Image Processing and Applications, Springer, Berlin, Heidelberg, New York, 2000.
37.
O. Scherzer, M. Grasmair, H. Grossauer, M. Haltmeier, and F. Lenzen, Variational Methods in Imaging, Springer, New York, 2009.
38.
S. P. Sethi and G. L. Thompson, Optimal Control Theory: Applications to Management Science and Economics, 2nd ed., Springer, New York, 2006.
39.
N. Slesareva, A. Bruhn, and J. Weickert, Optic flow goes stereo: A variational method for estimating discontinuity-preserving dense disparity maps, in Pattern Recognition. Proceedings of the 27th DAGM Symposium, Vienna, Austria, 2005, Lecture Notes in Comput. Sci. 3663, W. Kropatsch, R. Sablatnig, and A. Hanbury, eds., Springer, Berlin, Heidelberg, 2005, pp. 33–40.
40.
A. Talukder and L. Matthies, Real-time detection of moving objects for moving vehicles using dense stereo and optical flow, in Proceedings of the 2004 IEEE/RSJ Conference on Intelligent Robots and Systems (IROS), Sendai, Japan, 2004, Vol. IV, IEEE, Piscataway, NJ, 2004, pp. 3718–3725.
41.
A. Telea, T. Preusser, C. Garbe, M. Droske, and M. Rumpf, A variational approach to joint denoising, edge detection and motion estimation, in Pattern Recognition. Proceedings of the 28th DAGM Symposium, Berlin, Germany, 2006, Lecture Notes in Comput. Sci. 4174, K. Franke, K.-R. Müller, B. Nickolay, and R. Schäfer, eds., Springer, Berlin, Heidelberg, 2006, pp. 525–535.
42.
K. Theissen, Optimale Steuerprozesse unter partiellen Differentialgleichungs-Restriktionen mit linear eingehender Steuerfunktion, Ph.D. thesis, Westfälische Wilhelms-Universität Münster, Münster, Germany, 2006.
43.
G. Vossen and H. Maurer, On $L^1$-minimization in optimal control and applications to robotics, Optimal Control Appl. Methods, 27 (2006), pp. 301–321.
44.
A. Wächter and L. T. Biegler, On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming, Math. Program. Ser. A, 106 (2006), pp. 25–57.
45.
M. Wagner, Pontryagin's maximum principle for multidimensional control problems in image processing, J. Optim. Theory Appl., 140 (2009), pp. 543–576.
46.
J. Weickert and T. Brox, Diffusion and regularization of vector- and matrix-valued images, in Inverse Problems, Image Analysis, and Medical Imaging, Contemp. Math. 313, M. Zuhair Nashed and O. Scherzer, eds., AMS, Providence, RI, 2002, pp. 251–268.
47.
J. Weickert and C. Schnörr, A theoretical framework for convex regularizers in PDE-based computation of image motion, Int. J. Comput. Vision, 45 (2001), pp. 245–264.
48.
J. Weickert and C. Schnörr, Variational optic flow computation with a spatio-temporal smoothness constraint, J. Math. Imaging Vision, 14 (2001), pp. 245–255.
Information & Authors
Information
Published In
Copyright
Copyright © 2009 Society for Industrial and Applied Mathematics.
History
Submitted: 23 May 2008
Accepted: 23 June 2009
Published online: 11 November 2009
MSC codes
Keywords
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- A Time-Dependent Joint Segmentation and Registration Model: Application to Longitudinal Registration of Hepatic DCE-MRI Sequences2020 Tenth International Conference on Image Processing Theory, Tools and Applications (IPTA) | 9 Nov 2020
- A Variational Model Dedicated to Joint Segmentation, Registration, and Atlas Generation for Shape AnalysisSIAM Journal on Imaging Sciences, Vol. 13, No. 1 | 27 February 2020
- An Application of Quadratic Measure Filters to the Segmentation of Chorio-Retinal OCT DataJournal of Mathematical Imaging and Vision, Vol. 60, No. 2 | 4 August 2017
- Automated detection of the choroid boundary within OCT image data using quadratic measure filtersJournal of Biomedical Optics, Vol. 22, No. 2 | 11 Feb 2017
- Sobolev regularity of multipliers in multidimensional control problems of Dieudonné‐Rashevsky typePAMM, Vol. 14, No. 1 | 22 December 2014
- Geophysical Imaging of Fluid Flow in Porous MediaSIAM Journal on Scientific Computing, Vol. 36, No. 5 | 30 October 2014
- Multimodal image registration by elastic matching of edge sketches via optimal controlJournal of Industrial & Management Optimization, Vol. 10, No. 2 | 1 Jan 2014
- A Guide to the TV ZooLevel Set and PDE Based Reconstruction Methods in Imaging | 28 August 2013
- Group-Valued Regularization for Motion Segmentation of Articulated ShapesInnovations for Shape Analysis | 26 November 2012
- A direct method for the solution of an optimal control problem arising from image registrationNumerical Algebra, Control and Optimization, Vol. 2, No. 3 | 1 Aug 2012
- A discretization method for the numerical solution of Dieudonné–Rashevsky type problems with application to edge detection within noisy image dataOptimal Control Applications and Methods, Vol. 33, No. 3 | 30 September 2011
- Quasiconvex relaxation of multidimensional control problems with integrands f ( t , ξ , v )ESAIM: Control, Optimisation and Calculus of Variations, Vol. 17, No. 1 | 31 March 2010
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].