Abstract

We introduce a class of higher-order anisotropic total variation regularizers, which are defined for possibly inhomogeneous, smooth elliptic anisotropies, that extends the total generalized variation regularizer and its variants. We propose a primal-dual hybrid gradient approach to approximating numerically the associated gradient flow. This choice of regularizers allows us to preserve and enhance intrinsic anisotropic features in images. This is illustrated on various examples from different imaging applications: image denoising, wavelet-based image zooming, and reconstruction of surfaces from scattered height measurements.

Keywords

  1. total directional variation
  2. anisotropy
  3. denoising
  4. wavelet-based zooming
  5. digital elevation map

MSC codes

  1. 47A52
  2. 49M30
  3. 49N45
  4. 65J22
  5. 94A08

Get full access to this article

View all available purchase options and get full access to this article.

Supplementary Material

Index of Supplementary Materials

Title of paper: Higher-Order Total Directional Variation: Imaging Applications

Authors: Simone Parisotto, Jan Lellmann, Simon Masnou, and Carola-Bibiane Schönlieb

File: M123920_01.pdf

Type: PDF

Contents: More examples/applications.

References

1.
T. Aach, C. Mota, I. Stuke, M. Muhlich, and E. Barth, Analysis of superimposed oriented patterns, IEEE Trans. Image Process., 15 (2006), pp. 3690--3700, https://doi.org/10.1109/TIP.2006.884921.
2.
A. Almansa, F. Cao, Y. Gousseau, and B. Rouge, Interpolation of digital elevation models using AMLE and related methods, IEEE Trans. Geosci. Remote Sensing, 40 (2002), pp. 314--325, https://doi.org/10.1109/36.992791.
3.
L. Alvarez, F. Guichard, P.-L. Lions, and J.-M. Morel, Axioms and fundamental equations of image processing, Arch. Rational Mech. Anal., 123 (1993), pp. 199--257.
4.
C. Ballester, M. Bertalmio, V. Caselles, G. Sapiro, and J. Verdera, Filling-in by joint interpolation of vector fields and gray levels, IEEE Trans. Image Process., 10 (2001), pp. 1200--1211, https://doi.org/10.1109/83.935036.
5.
I. Bayram and M. E. Kamasak, Directional total variation, IEEE Signal Process. Lett., 19 (2012), pp. 781--784, https://doi.org/10.1109/LSP.2012.2220349.
6.
F. Beauducel, READHGT: Import/download NASA SRTM data files (.HGT), 2012, https://www.mathworks.com/matlabcentral/fileexchange/36379.
7.
B. Berkels, M. Burger, M. Droske, O. Nemitz, and M. Rumpf, Cartoon extraction based on anisotropic image classification, in Vision, Modeling, and Visualization Proceedings, 2006, pp. 293--300, http://numod.ins.uni-bonn.de/research/papers/public/BeBuDr06.pdf.
8.
T. R. Bin Wu and X.-C. Tai, Sparse-Data Based 3D Surface Reconstruction for Cartoon and Map, Internal report, UCLA, 2017, ftp://ftp.math.ucla.edu/pub/camreport/cam17-38.pdf.
9.
P. Blomgren and T. F. Chan, Color TV: Total variation methods for restoration of vector-valued images, IEEE Trans. Image Process., 7 (1998), pp. 304--309, https://doi.org/10.1109/83.661180.
10.
K. Bredies and M. Holler, A TGV regularized wavelet based zooming model, in Scale Space and Variational Methods in Computer Vision (SSVM 2013), Springer, 2013, pp. 149--160, https://doi.org/10.1007/978-3-642-38267-3_13.
11.
K. Bredies, K. Kunisch, and T. Pock, Total generalized variation, SIAM J. Imaging Sci., 3 (2010), pp. 492--526, https://doi.org/10.1137/090769521.
12.
V. Caselles, A. Chambolle, D. Cremers, M. Novaga, and T. Pock, An introduction to total variation for image analysis, Theoret. Found. Numer. Methods Sparse Recovery, 9 (2010), pp. 263--340, https://doi.org/10.1515/9783110226157.263.
13.
V. Caselles, A. Chambolle, and M. Novaga, The discontinuity set of solutions of the TV denoising problem and some extensions, Multiscale Model. Simul., 6 (2007), pp. 879--894, https://doi.org/10.1137/070683003.
14.
V. Caselles, B. Tang, and G. Sapiro, Direction diffusion, in Proceedings of the Seventh IEEE International Conference on Computer Vision (ICCV), Vol. 2, 1999, pp. 1245--1252, https://doi.org/10.1109/ICCV.1999.790423.
15.
A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vis., 40 (2011), pp. 120--145, https://doi.org/10.1007/s10851-010-0251-1.
16.
A. Chambolle and T. Pock, An introduction to continuous optimization for imaging, Acta Numer., 25 (2016), pp. 161--319, https://doi.org/10.1017/S096249291600009X.
17.
T. Chan, S. Esedoglu, and F. Park, A fourth order dual method for staircase reduction in texture extraction and image restoration problems, in 2010 IEEE International Conference on Image Processing, 2010, pp. 4137--4140, https://doi.org/10.1109/ICIP.2010.5653199.
18.
T. Chan, A. Marquina, and P. Mulet, High-order total variation-based image restoration, SIAM J. Sci. Comput., 22 (2000), pp. 503--516, https://doi.org/10.1137/S1064827598344169.
19.
K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, Image denoising by sparse 3-D transform-domain collaborative filtering, IEEE Trans. Image Process., 16 (2007), pp. 2080--2095, https://doi.org/10.1109/TIP.2007.901238.
20.
R. Dalgas Kongskov, Y. Dong, and K. Knudsen, Directional Total Generalized Variation Regularization, preprint, https://arxiv.org/abs/1701.02675, 2017.
21.
J. C. De los Reyes, C.-B. Schönlieb, and T. Valkonen, Bilevel parameter learning for higher-order total variation regularisation models, J. Math. Imaging Vis., 57 (2017), pp. 1--25, https://doi.org/10.1007/s10851-016-0662-8.
22.
Y. Dong and M. Hintermüller, Multi-scale Total Variation with Automated Regularization Parameter Selection for Color Image Restoration, Springer, 2009, pp. 271--281, https://doi.org/10.1007/978-3-642-02256-2_23.
23.
M. J. Ehrhardt and M. M. Betcke, Multicontrast MRI reconstruction with structure-guided total variation, SIAM J. Imaging Sci., 9 (2016), pp. 1084--1106, https://doi.org/10.1137/15M1047325.
24.
V. Estellers, S. Soatto, and X. Bresson, Adaptive regularization with the structure tensor, IEEE Trans. Image Process., 24 (2015), pp. 1777--1790, https://doi.org/10.1109/TIP.2015.2409562.
25.
J. Fehrenbach and J.-M. Mirebeau, Sparse non-negative stencils for anisotropic diffusion, J. Math. Imaging Vis., 49 (2014), pp. 123--147, https://doi.org/10.1007/s10851-013-0446-3.
26.
W. Förstner, A feature based correspondence algorithm for image matching, Int. Arch. Photogrammetry Remote Sensing, 26 (1986), pp. 150--166.
27.
M. Grasmair and F. Lenzen, Anisotropic total variation filtering, Appl. Math. Optim., 62 (2010), pp. 323--339, https://doi.org/10.1007/s00245-010-9105-x.
28.
P. K. Hansma, G. Schitter, G. E. Fantner, and C. Prater, High-speed atomic force microscopy, Science, 314 (2006), pp. 601--602, https://doi.org/10.1126/science.1133497.
29.
C. Harris and M. Stephens, A combined corner and edge detector, in Proceedings of the Alvey Vision Conference, Alvety Vision Club, 1988, pp. 23.1--23.6, https://doi.org/10.5244/C.2.23.
30.
M. Kass and A. Witkin, Analyzing oriented patterns, Computer Vision Graphics Image Process., 37 (1987), pp. 362--385, https://doi.org/10.1016/0734-189X(87)90043-0.
31.
R. Kimmel, R. Malladi, and N. Sochen, Images as embedded maps and minimal surfaces: Movies, color, texture, and volumetric medical images, Internat. J. Comput. Vis., 39 (2000), pp. 111--129, https://doi.org/10.1023/A:1008171026419.
32.
F. Knoll, K. Bredies, T. Pock, and R. Stollberger, Second order total generalized variation (TGV) for MRI, Magnetic Resonance in Medicine, 65 (2011), pp. 480--491, https://doi.org/10.1002/mrm.22595.
33.
S. Lefkimmiatis, A. Roussos, P. Maragos, and M. Unser, Structure tensor total variation, SIAM J. Imaging Sci., 8 (2015), pp. 1090--1122, https://doi.org/10.1137/14098154X.
34.
J. Lellmann, J.-M. Morel, and C.-B. Schönlieb, Anisotropic third-order regularization for sparse digital elevation models, in Scale Space and Variational Methods in Computer Vision, Springer, 2013, pp. 161--173, https://doi.org/10.1007/978-3-642-38267-3_14.
35.
F. Lenzen, F. Becker, J. Lellmann, S. Petra, and C. Schnörr, A class of quasi-variational inequalities for adaptive image denoising and decomposition, Comput. Optim. Appl., 54 (2013), pp. 371--398, https://doi.org/10.1007/s10589-012-9456-0.
36.
J. Meinguet, Surface Spline Interpolation: Basic Theory and Computational Aspects, Springer, 1984, pp. 127--142, https://doi.org/10.1007/978-94-009-6466-2_6.
37.
D. Múller and Y. Dufrêne, Atomic Force Microscopy: A nanoscopic window on the cell surface, Trends Cell Biol., 21 (2011), pp. 461 -- 469, https://doi.org/10.1016/j.tcb.2011.04.008.
38.
M. Nikolova, Local strong homogeneity of a regularized estimator, SIAM J. Appl. Math., 61 (2000), pp. 633--658, https://doi.org/10.1137/S0036139997327794.
39.
C. S. Oxvig, T. Arildsen, and T. Larsen, Structure assisted compressed sensing reconstruction of undersampled AFM images, Ultramicroscopy, 172 (2017), pp. 1--9, https://doi.org/10.1016/j.ultramic.2016.09.011.
40.
K. Papafitsoros and C. Schönlieb, A combined first and second order variational approach for image reconstruction, J. Math. Imaging Vis., 48 (2014), pp. 308--338, https://doi.org/10.1007/s10851-013-0445-4.
41.
S. Parisotto, Anisotropic Variational Models and PDEs for Inverse Imaging Problems, Ph.D. thesis, University of Cambridge Repository, 2019, https://doi.org/10.17863/cam.38240.
42.
S. Parisotto, S. Masnou, and C.-B. Schönlieb, Higher-order total directional variation: Analysis, SIAM J. Imaging Sci., 13 (2020), pp. 474--496, https://doi.org/10.1137/19M1239210.
43.
S. Parisotto and C.-B. Schönlieb, Total directional variation for video denoising, in Scale Space and Variational Methods in Computer Vision, J. Lellmann, M. Burger, and J. Modersitzki, eds., Springer, Cham, 2019, pp. 522--534, https://doi.org/10.1007/978-3-030-22368-7_41.
44.
C. Rankl, Atomic Force Microscopy Images of Cell Specimens, 2015, https://doi.org/10.5281/zenodo.17573.
45.
L. Rudin, S. Osher, and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), pp. 259--268, https://doi.org/10.1016/0167-2789(92)90242-F.
46.
G. Sapiro and D. L. Ringach, Anisotropic diffusion of multivalued images with applications to color filtering, IEEE Trans. Image Process., 5 (1996), pp. 1582--1586, https://doi.org/10.1109/83.541429.
47.
O. Savin, $C^1$ regularity for infinity harmonic functions in two dimensions, Arch. Ration. Mech. Anal., 176 (2005), pp. 351--361, https://doi.org/10.1007/s00205-005-0355-8.
48.
G. Schitter and M. Rost, Scanning probe microscopy at video-rate, Materials Today, 11 (2008), pp. 40--48, https://doi.org/10.1016/S1369-7021(09)70006-9.
49.
S. Setzer and G. Steidl, Variational methods with higher-order derivatives in image processing, in Approximation Theory XII, Nashboro Press, Brentwood, TN, 2008, pp. 360--386.
50.
S. Setzer, G. Steidl, and T. Teuber, Infimal convolution regularizations with discrete $\ell_1$-type functionals, Commun. Math. Sci., 9 (2011), pp. 797--827, https://doi.org/10.4310/CMS.2011.v9.n3.a7.
51.
G. Steidl and T. Teuber, Anisotropic smoothing using double orientations, in Scale Space and Variational Methods in Computer Vision, X.-C. Tai, K. Mørken, M. Lysaker, and K.-A. Lie, eds., Springer, Berlin, 2009, pp. 477--489, https://doi.org/10.1007/978-3-642-02256-2_40.
52.
U. Trottenberg, C. Oosterlee, and A. Schüller, Multigrid, Elsevier Science, Academic Press, San Diego, CA, 2001.
53.
D. Tschumperlé and R. Deriche, Vector-valued image regularization with PDEs: A common framework for different applications, IEEE Trans. Pattern Anal. Mach. Intell., 27 (2005), pp. 506--517, https://doi.org/10.1109/TPAMI.2005.87.
54.
USGS, Shuttle Radar Topography Mission, 2006, https://www2.jpl.nasa.gov/srtm/.
55.
T. Valkonen and T. Pock, Acceleration of the PDHGM on partially strongly convex functions, J. Math. Imaging Vis., 59 (2017), pp. 394--414, https://doi.org/10.1007/s10851-016-0692-2.
56.
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, Image quality assessment: From error visibility to structural similarity, IEEE Trans. Image Process., 13 (2004), pp. 600--612, https://doi.org/10.1109/TIP.2003.819861.
57.
J. Weickert, Anisotropic Diffusion in Image Processing, B. G. Teubner, Stuttgart, 1998.
58.
J. Weickert, Coherence-enhancing diffusion filtering, Internat. J. Comput. Vis., 31 (1999), pp. 111--127, https://doi.org/10.1023/A:1008009714131.
59.
J. Weickert and H. Scharr, A scheme for coherence-enhancing diffusion filtering with optimized rotation invariance, J. Visual Commun. Image Representation, 13 (2002), pp. 103--118, https://doi.org/10.1006/jvci.2001.0495.
60.
C. Wu and X.-C. Tai, Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models, SIAM J. Imaging Sci., 3 (2010), pp. 300--339, https://doi.org/10.1137/090767558.
61.
S. D. Zenzo, A note on the gradient of a multi-image, Computer Vision Graphics Image Process., 33 (1986), pp. 116--125, https://doi.org/10.1016/0734-189X(86)90223-9.
62.
H. Zhang and Y. Wang, Edge adaptive directional total variation, J. Engrg., (2013), https://doi.org/10.1049/joe.2013.0116.

Information & Authors

Information

Published In

cover image SIAM Journal on Imaging Sciences
SIAM Journal on Imaging Sciences
Pages: 2063 - 2104
ISSN (online): 1936-4954

History

Submitted: 17 January 2019
Accepted: 13 July 2020
Published online: 19 November 2020

Keywords

  1. total directional variation
  2. anisotropy
  3. denoising
  4. wavelet-based zooming
  5. digital elevation map

MSC codes

  1. 47A52
  2. 49M30
  3. 49N45
  4. 65J22
  5. 94A08

Authors

Affiliations

Carola-Bibiane Schönlieb

Funding Information

Agence Nationale de la Recherche https://doi.org/10.13039/501100001665 : MILYON/ANR-10-LABX-0070, ANR-14-CE27-0019
Alan Turing Institute https://doi.org/10.13039/100012338 : TU/B/000071
Engineering and Physical Sciences Research Council https://doi.org/10.13039/501100000266 : EP/M00483X/1, EP/K009745/1, EP/N014588/1, EP/L016516/1

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media

The SIAM Publications Library now uses SIAM Single Sign-On for individuals. If you do not have existing SIAM credentials, create your SIAM account https://my.siam.org.