Abstract

As second-order methods, Gauss--Newton-type methods can be more effective than first-order methods for the solution of nonsmooth optimization problems with expensive-to-evaluate smooth components. Such methods, however, often do not converge. Motivated by nonlinear inverse problems with nonsmooth regularization, we propose a new Gauss--Newton-type method with inexact relaxed steps. We prove that the method converges to a set of disjoint critical points given that the linearization of the forward operator for the inverse problem is sufficiently precise. We extensively evaluate the performance of the method on electrical impedance tomography (EIT).

Keywords

  1. Gauss--Newton
  2. nonsmooth
  3. nonconvex
  4. electrical impedance tomography (EIT)

MSC codes

  1. 35R30
  2. 68U10
  3. 49M15
  4. 90C26

Get full access to this article

View all available purchase options and get full access to this article.

Supplementary Material

Index of Supplementary Materials

Title of paper: Relaxed Gauss–Newton methods with applications to electrical impedance tomography

Authors: J. Jauhiainen, P. Kuusela, A. Seppänen, and T. Valkonen

File: M132171_01.pdf

Type: PDF

Contents: Additional numerical experiments. Additional reconstruction figures. Additional proximal mappings.

References

1.
F. J. Aragón Artacho and M. H. Geoffroy, Characterization of metric regularity of subdifferentials, J. Convex Anal., 15 (2008), pp. 365--380.
2.
K. Astala and L. Päivärinta, Calderón's inverse conductivity problem in the plane, Ann. of Math. (2), 163 (2006), pp. 265--299.
3.
H. Attouch, J. Bolte, and B. Svaiter, Convergence of descent methods for semi-algebraic and tame problems: Proximal algorithms, forward–backward splitting, and regularized Gauss--Seidel methods, Math. Program., 137 (2013), pp. 91--129, https://doi.org/10.1007/s10107-011-0484-9.
4.
J. M. Bardsley, A. Seppänen, A. Solonen, H. Haario, and J. Kaipio, Randomize-then-optimize for sampling and uncertainty quantification in electrical impedance tomography, SIAM/ASA J. Uncertain. Quantif., 3 (2015), pp. 1136--1158, https://doi.org/10.1137/140978272.
5.
A. Beck, First-Order Methods in Optimization, MOS-SIAM Ser. Optim. 25, SIAM, 2017, https://doi.org/10.1137/1.9781611974997.
6.
J. Bolte, S. Sabach, and M. Teboulle, Nonconvex Lagrangian-based optimization: Monitoring schemes and global convergence, Math. Oper. Res., 43 (2018), pp. 1051--1404, https://doi.org/10.1287/moor.2017.0900.
7.
J. V. Burke and M. C. Ferris, A Gauss--Newton method for convex composite optimization, Math. Programming, 71 (1995), pp. 179--194, https://doi.org/10.1007/BF01585997.
8.
A. P. Calderón, On an inverse boundary value problem, Comput. Appl. Math., 25 (2006), pp. 133--138.
9.
A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vis., 40 (2011), pp. 120--145, https://doi.org/10.1007/s10851-010-0251-1.
10.
K.-S. Cheng, D. Isaacson, J. Newell, and D. G. Gisser, Electrode models for electric current computed tomography, IEEE Trans. Biomed. Engrg., 36 (1989), pp. 918--924.
11.
F. Clarke, Optimization and Nonsmooth Analysis, Classics Appl. Math. 5, SIAM, 1990, https://doi.org/10.1137/1.9781611971309.
12.
C. Clason, S. Mazurenko, and T. Valkonen, Acceleration and global convergence of a first-order primal--dual method for nonconvex problems, SIAM J. Optim., 29 (2019), pp. 933--963, https://doi.org/10.1137/18M1170194.
13.
T. A. Davis, Algorithm 832: Umfpack v4.3---an unsymmetric-pattern multifrontal method, ACM Trans. Math. Softw., 30 (2004), pp. 196--199, https://doi.org/10.1145/992200.992206.
14.
M. H. DeGroot, Optimal Statistical Decisions, reprint of the 1970 original, Wiley Classics Library, Wiley-Interscience [John Wiley & Sons], 2005.
15.
O. Ferreira, M. Gonçalves, and P. Oliveira, Convergence of the Gauss--Newton method for convex composite optimization under a majorant condition, SIAM J. Optim., 23 (2013), pp. 1757--1783, https://doi.org/10.1137/110841606.
16.
G. Garrigos, L. Rosasco, and S. Villa, Convergence of the Forward-Backward Algorithm: Beyond the Worst Case with the Help of Geometry, preprint, https://arxiv.org/abs/1703.09477, 2017.
17.
M. Hallaji, A. Seppänen, and M. Pour-Ghaz, Electrical impedance tomography-based sensing skin for quantitative imaging of damage in concrete, Smart Mater. Struct., 23 (2014), 085001, https://doi.org/10.1088/0964-1726/23/8/085001.
18.
M. Hanke, A regularizing Levenberg-Marquardt scheme, with applications to inverse groundwater filtration problems, Inverse Problems, 13 (1997), pp. 79--95, https://doi.org/10.1088/0266-5611/13/1/007.
19.
J.-B. Hiriart-Urruty and C. Lemaréchal, Fundamentals of Convex Analysis, Springer-Verlag, 2001, https://doi.org/10.1007/978-3-642-56468-0.
20.
J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Appl. Math. Sci. 160, Springer-Verlag, 2005.
21.
B. Kaltenbacher, A. Neubauer, and O. Scherzer, Iterative Regularization Methods for Nonlinear Ill-Posed Problems, Radon Ser. Comput. Appl. Math. 6, De Gruyter, 2008.
22.
R. Kohn and M. Vogelius, Determining conductivity by boundary measurements, Comm. Pure Appl. Math., 37 (1984), pp. 289--298.
23.
R. V. Kohn and M. Vogelius, Determining conductivity by boundary measurements II: Interior results, Comm, Pure Appl. Math., 38 (1985), pp. 643--667.
24.
A. S. Lewis, Active sets, nonsmoothness, and sensitivity, SIAM J. Optim., 13 (2002), pp. 702--725, https://doi.org/10.1137/S1052623401387623.
25.
A. S. Lewis and S. Zhang, Partial smoothness, tilt stability, and generalized Hessians, SIAM J. Optim., 23 (2013), pp. 74--94, https://doi.org/10.1137/110852103.
26.
C. Li and X. Wang, On convergence of the Gauss-Newton method for convex composite optimization, Math. Program., 91 (2002), pp. 349--356, https://doi.org/10.1007/s101070100249.
27.
J. Liang, J. Fadili, and G. Peyré, Local linear convergence of forward--backward under partial smoothness, in Advances in Neural Information Processing Systems, 27, Curran Associates, Inc., 2014, pp. 1970--1978, http://papers.nips.cc/paper/5260-local-linear-convergence-of-forward-backward-under-partial-smoothness.pdf.
28.
A. Lipponen, A. Seppänen, and J. P. Kaipio, Electrical impedance tomography imaging with reduced-order model based on proper orthogonal decomposition, J. Electron. Imaging, 22 (2013), 023008.
29.
S. Mazurenko, J. Jauhiainen, and T. Valkonen, Primal-Dual Block-Proximal Splitting for a Class of Non-convex Problems, preprint, https://arxiv.org/abs/1911.06284, Electron. Trans. Numer. Anal., to appear.
30.
R. Mifflin, Semismooth and semiconvex functions in constrained optimization, SIAM J. Control Optim., 15 (1977), pp. 959--972, https://doi.org/10.1137/0315061.
31.
J. L. Mueller and S. Siltanen, Linear and Nonlinear Inverse Problems with Practical Applications, Comput. Sci. Eng. 10, SIAM, 2012, https://doi.org/10.1137/1.9781611972344.
32.
J. Nocedal and S. Wright, Numerical Optimization, Springer Ser. Oper. Res. Financial Eng., Springer, 2006, https://doi.org/10.1007/978-0-387-40065-5.
33.
J. S. Pang and L. Qi, Nonsmooth equations: Motivation and algorithms, SIAM J. Optim., 3 (1993), pp. 443--465, https://doi.org/10.1137/0803021.
34.
T. Pock and A. Chambolle, Diagonal preconditioning for first order primal-dual algorithms in convex optimization, in Proceedings of the 2011 IEEE International Conference on Computer Vision (ICCV), Barcelona, 2011, pp. 1762--1769, https://doi.org/10.1109/ICCV.2011.6126441.
35.
L. Q. Qi, Convergence analysis of some algorithms for solving nonsmooth equations, Math. Oper. Res., 18 (1993), pp. 227--244, https://doi.org/10.1287/moor.18.1.227.
36.
L. Qi and J. Sun, A nonsmooth version of Newton's method, Math. Programming, 58 (1993), pp. 353--367, https://doi.org/10.1007/bf01581275.
37.
L. I. Rudin, S. Osher, and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), pp. 259--268.
38.
M. Salo, Calderón Problem, lecture notes, Department of Mathematics and Statistics, University of Helsinki, Helsinki, Finland, 2008, http://users.jyu.fi/~salomi/lecturenotes/calderon_lectures.pdf.
39.
S. Salzo and S. Villa, Convergence analysis of a proximal Gauss-Newton method, Comput. Optim. Appl., 53 (2012), pp. 557--589, https://doi.org/10.1007/s10589-012-9476-9.
40.
E. Somersalo, M. Cheney, and D. Isaacson, Existence and uniqueness for electrode models for electric current computed tomography, SIAM J. Appl. Math., 52 (1992), pp. 1023--1040, https://doi.org/10.1137/0152060.
41.
J. Sylvester and G. Uhlmann, A global uniqueness theorem for an inverse boundary value problem, Ann. of Math. (2), 125 (1987), pp. 153--169.
42.
G. Uhlmann, Electrical impedance tomography and Calderón's problem, Inverse Problems, 25 (2009), 123011, https://doi.org/10.1088/0266-5611/25/12/123011.
43.
T. Valkonen, A primal--dual hybrid gradient method for nonlinear operators with applications to MRI, Inverse Problems, 30 (2014), 055012, https://doi.org/10.1088/0266-5611/30/5/055012.
44.
T. Valkonen, Preconditioned Proximal Point Methods and Notions of Partial Subregularity, preprint, https://arxiv.org/abs/1711.05123, 2017, J.Convex Anal., to appear.
45.
T. Valkonen, Block-proximal methods with spatially adapted acceleration, Electron. Trans. Numer. Anal., 51 (2019), pp. 15--49, https://doi.org/10.1553/etna_vol51s15.
46.
T. Valkonen, First-Order Primal-Dual Methods for Nonsmooth Nonconvex Optimisation, preprint, https://arxiv.org/abs/1910.00115, 2019, submitted.
47.
P. J. Vauhkonen, Image Reconstruction in Three-Dimensional Electrical Impedance Tomography, Ph.D. thesis, University of Kuopio, 2004, http://urn.fi/URN:ISBN:951-781-304-x.
48.
A. Voss, Imaging Moisture Flows in Cement-Based Materials Using Electrical Capacitance Tomography, Ph.D. thesis, University of Eastern Finland, 2020.

Information & Authors

Information

Published In

cover image SIAM Journal on Imaging Sciences
SIAM Journal on Imaging Sciences
Pages: 1415 - 1445
ISSN (online): 1936-4954

History

Submitted: 27 February 2020
Accepted: 21 May 2020
Published online: 27 August 2020

Keywords

  1. Gauss--Newton
  2. nonsmooth
  3. nonconvex
  4. electrical impedance tomography (EIT)

MSC codes

  1. 35R30
  2. 68U10
  3. 49M15
  4. 90C26

Authors

Affiliations

Funding Information

Academy of Finland https://doi.org/10.13039/501100002341 : 303801
Academy of Finland https://doi.org/10.13039/501100002341 : 314701, 320022
Horizon 2020 Framework Programme https://doi.org/10.13039/100010661 : 764810
Escuela Politécnica Nacional https://doi.org/10.13039/501100012511 : PIJ-18-03

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

View Options

View options

PDF

View PDF

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media

On May 28, 2024, our site will enter Read Only mode for a limited time in order to complete a platform upgrade. As a result, the following functions will be temporarily unavailable: registering new user accounts, any updates to existing user accounts, access token activations, and shopping cart transactions. Contact [email protected] with any questions.