Relaxed Gauss--Newton Methods with Applications to Electrical Impedance Tomography
Abstract
As second-order methods, Gauss--Newton-type methods can be more effective than first-order methods for the solution of nonsmooth optimization problems with expensive-to-evaluate smooth components. Such methods, however, often do not converge. Motivated by nonlinear inverse problems with nonsmooth regularization, we propose a new Gauss--Newton-type method with inexact relaxed steps. We prove that the method converges to a set of disjoint critical points given that the linearization of the forward operator for the inverse problem is sufficiently precise. We extensively evaluate the performance of the method on electrical impedance tomography (EIT).
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
Supplementary Material
Index of Supplementary Materials
Title of paper: Relaxed Gauss–Newton methods with applications to electrical impedance tomography
Authors: J. Jauhiainen, P. Kuusela, A. Seppänen, and T. Valkonen
File: M132171_01.pdf
Type: PDF
Contents: Additional numerical experiments. Additional reconstruction figures. Additional proximal mappings.
References
1.
F. J. Aragón Artacho and M. H. Geoffroy, Characterization of metric regularity of subdifferentials, J. Convex Anal., 15 (2008), pp. 365--380.
2.
K. Astala and L. Päivärinta, Calderón's inverse conductivity problem in the plane, Ann. of Math. (2), 163 (2006), pp. 265--299.
3.
H. Attouch, J. Bolte, and B. Svaiter, Convergence of descent methods for semi-algebraic and tame problems: Proximal algorithms, forward–backward splitting, and regularized Gauss--Seidel methods, Math. Program., 137 (2013), pp. 91--129, https://doi.org/10.1007/s10107-011-0484-9.
4.
J. M. Bardsley, A. Seppänen, A. Solonen, H. Haario, and J. Kaipio, Randomize-then-optimize for sampling and uncertainty quantification in electrical impedance tomography, SIAM/ASA J. Uncertain. Quantif., 3 (2015), pp. 1136--1158, https://doi.org/10.1137/140978272.
5.
A. Beck, First-Order Methods in Optimization, MOS-SIAM Ser. Optim. 25, SIAM, 2017, https://doi.org/10.1137/1.9781611974997.
6.
J. Bolte, S. Sabach, and M. Teboulle, Nonconvex Lagrangian-based optimization: Monitoring schemes and global convergence, Math. Oper. Res., 43 (2018), pp. 1051--1404, https://doi.org/10.1287/moor.2017.0900.
7.
J. V. Burke and M. C. Ferris, A Gauss--Newton method for convex composite optimization, Math. Programming, 71 (1995), pp. 179--194, https://doi.org/10.1007/BF01585997.
8.
A. P. Calderón, On an inverse boundary value problem, Comput. Appl. Math., 25 (2006), pp. 133--138.
9.
A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vis., 40 (2011), pp. 120--145, https://doi.org/10.1007/s10851-010-0251-1.
10.
K.-S. Cheng, D. Isaacson, J. Newell, and D. G. Gisser, Electrode models for electric current computed tomography, IEEE Trans. Biomed. Engrg., 36 (1989), pp. 918--924.
11.
F. Clarke, Optimization and Nonsmooth Analysis, Classics Appl. Math. 5, SIAM, 1990, https://doi.org/10.1137/1.9781611971309.
12.
C. Clason, S. Mazurenko, and T. Valkonen, Acceleration and global convergence of a first-order primal--dual method for nonconvex problems, SIAM J. Optim., 29 (2019), pp. 933--963, https://doi.org/10.1137/18M1170194.
13.
T. A. Davis, Algorithm 832: Umfpack v4.3---an unsymmetric-pattern multifrontal method, ACM Trans. Math. Softw., 30 (2004), pp. 196--199, https://doi.org/10.1145/992200.992206.
14.
M. H. DeGroot, Optimal Statistical Decisions, reprint of the 1970 original, Wiley Classics Library, Wiley-Interscience [John Wiley & Sons], 2005.
15.
O. Ferreira, M. Gonçalves, and P. Oliveira, Convergence of the Gauss--Newton method for convex composite optimization under a majorant condition, SIAM J. Optim., 23 (2013), pp. 1757--1783, https://doi.org/10.1137/110841606.
16.
G. Garrigos, L. Rosasco, and S. Villa, Convergence of the Forward-Backward Algorithm: Beyond the Worst Case with the Help of Geometry, preprint, https://arxiv.org/abs/1703.09477, 2017.
17.
M. Hallaji, A. Seppänen, and M. Pour-Ghaz, Electrical impedance tomography-based sensing skin for quantitative imaging of damage in concrete, Smart Mater. Struct., 23 (2014), 085001, https://doi.org/10.1088/0964-1726/23/8/085001.
18.
M. Hanke, A regularizing Levenberg-Marquardt scheme, with applications to inverse groundwater filtration problems, Inverse Problems, 13 (1997), pp. 79--95, https://doi.org/10.1088/0266-5611/13/1/007.
19.
J.-B. Hiriart-Urruty and C. Lemaréchal, Fundamentals of Convex Analysis, Springer-Verlag, 2001, https://doi.org/10.1007/978-3-642-56468-0.
20.
J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Appl. Math. Sci. 160, Springer-Verlag, 2005.
21.
B. Kaltenbacher, A. Neubauer, and O. Scherzer, Iterative Regularization Methods for Nonlinear Ill-Posed Problems, Radon Ser. Comput. Appl. Math. 6, De Gruyter, 2008.
22.
R. Kohn and M. Vogelius, Determining conductivity by boundary measurements, Comm. Pure Appl. Math., 37 (1984), pp. 289--298.
23.
R. V. Kohn and M. Vogelius, Determining conductivity by boundary measurements II: Interior results, Comm, Pure Appl. Math., 38 (1985), pp. 643--667.
24.
A. S. Lewis, Active sets, nonsmoothness, and sensitivity, SIAM J. Optim., 13 (2002), pp. 702--725, https://doi.org/10.1137/S1052623401387623.
25.
A. S. Lewis and S. Zhang, Partial smoothness, tilt stability, and generalized Hessians, SIAM J. Optim., 23 (2013), pp. 74--94, https://doi.org/10.1137/110852103.
26.
C. Li and X. Wang, On convergence of the Gauss-Newton method for convex composite optimization, Math. Program., 91 (2002), pp. 349--356, https://doi.org/10.1007/s101070100249.
27.
J. Liang, J. Fadili, and G. Peyré, Local linear convergence of forward--backward under partial smoothness, in Advances in Neural Information Processing Systems, 27, Curran Associates, Inc., 2014, pp. 1970--1978, http://papers.nips.cc/paper/5260-local-linear-convergence-of-forward-backward-under-partial-smoothness.pdf.
28.
A. Lipponen, A. Seppänen, and J. P. Kaipio, Electrical impedance tomography imaging with reduced-order model based on proper orthogonal decomposition, J. Electron. Imaging, 22 (2013), 023008.
29.
S. Mazurenko, J. Jauhiainen, and T. Valkonen, Primal-Dual Block-Proximal Splitting for a Class of Non-convex Problems, preprint, https://arxiv.org/abs/1911.06284, Electron. Trans. Numer. Anal., to appear.
30.
R. Mifflin, Semismooth and semiconvex functions in constrained optimization, SIAM J. Control Optim., 15 (1977), pp. 959--972, https://doi.org/10.1137/0315061.
31.
J. L. Mueller and S. Siltanen, Linear and Nonlinear Inverse Problems with Practical Applications, Comput. Sci. Eng. 10, SIAM, 2012, https://doi.org/10.1137/1.9781611972344.
32.
J. Nocedal and S. Wright, Numerical Optimization, Springer Ser. Oper. Res. Financial Eng., Springer, 2006, https://doi.org/10.1007/978-0-387-40065-5.
33.
J. S. Pang and L. Qi, Nonsmooth equations: Motivation and algorithms, SIAM J. Optim., 3 (1993), pp. 443--465, https://doi.org/10.1137/0803021.
34.
T. Pock and A. Chambolle, Diagonal preconditioning for first order primal-dual algorithms in convex optimization, in Proceedings of the 2011 IEEE International Conference on Computer Vision (ICCV), Barcelona, 2011, pp. 1762--1769, https://doi.org/10.1109/ICCV.2011.6126441.
35.
L. Q. Qi, Convergence analysis of some algorithms for solving nonsmooth equations, Math. Oper. Res., 18 (1993), pp. 227--244, https://doi.org/10.1287/moor.18.1.227.
36.
L. Qi and J. Sun, A nonsmooth version of Newton's method, Math. Programming, 58 (1993), pp. 353--367, https://doi.org/10.1007/bf01581275.
37.
L. I. Rudin, S. Osher, and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), pp. 259--268.
38.
M. Salo, Calderón Problem, lecture notes, Department of Mathematics and Statistics, University of Helsinki, Helsinki, Finland, 2008, http://users.jyu.fi/~salomi/lecturenotes/calderon_lectures.pdf.
39.
S. Salzo and S. Villa, Convergence analysis of a proximal Gauss-Newton method, Comput. Optim. Appl., 53 (2012), pp. 557--589, https://doi.org/10.1007/s10589-012-9476-9.
40.
E. Somersalo, M. Cheney, and D. Isaacson, Existence and uniqueness for electrode models for electric current computed tomography, SIAM J. Appl. Math., 52 (1992), pp. 1023--1040, https://doi.org/10.1137/0152060.
41.
J. Sylvester and G. Uhlmann, A global uniqueness theorem for an inverse boundary value problem, Ann. of Math. (2), 125 (1987), pp. 153--169.
42.
G. Uhlmann, Electrical impedance tomography and Calderón's problem, Inverse Problems, 25 (2009), 123011, https://doi.org/10.1088/0266-5611/25/12/123011.
43.
T. Valkonen, A primal--dual hybrid gradient method for nonlinear operators with applications to MRI, Inverse Problems, 30 (2014), 055012, https://doi.org/10.1088/0266-5611/30/5/055012.
44.
T. Valkonen, Preconditioned Proximal Point Methods and Notions of Partial Subregularity, preprint, https://arxiv.org/abs/1711.05123, 2017, J.Convex Anal., to appear.
45.
T. Valkonen, Block-proximal methods with spatially adapted acceleration, Electron. Trans. Numer. Anal., 51 (2019), pp. 15--49, https://doi.org/10.1553/etna_vol51s15.
46.
T. Valkonen, First-Order Primal-Dual Methods for Nonsmooth Nonconvex Optimisation, preprint, https://arxiv.org/abs/1910.00115, 2019, submitted.
47.
P. J. Vauhkonen, Image Reconstruction in Three-Dimensional Electrical Impedance Tomography, Ph.D. thesis, University of Kuopio, 2004, http://urn.fi/URN:ISBN:951-781-304-x.
48.
A. Voss, Imaging Moisture Flows in Cement-Based Materials Using Electrical Capacitance Tomography, Ph.D. thesis, University of Eastern Finland, 2020.
Information & Authors
Information
Published In
Copyright
© 2020, Society for Industrial and Applied Mathematics.
History
Submitted: 27 February 2020
Accepted: 21 May 2020
Published online: 27 August 2020
Keywords
MSC codes
Authors
Funding Information
Academy of Finland https://doi.org/10.13039/501100002341 : 303801
Academy of Finland https://doi.org/10.13039/501100002341 : 314701, 320022
Horizon 2020 Framework Programme https://doi.org/10.13039/100010661 : 764810
Escuela Politécnica Nacional https://doi.org/10.13039/501100012511 : PIJ-18-03
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- A novel real-time efficacy assessment method for tumor treating fieldsEngineering Analysis with Boundary Elements, Vol. 177 | 1 Aug 2025
- Online optimisation for dynamic electrical impedance tomographyInverse Problems, Vol. 41, No. 5 | 22 April 2025
- Reconstruction of admittivity distribution without phase measurements in multi-frequency electrical impedance tomography (MFEIT)Measurement Science and Technology | 20 February 2025
- Model-Based Deep Unrolling Framework for Electrical Impedance Tomography Image ReconstructionIEEE Transactions on Instrumentation and Measurement, Vol. 74 | 1 Jan 2025
- A Cloud-Edge Collaborative Learning-Based Electrical Impedance Tomography MethodIEEE Transactions on Instrumentation and Measurement, Vol. 74 | 1 Jan 2025
- Integrating Kolmogorov–Arnold Network and UNet Architecture for Reconstruction of 3-D Electrical Impedance Tomography to Assess the Efficacy of Tumor Treating FieldsIEEE Transactions on Instrumentation and Measurement, Vol. 74 | 1 Jan 2025
- Unrolling Reweighted Total Variation-Based Split Bregman Iterative Framework for Electrical Impedance Tomography Image ReconstructionIEEE Transactions on Computational Imaging, Vol. 11 | 1 Jan 2025
- Impact of ISTA and FISTA iterative optimization algorithms on electrical impedance tomography image reconstructionJournal of Electrical Bioimpedance, Vol. 16, No. 1 | 11 March 2025
- A nonsmooth primal-dual method with interwoven PDE constraint solverComputational Optimization and Applications, Vol. 89, No. 1 | 8 June 2024
- A Second-Order Generalized Total Variation with Improved Alternating Direction Method of Multipliers Algorithm for Electrical Impedance Tomography ReconstructionApplied Sciences, Vol. 14, No. 4 | 12 February 2024
- Current Sources for Electrical Impedance Tomography IMPETOMAdvances in Bioengineering and Clinical Engineering | 31 May 2024
- Electrical Impedance Tomography Deep Imaging With Dual-Branch U-Net Based on Deformable Convolution and Hyper-ConvolutionIEEE Transactions on Instrumentation and Measurement, Vol. 73 | 1 Jan 2024
- A Graph Neural Network Approach with Improved Levenberg–Marquardt for Electrical Impedance TomographyApplied Sciences, Vol. 14, No. 2 | 10 January 2024
- Image reconstruction method for electrical impedance tomography based on RBF and attention mechanismComputers and Electrical Engineering, Vol. 110 | 1 Sep 2023
- Image reconstruction for ECT based on high-order approximate sensitivity matrixMeasurement Science and Technology, Vol. 34, No. 9 | 21 June 2023
- Electrical impedance myography method of measuring anisotropic tongue tissuePhysiological Measurement, Vol. 44, No. 5 | 31 May 2023
- A multithreaded real-time solution for 2D EIT reconstruction with the D-bar algorithmJournal of Computational Science, Vol. 67 | 1 Mar 2023
- First-Order Primal–Dual Methods for Nonsmooth Non-convex OptimizationHandbook of Mathematical Models and Algorithms in Computer Vision and Imaging | 7 December 2022
- First-Order Primal–Dual Methods for Nonsmooth Non-convex OptimizationHandbook of Mathematical Models and Algorithms in Computer Vision and Imaging | 25 February 2023
- Limited Electrodes Models in Electrical Impedance Tomography ReconstructionScale Space and Variational Methods in Computer Vision | 10 May 2023
- Electrical Impedance Tomography Image Reconstruction With Attention-Based Deep Convolutional Neural NetworkIEEE Transactions on Instrumentation and Measurement, Vol. 72 | 1 Jan 2023
- HIHU-Net: A Hyper-Information Hybrid U-Net for Image Reconstruction with Electrical Impedance Tomography2022 IEEE International Conference on Imaging Systems and Techniques (IST) | 21 Jun 2022
- Mumford–Shah regularization in electrical impedance tomography with complete electrode modelInverse Problems, Vol. 38, No. 6 | 2 May 2022
- Contact Adapting Electrode Model for Electrical Impedance TomographySIAM Journal on Applied Mathematics, Vol. 82, No. 2 | 15 March 2022
- Dynamic Imaging for Time-Varying Distribution Using Electrical/Ultrasonic Dual-Modality TomographyIEEE Transactions on Instrumentation and Measurement, Vol. 71 | 1 Jan 2022
- Nonplanar sensing skins for structural health monitoring based on electrical resistance tomographyComputer-Aided Civil and Infrastructure Engineering, Vol. 36, No. 12 | 7 May 2021
- Electrical Impedance Tomography for Robot-Aided Internal Radiation TherapyFrontiers in Bioengineering and Biotechnology, Vol. 9 | 21 June 2021
- Comparison of Machine Learning Methods in Electrical Tomography for Detecting Moisture in Building WallsEnergies, Vol. 14, No. 10 | 12 May 2021
- Regularisation, optimisation, subregularityInverse Problems, Vol. 37, No. 4 | 16 March 2021
- Predictive Online Optimisation with Applications to Optical FlowJournal of Mathematical Imaging and Vision, Vol. 63, No. 3 | 4 January 2021
- Variable Metric Forward-Backward Algorithm for Composite Minimization ProblemsSIAM Journal on Optimization, Vol. 31, No. 2 | 6 May 2021
- First-Order Primal–Dual Methods for Nonsmooth Non-convex OptimisationHandbook of Mathematical Models and Algorithms in Computer Vision and Imaging | 31 July 2021
- A Forward-Backward Strategy for Handling Non-linearity in Electrical Impedance TomographyComputational Science and Its Applications – ICCSA 2021 | 11 September 2021
- Supershape Recovery From Electrical Impedance Tomography DataIEEE Transactions on Instrumentation and Measurement, Vol. 70 | 1 Jan 2021
- Supershape Augmented Reconstruction Method Based on Boolean Operations in Electrical Impedance TomographyIEEE Transactions on Instrumentation and Measurement, Vol. 70 | 1 Jan 2021
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].