On Learned Operator Correction in Inverse Problems
Abstract
We discuss the possibility of learning a data-driven explicit model correction for inverse problems and whether such a model correction can be used within a variational framework to obtain regularized reconstructions. This paper discusses the conceptual difficulty of learning such a forward model correction and proceeds to present a possible solution as a forward-adjoint correction that explicitly corrects in both data and solution spaces. We then derive conditions under which solutions to the variational problem with a learned correction converge to solutions obtained with the correct operator. The proposed approach is evaluated on an application to limited view photoacoustic tomography and compared to the established framework of the Bayesian approximation error method.
Keywords
MSC codes
Formats available
You can view the full content in the following formats:
References
1.
J. Adler and O. Öktem, Solving ill-posed inverse problems using iterative deep neural networks, Inverse Problems, 33 (2017), 124007.
2.
J. Adler and O. Öktem, Learned primal-dual reconstruction, IEEE Trans. Med. Imaging, 37 (2018), pp. 1322--1332.
3.
S. Antholzer, M. Haltmeier, and J. Schwab, Deep learning for photoacoustic tomography from sparse data, Inverse Prob. Sci. Eng., 27 (2019), pp. 987--1005.
4.
S. Arridge, J. Kaipio, V. Kolehmainen, M. Schweiger, E. Somersalo, T. Tarvainen, and M. Vauhkonen, Approximation errors and model reduction with an application in optical diffusion tomography, Inverse Problems, 22 (2006), pp. 175--195.
5.
S. Arridge, P. Maass, O. Öktem, and C.-B. Schönlieb, Solving inverse problems using data-driven models, Acta Numer., 28 (2019), pp. 1--174.
6.
P. Beard, Biomedical photoacoustic imaging, Interface Focus, 1 (2011), pp. 602--631.
7.
Y. E. Boink and C. Brune, Learned SVD: Solving Inverse Problems via Hybrid Autoencoding, preprint, https://arxiv.org/abs/1912.10840 (2019).
8.
L. Borcea, V. Druskin, A. Mamanov, S. Moskow, and M. Zaslvsky, Reduced order models for spectral domain inversion: Embedding into the continuous problem and generation of internal data, Inverse Problems, 36 (2020), 055010.
9.
T. A. Bubba, G. Kutyniok, M. Lassas, M. Maerz, W. Samek, S. Siltanen, and V. Srinivasan, Learning the invisible: A hybrid deep learning-shearlet framework for limited angle computed tomography, Inverse Problems, 35 (2019), 064002.
10.
M. Burger, Y. Korolev, and J. Rasch, Convergence rates and structure of solutions of inverse problems with imperfect forward models, Inverse Problems, 35 (2019), 024006.
11.
B. Cox and P. Beard, Fast calculation of pulsed photoacoustic fields in fluids using k-space methods, J. Acoust. Soc. Amer., 117 (2005), pp. 3616--3627.
12.
N. Davoudi, X. L. Deán-Ben, and D. Razansky, Deep learning optoacoustic tomography with sparse data, Nature Mach. Intell., 1 (2019), pp. 453--460.
13.
H. Egger, J.-F. Pietschmann, and M. Schlottbom, Identification of chemotaxis models with volume-filling, SIAM J. Appl. Math., 75 (2015), pp. 275--288.
14.
R. W. Freund, Model reduction methods based on Krylov subspaces, Acta Numer., 12 (2003), pp. 267--319.
15.
S. Guan, A. Khan, S. Sikdar, and P. Chitnis, Fully dense UNet for $2$D sparse photoacoustic tomography artifact removal, IEEE J. Biomed. Health Inform., 24 (2020), pp. 568--576.
16.
S. J. Hamilton and A. Hauptmann, Deep D-bar: Real-time electrical impedance tomography imaging with deep neural networks, IEEE Trans. Med. Imaging, 37 (2018), pp. 2367--2377.
17.
K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson, T. Pock, and F. Knoll, Learning a variational network for reconstruction of accelerated MRI data, Magn. Reson. Med., 79 (2018), pp. 3055--3071.
18.
A. Hauptmann, J. Adler, S. Arridge, and O. Öktem, Multi-scale learned iterative reconstruction, IEEE Trans. Comput. Imaging, to appear.
19.
A. Hauptmann, S. Arridge, F. Lucka, V. Muthurangu, and J. A. Steeden, Real-time cardiovascular MR with spatio-temporal artifact suppression using deep learning--proof of concept in congenital heart disease, Magn. Reson. Med., 81 (2019), pp. 1143--1156.
20.
A. Hauptmann, B. Cox, F. Lucka, N. Huynh, M. Betcke, P. Beard, and S. Arridge, Approximate k-space models and deep learning for fast photoacoustic reconstruction, in International Workshop on Machine Learning for Medical Image Reconstruction, Springer, Cham, Switzerland, 2018, pp. 103--111.
21.
A. Hauptmann, F. Lucka, M. Betcke, N. Huynh, J. Adler, B. Cox, P. Beard, S. Ourselin, and S. Arridge, Model based learning for accelerated, limited-view 3d photoacoustic tomography, IEEE Trans. Med. Imaging, 39 (2018), pp. 1382--1393.
22.
K. H. Jin, M. T. McCann, E. Froustey, and M. Unser, Deep convolutional neural network for inverse problems in imaging, IEEE Trans. Image Process., 26 (2017), pp. 4509--4522.
23.
J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Appl. Math. Sci. 160, Springer, New York, 2005.
24.
J. Kaipio and E. Somersalo, Statistical inverse problems: Discretization, model reduction and inverse crimes, J. Comput. Appl. Math., 198 (2007), pp. 493--504.
25.
E. Kang, J. Min, and J. C. Ye, A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction, Med. Phys., 44 (2017), pp. e360--e375.
26.
M. C. Kennedy and A. O'Hagan, Bayesian calibration of computer models, J. R. Stat. Soc. Ser. B Stat. Methodol., 63 (2001), pp. 425--464.
27.
E. Kobler, A. Effland, K. Kunisch, and T. Pock, Total deep variation for linear inverse problems, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Piscataway, NJ, 2020, pp. 7549--7558.
28.
K. P. Koestli, M. Frenz, H. Bebie, and H. P. Weber, Temporal backward projection of optoacoustic pressure transients using Fourier transform methods, Phys. Med. Biol., 46 (2001), pp. 1863--1872.
29.
Y. Korolev and J. Lellmann, Image reconstruction with imperfect forward models and applications in deblurring, SIAM J. Imaging Sci., 11 (2018), pp. 197--218.
30.
P. Kuchment and L. Kunyansky, Mathematics of thermoacoustic tomography, European J. Appl. Math., 19 (2008), pp. 191--224.
31.
H. Li, J. Schwab, S. Antholzer, and M. Haltmeier, Nett: Solving inverse problems with deep neural networks, Inverse Problems, 36 (2020), 065005.
32.
A. Lorz, J.-F. Pietschmann, and M. Schlottbom, Parameter identification in a structured population model, Inverse problems, 35 (2019), 095008.
33.
S. Lunz, O. Öktem, and C.-B. Schönlieb, Adversarial regularizers in inverse problems, in Advances in Neural Information Processing Systems, Curran Associates, Red Hook, NY, 2018, pp. 8507--8516.
34.
O. Ronneberger, P. Fischer, and T. Brox, U-net: Convolutional networks for biomedical image segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Cham, Switzerland, 2015, pp. 234--241.
35.
T. Sahlström, A. Pulkkinen, J. Tick, J. Leskinen, and T. Tarvainen, Modeling of errors due to uncertainties in ultrasound sensor locations in photoacoustic tomography, IEEE Trans. Med. Imaging, 39 (2020), pp. 2140--2150.
36.
J. Schlemper, J. Caballero, J. V. Hajnal, A. N. Price, and D. Rueckert, A deep cascade of convolutional neural networks for dynamic MR image reconstruction, IEEE Trans. Med. Imaging, 37 (2017), pp. 491--503.
37.
J. Schwab, S. Antholzer, and M. Haltmeier, Deep null space learning for inverse problems: Convergence analysis and rates, Inverse Problems, 35 (2019), 025008.
38.
J. H. Siewerdsen and D. A. Jaffray, Cone-beam computed tomography with a flat-panel imager: Magnitude and effects of X-ray scatter, Med. Phys., 28 (2001), pp. 220--231.
39.
D. Smyl and D. Liu, Less is often more: Applied inverse problems using hp-forward models, J. Comput. Phys., 399 (2019), 108949.
40.
D. Smyl, T. N. Tallman, J. A. Black, A. Hauptmann, and D. Liu, Learning and Correcting Non-Gaussian Model Errors, preprint, https://arxiv.org/abs/2005.14592 (2020).
41.
T. Tarvainen, A. Pulkkinen, B. T. Cox, J. P. Kaipio, and S. R. Arridge, Bayesian image reconstruction in quantitative photoacoustic tomography, IEEE Trans. Med. Imaging, 32 (2013), pp. 2287--2298.
42.
B. E. Treeby and B. T. Cox, k-wave: Matlab toolbox for the simulation and reconstruction of photoacoustic wave fields, J. Biomed. Opt., 15 (2010), 021314.
43.
B. E. Treeby, J. Jaros, A. P. Rendell, and B. T. Cox, Modeling nonlinear ultrasound propagation in heterogeneous media with power law absorption using a k-space pseudospectral method., J. Acoust. Soc. Amer., 131 (2012), pp. 4324--4336, https://doi.org/10.1121/1.4712021.
44.
V. Vishnevskiy, R. Rau, and O. Goksel, Deep variational networks with exponential weighting for learning computed tomography, in International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Cham, Switzerland, 2019, pp. 310--318.
45.
L. W. Y Xu, G. Ambartsoumian, and P. Kuchment, Reconstructions in limited-view thermoacoustic tomography, Med. Phys., 31 (2004), pp. 724--733.
46.
E. Zhang, J. Laufer, and P. Beard, Backward-mode multiwavelength photoacoustic scanner using a planar Fabry-Perot polymer film ultrasound sensor for high-resolution three-dimensional imaging of biological tissues, Appl. Opt., 47 (2008), pp. 561--577.
47.
L. Zhu, Y. Xie, J. Wang, and L. Xing, Scatter correction for cone-beam CT in radiation therapy, Med. Phys., 36 (2009), pp. 2258--2268.
Information & Authors
Information
Published In
Copyright
© 2021, Society for Industrial and Applied Mathematics. This work is licensed under a Creative Commons Attribution 4.0 International License
History
Submitted: 15 May 2020
Accepted: 22 October 2020
Published online: 26 January 2021
Keywords
MSC codes
Authors
Funding Information
Philip Leverhulme Prize
Funding Information
Wellcome Innovator Award : RG98755
Funding Information
RISE : CHiPS, NoMADS
Funding Information
Cantab Capital Institute for the Mathematics of Information
Funding Information
Cambridge Center for Analysis
Funding Information
Academy of Finland https://doi.org/10.13039/501100002341 : 312123, 312342, 334817, 314411
Funding Information
Alan Turing Institute https://doi.org/10.13039/100012338
Funding Information
British Heart Foundation https://doi.org/10.13039/501100000274 : NH/18/1/33511
Funding Information
Jane ja Aatos Erkon Säätiö https://doi.org/10.13039/501100004012
Funding Information
Leverhulme Trust https://doi.org/10.13039/501100000275
Funding Information
Engineering and Physical Sciences Research Council https://doi.org/10.13039/501100000266 : EP/M020533/1, WT101957
Funding Information
Engineering and Physical Sciences Research Council https://doi.org/10.13039/501100000266 : EP/S026045/1, EP/T003553/1, EP/N014588/1
Funding Information
Engineering and Physical Sciences Research Council https://doi.org/10.13039/501100000266 : EP/L016516/1
Funding Information
Engineering and Physical Sciences Research Council https://doi.org/10.13039/501100000266 : EP/N022750/1, EP/T000864/1
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Sequential Model Correction for Nonlinear Inverse ProblemsSIAM Journal on Imaging Sciences, Vol. 16, No. 4 | 19 October 2023
- Operator shifting for noisy elliptic systemsResearch in the Mathematical Sciences, Vol. 10, No. 4 | 12 November 2023
- Divide-and-conquer DNN approach for the inverse point source problem using a few single frequency measurementsInverse Problems, Vol. 39, No. 11 | 9 October 2023
- On global normal linear approximations for nonlinear Bayesian inverse problemsInverse Problems, Vol. 39, No. 5 | 17 March 2023
- Utilizing Variational Autoencoders in the Bayesian Inverse Problem of Photoacoustic TomographySIAM Journal on Imaging Sciences, Vol. 16, No. 1 | 24 January 2023
- Physics-Informed Compressed Sensing for PC-MRI: An Inverse Navier-Stokes ProblemIEEE Transactions on Image Processing, Vol. 32 | 1 Jan 2023
- Learned Reconstruction Methods With Convergence Guarantees: A survey of concepts and applicationsIEEE Signal Processing Magazine, Vol. 40, No. 1 | 1 Jan 2023
- Operator Shifting for General Noisy Matrix SystemsSIAM Journal on Mathematics of Data Science, Vol. 4, No. 4 | 20 December 2022
- Compensation of Modeling Errors for the Aeroacoustic Inverse Problem with Tools from Deep LearningAcoustics, Vol. 4, No. 4 | 27 September 2022
- Proximal nested sampling for high-dimensional Bayesian model selectionStatistics and Computing, Vol. 32, No. 5 | 5 October 2022
- A Model-Based Iterative Learning Approach for Diffuse Optical TomographyIEEE Transactions on Medical Imaging, Vol. 41, No. 5 | 1 May 2022
- Structural engineering from an inverse problems perspectiveProceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 478, No. 2257 | 26 January 2022
- Photon-Counting CT Reconstruction With a Learned Forward OperatorIEEE Transactions on Computational Imaging, Vol. 8 | 1 Jan 2022
- Efficient learning methods for large-scale optimal inversion designNumerical Algebra, Control and Optimization, Vol. 0, No. 0 | 1 Jan 2022
- An Educated Warm Start for Deep Image Prior-Based Micro CT ReconstructionIEEE Transactions on Computational Imaging, Vol. 8 | 1 Jan 2022
- Model reduction in acoustic inversion by artificial neural networkThe Journal of the Acoustical Society of America, Vol. 150, No. 5 | 8 November 2021
- Learning and correcting non-Gaussian model errorsJournal of Computational Physics, Vol. 432 | 1 May 2021
- Model Adaptation for Inverse Problems in ImagingIEEE Transactions on Computational Imaging, Vol. 7 | 1 Jan 2021
- Graph Convolutional Networks for Model-Based Learning in Nonlinear Inverse ProblemsIEEE Transactions on Computational Imaging, Vol. 7 | 1 Jan 2021