Abstract.

In this work we connect machine learning techniques, in particular kernel machine learning, to inverse source and scattering problems. We show the proposed kernel machine learning has demonstrated generalization capability and has a rigorous mathematical foundation. The proposed learning is based on the Mercer kernel, the reproducing kernel Hilbert space, the kernel trick, as well as the mathematical theory of inverse source and scattering theory, and the restricted Fourier integral operator. The kernel machine learns a multilayer neural network which outputs an \(\epsilon\)-neighborhood average of the unknown or its nonlinear transformation. We then apply the general architecture to the multifrequency inverse source problem for a fixed observation direction and the Born inverse medium scattering problem. We establish a mathematically justified kernel machine indicator with demonstrated capability in both shape identification and parameter identification, under very general assumptions on the physical unknowns. More importantly, stability estimates are established in the case of both noiseless and noisy measurement data. Of central importance is the interplay between a restricted Fourier integral operator and a corresponding Sturm–Liouville differential operator. Several numerical examples are presented to demonstrate the capability of the proposed kernel machine learning.

Keywords

  1. kernel machine
  2. neural network
  3. inverse problem
  4. target identification
  5. stability
  6. prolate spheroidal wave functions

MSC codes

  1. 78A46
  2. 65R32
  3. 65N21
  4. 33C47
  5. 33E30
  6. 41A30
  7. 65N35

Get full access to this article

View all available purchase options and get full access to this article.

Supplementary Materials

PLEASE NOTE: These supplementary files have not been peer-reviewed.
Index of Supplementary Materials
Title of paper: A Kernel Machine Learning for Inverse Source and Scattering Problems
Authors: Shixu Meng and Bo Zhang
File: KMI_SINUM_Supplement.pdf
Type: PDF
Contents: Background, regression and feature map, model of Born inverse scattering problem, proof of Lemma 5.1, learning the feature space, Legendre polynomials and Legendre-Gauss-Lobatto quadrature, derivations for the Numerical Examples, and prolate spheroidal wave functions and their generalization.

References

1.
A. Aizerman, Theoretical foundations of the potential function method in pattern recognition learning, Automat. Remote Control, 25 (1964), pp. 821–837.
2.
N. Aronszajn, Theory of reproducing kernels, Trans. Amer. Math. Soc., 68 (1950), pp. 337–404.
3.
S. Arora, S. S. Du, W. Hu, Z. Li, R. R. Salakhutdinov, and R. Wang, On exact computation with an infinitely wide neural net, in Advances in Neural Information Processing Systems 32, Curran Associates, 2019, pp. 8141–8150.
4.
L. Audibert and S. Meng, Shape and Parameter Identification by the Linear Sampling Method for a Restricted Fourier Integral Operator, preprint, arXiv:2306.16199, 2023.
5.
M. Belkin, S. Ma, and S. Mandal, To understand deep learning we need to understand kernel learning, in International Conference on Machine Learning, PMLR, 2018, pp. 541–549.
6.
B. E. Boser, I. M. Guyon, and V. N. Vapnik, A training algorithm for optimal margin classifiers, in Proceedings of the Fifth Annual Workshop on Computational Learning Theory, 1992, pp. 144–152.
7.
J. P. Boyd, Algorithm 840: Computation of grid points, quadrature weights and derivatives for spectral element methods using prolate spheroidal wave functions—Prolate elements, ACM Trans. Math. Software, 31 (2005), pp. 149–165.
8.
F. Cakoni and D. Colton, A Qualitative Approach to Inverse Scattering Theory, Appl. Math. Sci. 188, Springer, 2014.
9.
F. Cakoni, D. Colton, and H. Haddar, Inverse Scattering Theory and Transmission Eigenvalues, 2nd ed., SIAM, Philadelphia, 2022, https://doi.org/10.1137/1.9781611977424.
10.
B. Chen, Y. Guo, F. Ma, and Y. Sun, Numerical schemes to reconstruct three-dimensional time-dependent point sources of acoustic waves, Inverse Problems, 36 (2020), 075009.
11.
Y. Chen, W. Huang, L. Nguyen, and T.-W. Weng, On the equivalence between neural network and support vector machine, in Advances in Neural Information Processing Systems 34, Curran Associates, 2021, pp. 23478–23490.
12.
Z. Chen, W. Zhang, and J. Zou, Stochastic convergence of regularized solutions and their finite element approximations to inverse source problems, SIAM J. Numer. Anal., 60 (2022), pp. 751–780, https://doi.org/10.1137/21M1409779.
13.
D. Colton and A. Kirsch, A simple method for solving inverse scattering problems in the resonance region, Inverse Problems, 12 (1996), pp. 383–393.
14.
D. Colton and R. Kress, Inverse Acoustic and Electromagnetic Scattering Theory, Appl. Math. Sci. 93, Springer Nature, 2019.
15.
D. Colton and P. Monk, A novel method for solving the inverse scattering problem for time-harmonic acoustic waves in the resonance region, SIAM J. Appl. Math., 45 (1985), pp. 1039–1053, https://doi.org/10.1137/0145064.
16.
C. Cortes and V. Vapnik, Support-vector networks, Mach. Learn., 20 (1995), pp. 273–297.
17.
Y. Gao, H. Liu, X. Wang, and K. Zhang, On an artificial neural network for inverse scattering problems, J. Comput. Phys., 448 (2022), 110771.
18.
B. Ghojogh, A. Ghodsi, F. Karray, and M. Crowley, Reproducing Kernel Hilbert Space, Mercer’s Theorem, Eigenfunctions, Nyström Method, and Use of Kernels in Machine Learning: Tutorial and Survey, preprint, arXiv:2106.08443, 2021.
19.
I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, MIT Press, 2016.
20.
R. Griesmaier and C. Schmiedecke, A factorization method for multifrequency inverse source problems with sparse far field measurements, SIAM J. Imaging Sci., 10 (2017), pp. 2119–2139, https://doi.org/10.1137/17M111290X.
21.
R. Herbrich, Learning Kernel Classifiers: Theory and Algorithms, MIT Press, 2001.
22.
D. Hilbert, Grundzüge einer allgemeinen Theorie der linearen Integralgleichungen, Nachr. Akad. Wiss. Göttingen Math.-Phys, KI (1904), pp. 49–91.
23.
G. E. Hinton and R. R. Salakhutdinov, Reducing the dimensionality of data with neural networks, Science, 313 (2006), pp. 504–507.
24.
T. Hofmann, B. Schölkopf, and A. J. Smola, Kernel methods in machine learning, Ann. Statist., 36 (2008), pp. 1171–1220.
25.
M. Isaev and R. G. Novikov, Reconstruction from the Fourier transform on the ball via prolate spheroidal wave functions, J. Math. Pures Appl. (9), 163 (2022), pp. 318–333.
26.
A. Jacot, F. Gabriel, and C. Hongler, Neural tangent kernel: Convergence and generalization in neural networks, in Advances in Neural Information Processing Systems 31, Curran Associates, 2018, pp. 8571–8580.
27.
K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. LeCun, What is the best multi-stage architecture for object recognition?, in 2009 IEEE 12th International Conference on Computer Vision, IEEE, 2009, pp. 2146–2153.
28.
Y. Khoo and L. Ying, SwitchNet: A neural network model for forward and inverse scattering problems, SIAM J. Sci. Comput., 41 (2019), pp. A3182–A3201, https://doi.org/10.1137/18M1222399.
29.
A. Kirsch, Characterization of the shape of a scattering obstacle using the spectral data of the far field operator, Inverse Problems, 14 (1998), pp. 1489–1512.
30.
A. Kirsch, An Introduction to the Mathematical Theory of Inverse Problems, Appl. Math. Sci. 120, Springer Nature, 2021.
31.
A. Kirsch and N. Grinberg, The Factorization Method for Inverse Problems, Oxford Lecture Ser. Math. Appl. 36, Oxford University Press, Oxford, 2008.
32.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, ImageNet classification with deep convolutional neural networks, in Advances in Neural Information Processing Systems 25, Curran Associates, 2012, pp. 1097–1105.
33.
F. Li, H. Qiao, and B. Zhang, Discriminatively boosted image clustering with fully convolutional auto-encoders, Pattern Recognition, 83 (2018), pp. 161–173.
34.
Y. Liu, Z. Wu, J. Sun, and Z. Zhang, Deterministic-statistical approach for an inverse acoustic source problem using multiple frequency limited aperture data, Inverse Probl. Imaging, 17 (2023), pp. 1329–1345.
35.
W. C. H. McLean, Strongly Elliptic Systems and Boundary Integral Equations, Cambridge University Press, 2000.
36.
S. Meng, Data-driven basis for reconstructing the contrast in inverse scattering: Picard criterion, regularity, regularization, and stability, SIAM J. Appl. Math., 83 (2023), pp. 2003–2026, https://doi.org/10.1137/23M1545409.
37.
S. Meng, Single mode multi-frequency factorization method for the inverse source problem in acoustic waveguides, SIAM J. Appl. Math., 83 (2023), pp. 394–417, https://doi.org/10.1137/21M1462891.
38.
J. Mercer, Functions of positive and negative type, and their connection with the theory of integral equations, Philos. Trans. Roy. Soc. London Ser. A, 209 (1909), pp. 415–446.
39.
V. Nair and G. E. Hinton, Rectified linear units improve restricted Boltzmann machines, in Proceedings of the 27th International Conference on Machine Learning (ICML-10), 2010, pp. 807–814.
40.
F. Natterer, The Mathematics of Computerized Tomography, SIAM, Philadelphia, 2001, https://doi.org/10.1137/1.9780898719284.
41.
J. Ning, F. Han, and J. Zou, A direct sampling-based deep learning approach for inverse medium scattering problems, Inverse Problems, 40 (2023), 015005.
42.
A. Osipov, V. Rokhlin, and H. Xiao, Prolate Spheroidal Wave Functions of Order Zero: Mathematical Tools for Bandlimited Approximation, Appl. Math. Sci. 187, Springer, 2013.
43.
V. I. Paulsen and M. Raghupathi, An Introduction to the Theory of Reproducing Kernel Hilbert Spaces, Cambridge Stud. Adv. Math. 152, Cambridge University Press, Cambridge, 2016.
44.
F. Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms, Spartan Books Washington, DC, 1962.
45.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Learning representations by back-propagating errors, Nature, 323 (1986), pp. 533–536.
46.
D. Slepian, Prolate spheroidal wave functions, Fourier analysis and uncertainty—IV: Extensions to many dimensions; generalized prolate spheroidal functions, Bell Syst. Tech. J., 43 (1964), pp. 3009–3057.
47.
D. Slepian, Prolate spheroidal wave functions, Fourier analysis, and uncertainty—V: The discrete case, Bell Syst. Tech. J., 57 (1978), pp. 1371–1430.
48.
D. Slepian and H. O. Pollak, Prolate spheroidal wave functions, Fourier analysis and uncertainty—I, Bell Syst. Tech. J., 40 (1961), pp. 43–63.
49.
V. Vapnik, The Nature of Statistical Learning Theory, Springer Science & Business Media, 1999.
50.
L.-L. Wang, Analysis of spectral approximations using prolate spheroidal wave functions, Math. Comp., 79 (2010), pp. 807–827.
51.
Y. Yin, W. Yin, P. Meng, and H. Liu, On a hybrid approach for recovering multiple obstacles, Commun. Comput. Phys., 31 (2022), pp. 869–892.
52.
J. Zhang, H. Li, L.-L. Wang, and Z. Zhang, Ball prolate spheroidal wave functions in arbitrary dimensions, Appl. Comput. Harmon. Anal., 48 (2020), pp. 539–569.

Information & Authors

Information

Published In

cover image SIAM Journal on Numerical Analysis
SIAM Journal on Numerical Analysis
Pages: 1443 - 1464
ISSN (online): 1095-7170

History

Submitted: 29 August 2023
Accepted: 1 February 2024
Published online: 19 June 2024

Keywords

  1. kernel machine
  2. neural network
  3. inverse problem
  4. target identification
  5. stability
  6. prolate spheroidal wave functions

MSC codes

  1. 78A46
  2. 65R32
  3. 65N21
  4. 33C47
  5. 33E30
  6. 41A30
  7. 65N35

Authors

Affiliations

Department of Mathematics, Virginia Tech, Blacksburg, VA 24060 USA. Part of this work was conducted while this author was affiliated with the Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, 100190, China.
Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, 100190, China, and School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing, 100049, China.

Funding Information

National Key R&D Program of China: 2018YFA0702502
Funding: The work of the authors was supported by the National Key R&D Program of China grant 2018YFA0702502.

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

There are no citations for this item

View Options

View options

PDF

View PDF

Full Text

View Full Text

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media