Abstract.

This paper develops a new empirical Bayesian inference algorithm for solving a linear inverse problem given multiple measurement vectors of noisy observable data. Specifically, by exploiting the joint sparsity across the multiple measurements in the sparse domain of the underlying signal or image, we construct a new support informed prior. Several applications can be modeled using this framework, including synthetic aperture radar observations using nearby azimuth angles and parallel magnetic resonance imaging. Our numerical experiments suggest that using the support informed prior usually improves accuracy of the recovery in the form of the sampled posterior mean and reduces its uncertainty when compared to posteriors constructed using some more standard priors.

Keywords

  1. empirical Bayesian inference
  2. uncertainty quantification
  3. sparsity promoting priors
  4. multiple measurement vectors
  5. joint sparsity

MSC codes

  1. 62C12
  2. 65C40
  3. 68U10

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
B. Adcock, A. Gelb, G. Song, and Y. Sui, Joint sparse recovery based on variances, SIAM J. Sci. Comput., 41 (2019), pp. A246–A268, https://doi.org/10.1137/17M1155983.
2.
R. Archibald and A. Gelb, Reducing the effects of noise in image reconstruction, J. Sci. Comput., 17 (2002), pp. 167–180.
3.
R. Archibald, A. Gelb, and R. Platte, Image reconstruction from undersampled Fourier data using the polynomial annihilation transform, J. Sci. Comput., 67 (2016), pp. 432–452, https://doi.org/10.1007/s10915-015-0088-2.
4.
R. Archibald, A. Gelb, and J. Yoon, Polynomial fitting for edge detection in irregularly sampled signals and images, SIAM J. Numer. Anal., 43 (2005), pp. 259–279, https://doi.org/10.1137/S0036142903435259.
5.
J. M. Bardsley, MCMC-based image reconstruction with uncertainty quantification, SIAM J. Sci. Comput., 34 (2012), pp. A1316–A1332, https://doi.org/10.1137/11085760X.
6.
A. Bhadra, J. Datta, N. G. Polson, and B. Willard, Lasso meets horseshoe: A survey, Statist. Sci., 34 (2019), pp. 405–427.
7.
D. Calvetti, M. Pragliola, and E. Somersalo, Sparsity Promoting Hybrid Solvers for Hierarchical Bayesian Inverse Problems, preprint, https://arxiv.org/abs/2003.06532, 2020.
8.
D. Calvetti, M. Pragliola, and E. Somersalo, Sparsity promoting hybrid solvers for hierarchical Bayesian inverse problems, SIAM J. Sci. Comput., 42 (2020), pp. A3761–A3784, https://doi.org/10.1137/20M1326246.
9.
D. Calvetti, M. Pragliola, E. Somersalo, and A. Strang, Sparse reconstructions from few noisy data: Analysis of hierarchical Bayesian models with generalized gamma hyperpriors, Inverse Problems, 36 (2020), 025010.
10.
D. Calvetti, E. Somersalo, and A. Strang, Hierarchical Bayesian models and sparsity: \(\ell_2\) -magic, Inverse Problems, 35 (2019), 035003.
11.
E. J. Candes, J. Romberg, and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory, 52 (2006), pp. 489–509.
12.
E. J. Candes, M. B. Wakin, and S. P. Boyd, Enhancing sparsity by reweighted \(\ell _1\) minimization, J. Fourier Anal. Appl., 14 (2008), pp. 877–905.
13.
E. J. Candès, J. K. Romberg, and T. Tao, Stable signal recovery from incomplete and inaccurate measurements, Commun. Pure Appl. Math., 59 (2006), pp. 1207–1223, https://doi.org/10.1002/cpa, https://onlinelibrary.wiley.com/doi/abs/10.1002/cpa.
14.
J. Canny, A computational approach to edge detection, IEEE Trans. Pattern Anal. Mach. Intell., 8 (1986), pp. 679–698.
15.
G. Casella and R. Berger, Statistical Inference, Duxbury Advanced Series, Duxbury Thomson Learning, 2002, https://books.google.com/books?id=ZpkPPwAACAAJ.
16.
V. Churchill, R. Archibald, and A. Gelb, Edge-Adaptive Regularization Image Reconstruction from Non-uniform Fourier Data, preprint, https://arxiv.org/abs/1811.08487, 2018.
17.
V. Churchill and A. Gelb, Detecting edges from non-uniform Fourier data via sparse Bayesian learning, J. Sci. Comput., 80(2019), pp. 762–783.
18.
I. Daubechies, R. DeVore, M. Fornasier, and C. S. Güntürk, Iteratively reweighted least squares minimization for sparse recovery, Commun. Pure Appl. Math., 63 (2010), pp. 1–38.
19.
A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum likelihood from incomplete data via the em algorithm, J. R. Stat. Soc. Series B (Methodol.), 39 (1977), pp. 1–38.
20.
D. L. Donoho, Compressed sensing, IEEE Trans. Inform. Theory, 52 (2006), pp. 1289–1306.
21.
J. Ellis, A practical guide to MCMC part: MCMC basics, https://jellis18.github.io/post/2018-01-02-mcmc-part1/, 2018 (accessed 2020-04-29).
22.
D. Gamerman and H. F. Lopes, Markov Chain Monte Carlo, Chapman & Hall/CRC, 2006.
23.
A. Gelb and T. Scarnati, Reducing effects of bad data using variance based joint sparsity recovery, J. Sci. Comput., 78 (2019), pp. 94–120.
24.
A. Gelb and E. Tadmor, Detection of edges in spectral data, Appl. Comput. Harmon. Anal., 7 (1999), pp. 101–135.
25.
A. Gelb and E. Tadmor, Adaptive edge detectors for piecewise smooth data based on the minmod limiter, J. Sci. Comput., 28 (2006), pp. 279–306.
26.
W. K. Hastings, Monte Carlo sampling methods using Markov chains and their applications, Biometrika, 57 (1970), pp. 97–109, https://doi.org/10.1093/biomet/57.1.97.
27.
J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Appl. Math. Sci., Springer, New York, 2006, https://books.google.com/books?id=h0i-Gi4rCZIC.
28.
N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, Equation of state calculations by fast computing machines, J. Chem. Phys., 21 (1953), pp. 1087–1092, https://doi.org/10.1063/1.1699114, https://doi.org/10.1063/1.1699114.
29.
K. Murphy, Machine Learning: A Probabilistic Perspective, Adaptive Computation and Machine Learning Series, MIT Press, 2012, https://books.google.com/books?id=NZP6AQAAQBAJ.
30.
J. Piironen and A. Vehtari, Sparsity information and regularization in the horseshoe and other shrinkage priors, Electron. J. Stat., 11 (2017), pp. 5018–5051.
31.
C. Price, Autocorrelation function (ACF), https://www.mathworks.com/matlabcentral/fileexchange/30540-autocorrelation-function-acf, 2020 (accessed October 2020).
32.
C. Robert and G. Casella, Introducing Monte Carlo Methods with R, Use R!, Springer, 2010, https://books.google.com/books?id=WIjMyiEiHCsC.
33.
T. Sanders, A. Gelb, and R. Platte, Composite SAR imaging using sequential joint sparsity, J. Comput. Phys., 338 (2017), https://doi.org/10.1016/j.jcp.2017.02.071.
34.
T. Sanders, R. B. Platte, and R. D. Skeel, Effective new methods for automated parameter selection in regularized inverse problems, Appl. Numer. Math., 152 (2020), pp. 29–48.
35.
T. Scarnati and A. Gelb, Joint image formation and two-dimensional autofocusing for synthetic aperture radar data, J. Comput. Phys., 374 (2018), pp. 803–821.
36.
T. Scarnati and A. Gelb, Accurate and efficient image reconstruction from multiple measurements of Fourier samples, J. Comput. Math., 88 (2020), pp. 798–828.
37.
I. Sobel and G. Feldman, A \(3 \times 3\) Isotropic Gradient Operator for Image Processing, a talk at the Stanford Artificial Project, 1968, pp. 271–272.
38.
M. E. Tipping, Sparse Bayesian learning and the relevance vector machine, J. Mach. Learn. Res., 1 (2001), pp. 211–244.
39.
D. P. Wipf and B. D. Rao, Sparse Bayesian learning for basis selection, IEEE Trans. Signal Proces., 52 (2004), pp. 2153–2164.

Information & Authors

Information

Published In

cover image SIAM/ASA Journal on Uncertainty Quantification
SIAM/ASA Journal on Uncertainty Quantification
Pages: 745 - 774
ISSN (online): 2166-2525

History

Submitted: 29 March 2021
Accepted: 22 February 2022
Published online: 29 June 2022

Keywords

  1. empirical Bayesian inference
  2. uncertainty quantification
  3. sparsity promoting priors
  4. multiple measurement vectors
  5. joint sparsity

MSC codes

  1. 62C12
  2. 65C40
  3. 68U10

Authors

Affiliations

Jiahui Zhang
Department of Mathematics, University of Minnesota, Minneapolis, MN 55455 USA ([email protected]).
Department of Mathematics, Dartmouth College, Hanover, NH 03755 USA ([email protected]).
Theresa Scarnati
Qualis Corporation, Huntsville, AL 35806 USA ([email protected]).

Funding Information

National Science Foundation (NSF): DMS-1502640, DMS-1912685
The work of the second author was partially supported by NSF grants DMS-1502640 and DMS-1912685, AFOSR grant FA9550-18-1-0316, and ONR MURI grant N00014-20-1-2595. The work of the third author was partially supported by AFOSR LRIR 18RYCOR011.

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media

The SIAM Publications Library now uses SIAM Single Sign-On for individuals. If you do not have existing SIAM credentials, create your SIAM account https://my.siam.org.