Generalized Sparse Bayesian Learning and Application to Image Reconstruction
Abstract.
Image reconstruction based on indirect, noisy, or incomplete data remains an important yet challenging task. While methods such as compressive sensing have demonstrated high-resolution image recovery in various settings, there remain issues of robustness due to parameter tuning. Moreover, since the recovery is limited to a point estimate, it is impossible to quantify the uncertainty, which is often desirable. Due to these inherent limitations, a sparse Bayesian learning approach is sometimes adopted to recover a posterior distribution of the unknown. Sparse Bayesian learning assumes that some linear transformation of the unknown is sparse. However, most of the methods developed are tailored to specific problems, with particular forward models and priors. Here, we present a generalized approach to sparse Bayesian learning. It has the advantage that it can be used for various types of data acquisitions and prior information. Some preliminary results on image reconstruction/recovery indicate its potential use for denoising, deblurring, and magnetic resonance imaging.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgment.
The authors thank the anonymous reviewers for their helpful comments on an earlier draft of the manuscript.
References
1.
B. Adcock, A. Gelb, G. Song, and Y. Sui, Joint sparse recovery based on variances, SIAM J. Sci. Comput., 41 (2019), pp. A246–A268, https://doi.org/10.1137/17M1155983.
2.
R. Archibald, A. Gelb, and R. B. Platte, Image reconstruction from undersampled Fourier data using the polynomial annihilation transform, J. Sci. Comput., 67 (2016), pp. 432–452.
3.
E. Artin, The Gamma Function, Dover Publications, Mineola, NY, 2015.
4.
S. D. Babacan, R. Molina, and A. K. Katsaggelos, Parameter estimation in TV image restoration using variational distribution approximation, IEEE Trans. Image Process., 17 (2008), pp. 326–339.
5.
S. D. Babacan, R. Molina, and A. K. Katsaggelos, Sparse Bayesian image restoration, in Proceedings of the 2010 IEEE International Conference on Image Processing, IEEE, 2010, pp. 3577–3580.
6.
J. M. Bardsley, MCMC-based image reconstruction with uncertainty quantification, SIAM J. Sci. Comput., 34 (2012), pp. A1316–A1332, https://doi.org/10.1137/11085760X.
7.
J. M. Bardsley, D. Calvetti, and E. Somersalo, Hierarchical regularization for edge-preserving reconstruction of PET images, Inverse Problems, 26 (2010), 035010.
8.
S. Boyd, N. Parikh, and E. Chu, Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers, Now Publishers, Delft, The Netherlands, 2011.
9.
D. Calvetti, A. Pascarella, F. Pitolli, E. Somersalo, and B. Vantaggi, A hierarchical Krylov-Bayes iterative inverse solver for MEG with physiological preconditioning, Inverse Problems, 31 (2015), 125005.
10.
D. Calvetti, M. Pragliola, and E. Somersalo, Sparsity promoting hybrid solvers for hierarchical Bayesian inverse problems, SIAM J. Sci. Comput., 42 (2020), pp. A3761–A3784, https://doi.org/10.1137/20M1326246.
11.
D. Calvetti, M. Pragliola, E. Somersalo, and A. Strang, Sparse reconstructions from few noisy data: Analysis of hierarchical Bayesian models with generalized gamma hyperpriors, Inverse Problems, 36 (2020), 025010.
12.
D. Calvetti and E. Somersalo, A Gaussian hypermodel to recover blocky objects, Inverse Problems, 23 (2007), 733.
13.
D. Calvetti and E. Somersalo, An Introduction to Bayesian Scientific Computing: Ten Lectures on Subjective Computing, Vol. 2, Springer, New York, 2007.
14.
D. Calvetti and E. Somersalo, Hypermodels in the Bayesian imaging framework, Inverse Problems, 24 (2008), 034013.
15.
D. Calvetti and E. Somersalo, Subjective knowledge or objective belief? An oblique look to Bayesian methods, Large-Scale Inverse Problems and Quantification of Uncertainty, Wiley Ser. Comput. Stat., Wiley, Chichester, 2011, pp. 33–70.
16.
D. Calvetti, E. Somersalo, and A. Strang, Hierachical Bayesian models and sparsity: \(\ell_2\) -magic, Inverse Problems, 35 (2019), 035003.
17.
E. J. Candès, M. B. Wakin, and S. P. Boyd, Enhancing sparsity by reweighted \(\ell_1\) minimization, J. Fourier Anal. Appl., 14 (2008), pp. 877–905.
18.
G. Chantas, N. Galatsanos, A. Likas, and M. Saunders, Variational Bayesian image restoration based on a product of t-distributions image prior, IEEE Trans. Image Process., 17 (2008), pp. 1795–1805.
19.
G. K. Chantas, N. P. Galatsanos, and A. C. Likas, Bayesian restoration using a new nonstationary edge-preserving image prior, IEEE Trans. Image Process., 15 (2006), pp. 2987–2997.
20.
R. Chartrand and W. Yin, Iteratively reweighted algorithms for compressive sensing, in Proceedings of the 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, 2008, pp. 3869–3872.
21.
I. Y. Chun and B. Adcock, Compressed sensing and parallel acquisition, IEEE Trans. Inform. Theory, 63 (2017), pp. 4860–4882.
22.
V. Churchill and A. Gelb, Detecting edges from non-uniform Fourier data via sparse Bayesian learning, J. Sci. Comput., 80 (2019), pp. 762–783.
23.
V. Churchill and A. Gelb, Estimation and uncertainty quantification for piecewise smooth signal recovery, J. Comput. Math., 41 (2023), pp. 246–262.
24.
D. Colton, M. Piana, and R. Potthast, A simple method using Morozov’s discrepancy principle for solving inverse scattering problems, Inverse Problems, 13 (1997), pp. 1477–1493.
25.
I. Daubechies, R. DeVore, M. Fornasier, and C. S. Güntürk, Iteratively reweighted least squares minimization for sparse recovery, Comm. Pure Appl. Math., 63 (2010), pp. 1–38.
26.
D. L. Donoho, Compressed sensing, IEEE Trans. Inform. Theory, 52 (2006), pp. 1289–1306.
27.
Y. C. Eldar and G. Kutyniok, Compressed Sensing: Theory and Applications, Cambridge University Press, 2012.
28.
R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman, Removing camera shake from a single photograph, ACM Trans.Graph., 25 (2006), pp. 787–794.
29.
M. A. Figueiredo, J. M. Bioucas-Dias, and R. D. Nowak, Majorization-minimization algorithms for wavelet-based image restoration, IEEE Trans. Image Process., 16 (2007), pp. 2980–2991.
30.
D. Fink, A Compendium of Conjugate Priors, 1997, available online at https://www.johndcook.com/CompendiumOfConjugatePriors.pdf.
31.
S. Foucart and H. Rauhut, A mathematical introduction to compressive sensing, Bull. Amer. Math. Soc. (N.S.), 54 (2017), pp. 151–165.
32.
J. Friedman, T. Hastie, and R. Tibshirani, Regularization paths for generalized linear models via coordinate descent, J. Stat. Softw., 33 (2010), pp. 1–22.
33.
A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin, Bayesian data analysis, 3rd ed., Chapman and Hall/CRC, Boca Raton, FL, 2021, http://www.stat.columbia.edu/∼gelman/book/.
34.
G. H. Golub, M. Heath, and G. Wahba, Generalized cross-validation as a method for choosing a good ridge parameter, Technometrics, 21 (1979), pp. 215–223.
35.
C. W. Groetsch and C. Groetsch, Inverse Problems in the Mathematical Sciences, Vol. 52, Vieweg+Teubner Verlag, Wiesbaden, 1993.
36.
M. Guerquin-Kern, L. Lejeune, K. P. Pruessmann, and M. Unser, Realistic analytical phantoms for parallel magnetic resonance imaging, IEEE Trans. Med. Imaging, 31 (2011), pp. 626–636.
37.
F. Gustafsson, Statistical Sensor Fusion, Studentlitteratur, 2010.
38.
P. C. Hansen, The L-curve and its use in the numerical treatment of inverse problems, Computational Inverse Problems in Electrocardiology, WT Press, 1999, pp. 119–142.
39.
P. C. Hansen, Discrete Inverse Problems: Insight and Algorithms, Fundam. Algorithms 7, SIAM, Philadelphia, 2010, https://doi.org/10.1137/1.9780898718836.
40.
P. C. Hansen, J. G. Nagy, and D. P. O’leary, Deblurring Images: Matrices, Spectra, and Filterings, Fundam. Algorithms 3, SIAM, Philadelphia, 2006, https://doi.org/10.1137/1.9780898718874.
41.
J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Appl. Math. Sci. 160, Springer-Verlag, New York, 2006.
42.
J. P. Kaipio, V. Kolehmainen, E. Somersalo, and M. Vauhkonen, Statistical inversion and Monte Carlo sampling methods in electrical impedance tomography, Inverse Problems, 16 (2000), 1487.
43.
D. Krishnan and R. Fergus, Fast image deconvolution using hyper-Laplacian priors, in Proceedings of the 22nd International Conference on Neural Information Processing Systems, 2009, pp. 1033–1041.
44.
A. Lanza, M. Pragliola, and F. Sgallari, Residual whiteness principle for parameter-free image restoration, Electron. Trans. Numer. Anal., 53 (2020), pp. 329–351.
45.
A. Levin, R. Fergus, F. Durand, and W. T. Freeman, Image and depth from a conventional camera with a coded aperture, ACM Trans. Graph., 26 (2007), pp. 70–es.
46.
D. J. MacKay, Bayesian interpolation, Neural Comput., 4 (1992), pp. 415–447.
47.
K. P. Murphy, Conjugate Bayesian Analysis of the Gaussian Distribution, 2007, https://www.cs.ubc.ca/∼murphyk/Papers/bayesGauss.pdf.
48.
F. Natterer and F. Wübbeling, Mathematical Methods in Image Reconstruction, SIAM Monogr. Math. Model. Comput., SIAM, Philadelphia, 2001, https://doi.org/10.1137/1.9780898718324.
49.
Y. Saad, Iterative Methods for Sparse Linear Systems, SIAM, Philadelphia, 2003, https://doi.org/10.1137/1.9780898718003.
50.
T. Sanders, R. B. Platte, and R. D. Skeel, Effective new methods for automated parameter selection in regularized inverse problems, Appl. Numer. Math., 152 (2020), pp. 29–48.
51.
H. Stark, Image Recovery: Theory and Application, Elsevier, 2013.
52.
W. Stefan, R. A. Renaut, and A. Gelb, Improved total variation-type regularization using higher order edge detectors, SIAM J. Imaging Sci., 3 (2010), pp. 232–251, https://doi.org/10.1137/080730251.
53.
A. N. Tikhonov, A. Goncharsky, V. Stepanov, and A. G. Yagola, Numerical Methods for the Solution of Ill-Posed Problems, Math. Appl. 328, Kluwer Academic Publishers, Dordrecht, 1995.
54.
M. E. Tipping, Sparse Bayesian learning and the relevance vector machine, J. Mach. Learn. Res., 1 (2001), pp. 211–244.
55.
C. R. Vogel, Computational Methods for Inverse Problems, Frontiers Appl. Math. 23, SIAM, Philadelphia, 2002, https://doi.org/10.1137/1.9780898717570.
56.
D. Wipf and S. Nagarajan, A new view of automatic relevance determination, in Proceedings of the 20th International Conference on Neural Information Processing Systems, 2007, pp. 1625–1632.
57.
D. P. Wipf and B. D. Rao, Sparse Bayesian learning for basis selection, IEEE Trans. Signal Process., 52 (2004), pp. 2153–2164.
58.
S. J. Wright, Coordinate descent algorithms, Math. Program., 151 (2015), pp. 3–34.
59.
Z. Zhang, T.-P. Jung, S. Makeig, Z. Pi, and B. D. Rao, Spatiotemporal sparse Bayesian learning with applications to compressed sensing of multichannel physiological signals, IEEE Trans. Neural Syst. Rehabil. Eng., 22 (2014), pp. 1186–1197.
60.
Z. Zhang and B. D. Rao, Clarify Some Issues on the Sparse Bayesian Learning for Sparse Signal Recovery, Tech. report, University of California, San Diego, San Diego, CA, 2011.
61.
Z. Zhang and B. D. Rao, Sparse signal recovery with temporally correlated source vectors using sparse Bayesian learning, IEEE J. Sel. Top. Signal Process., 5 (2011), pp. 912–926.
Information & Authors
Information
Published In
SIAM/ASA Journal on Uncertainty Quantification
Pages: 262 - 284
DOI: 10.1137/22M147236X
ISSN (online): 2166-2525
Copyright
© 2023 Society for Industrial and Applied Mathematics and American Statistical Association.
History
Submitted: 18 January 2022
Accepted: 1 August 2022
Published online: 1 March 2023
Keywords
MSC codes
Authors
Funding Information
Air Force Office of Scientific Research (AFOSR): F9550-18-1-0316
Office of Naval Research (ONR): N00014-20-1-2595
Division of Mathematical Sciences (DMS): DMS-1502640, DMS-1912685
Division of Mathematical Sciences (DMS): DMS-1521661, DMS-1939203
Funding: The work of the first and second authors was partially supported by AFOSR grant F9550-18-1-0316. The work of the second author was partially supported by NSF grants DMS-1502640 and DMS-1912685 and ONR grant N00014-20-1-2595. The work of the third author was partially supported by NSF grants DMS-1521661 and DMS-1939203.
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Complex-Valued Signal Recovery Using a Generalized Bayesian LASSOSIAM/ASA Journal on Uncertainty Quantification, Vol. 13, No. 2 | 6 June 2025
- Image Denoising Based on Deep Image Prior Combined Sparsity with Regularization by DenoisingCircuits, Systems, and Signal Processing, Vol. 12 | 5 April 2025
- A Bayesian Framework for Spectral ReprojectionJournal of Scientific Computing, Vol. 102, No. 3 | 31 January 2025
- Efficient sparsity-promoting MAP estimation for Bayesian linear inverse problemsInverse Problems, Vol. 41, No. 2 | 13 January 2025
- Image restoration for both deblurring and dehazing based on multi-channel frequency information using deep neural networkThe Visual Computer, Vol. 23 | 19 January 2025
- SBL-LCGL: sparse Bayesian learning based on Laplace distribution for robust cone-beam x-ray luminescence computed tomographyPhysics in Medicine & Biology, Vol. 69, No. 17 | 30 August 2024
- Leveraging Joint Sparsity in Hierarchical Bayesian LearningSIAM/ASA Journal on Uncertainty Quantification, Vol. 12, No. 2 | 24 May 2024
- Computationally Efficient Sampling Methods for Sparsity Promoting Hierarchical Bayesian ModelsSIAM/ASA Journal on Uncertainty Quantification, Vol. 12, No. 2 | 7 June 2024
- Adaptive anisotropic Bayesian meshing for inverse problemsInverse Problems, Vol. 40, No. 4 | 23 February 2024
- Application of Virtualization Digital Technology in Intelligent Platform Construction Pattern Analysis SystemProcedia Computer Science, Vol. 243 | 1 Jan 2024
- Exploring the potential of channel interactions for image restorationKnowledge-Based Systems, Vol. 282 | 1 Dec 2023
- Sequential Image Recovery Using Joint Hierarchical Bayesian LearningJournal of Scientific Computing, Vol. 96, No. 1 | 18 May 2023
- UnCRtainTS: Uncertainty Quantification for Cloud Removal in Optical Satellite Time Series2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) | 1 Jun 2023
- Sub-aperture SAR imaging with uncertainty quantificationInverse Problems, Vol. 39, No. 5 | 31 March 2023
- Leveraging joint sparsity in 3D synthetic aperture radar imagingApplied Mathematics for Modern Challenges, Vol. 1, No. <![CDATA[1]]> | 1 Jan 2023
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].