Two Metropolis--Hastings Algorithms for Posterior Measures with Non-Gaussian Priors in Infinite Dimensions

We introduce two classes of Metropolis--Hastings algorithms for sampling target measures that are absolutely continuous with respect to non-Gaussian prior measures on infinite-dimensional Hilbert spaces. In particular, we focus on certain classes of prior measures for which prior-reversible proposal kernels of the autoregressive type can be designed. We then use these proposal kernels to design algorithms that satisfy detailed balance with respect to the target measures. Afterwards, we introduce a new class of prior measures, called the Bessel-K priors, as a generalization of the gamma distribution to measures in infinite dimensions. The Bessel-K priors interpolate between well-known priors such as the gamma distribution and Besov priors and can model sparse or compressible parameters. We present concrete instances of our algorithms for the Bessel-K priors in the context of numerical examples in density estimation, finite-dimensional denoising, and deconvolution on the circle.

  • 1.  Rice wavelet toolbox (RWT) version 3.0. https://github.com/ricedsp/rwt.Google Scholar

  • 2.  O. E. Barndorff-Nielsen J. Pedersen K.-I. Sato and  Multivariate , self-decomposability and stability , Adv. Appl. Probab. , 33 ( 2001 ), pp. 160 -- 187 . CrossrefISIGoogle Scholar

  • 3.  O. E. Barndorff-Nielsen and  S. Thorbjørnsen , Self-decomposability and Lévy processes in free probability , Bernoulli , 8 ( 2002 ), pp. 323 -- 366 . ISIGoogle Scholar

  • 4.  A. Beskos M. Girolami S. Lan P. E. Farrell and  A. M. Stuart , Geometric MCMC for infinite-dimensional inverse problems , J. Comput. Phys. , 335 ( 2017 ), pp. 327 -- 351 . CrossrefISIGoogle Scholar

  • 5.  A. Bhattacharya D. Pati N. S. Pillai and  D. B. Dunson , Dirichlet--Laplace priors for optimal shrinkage , J. Amer. Statist. Assoc. , 110 ( 2015 ), pp. 1479 -- 1490 . CrossrefISIGoogle Scholar

  • 6.  V. I. Bogachev, Gaussian Measures, Math. Surveys Monogr. 62, American Mathematical Society, Providence, RI, 1998.Google Scholar

  • 7.  V. I. Bogachev, Measure Theory, Vol. 1, Springer, New York, 2007.Google Scholar

  • 8.  D. Calvetti and E. Somersalo, An introduction to Bayesian scientific computing: Ten Lectures on Subjective Computing, Surveys Tutor. Appl. Math. Sci., Vol. 2, Springer, New York, 2007.Google Scholar

  • 9.  V. Chen, M. M. Dunlop, O. Papaspiliopoulos, and A. M. Stuart, Robust MCMC Sampling with Non-Gaussian and Hierarchical Priors in High Dimensions, preprint, arXiv:1803.03344, 2018.Google Scholar

  • 10.  S. L. Cotter G. O. Roberts A. M. Stuart and  D. White , MCMC methods for functions: Modifying old algorithms to make them faster , Statist. Sci. , 28 ( 2013 ), pp. 424 -- 446 . CrossrefISIGoogle Scholar

  • 11.  T. Cui K. J. Law and  Y. M. Marzouk , Dimension-independent likelihood-informed MCMC , J. Comput. Phys. , 304 ( 2016 ), pp. 109 -- 137 . CrossrefISIGoogle Scholar

  • 12.  M. Dashti S. Harris and  A. M. Stuart , Besov priors for Bayesian inverse problems , Inverse Probl. Imaging , 6 ( 2012 ), pp. 183 -- 200 . CrossrefISIGoogle Scholar

  • 13.  M. Dashti and  A. M. Stuart , The Bayesian approach to inverse problems, in Handbook of Uncertainty Quantification, R. Ghanem, D. Higdon, and H. Owhadi, eds., Springer, Cham , Switzerland , 2016 , pp. 1 -- 118 , https://doi.org/10.1007/978-3-319-11259-6_7-1. Google Scholar

  • 14.  B. Hosseini Bayesian inverse problems with infinitely divisible and heavy-tailed prior measures , SIAM/ASA J. Uncertain. Quantif. , 5 ( 2017 ), pp. 1024 -- 1060 , https://doi.org/10.1137/16M1096372. LinkISIGoogle Scholar

  • 15.  B. Hosseini and  N. Nigam , Well-posed Bayesian inverse problems: Priors with exponential tails , SIAM/ASA J. Uncertain. Quantif. , 5 ( 2017 ), pp. 436 -- 465 , https://doi.org/10.1137/16M1076824. LinkISIGoogle Scholar

  • 16.  N. L. Johnson, S. Kotz, and N. Balakrishnan, Continuous Univariate Distributions, Vol. 1, Models and Applications, 2nd ed., Wiley, New York, 2002.Google Scholar

  • 17.  Z. J. Jurek , Different aspects of self-decomposability, in Lévy Processes: Theory and Applications, O. E. Barndorff-Nielsen, ed., Birkhäuser , Boston , 2001 , pp. 367 -- 377 . Google Scholar

  • 18.  Z. J. Jurek : An exception or a rule, Ann. Univ. Maria Curie-Sklodowska , Sect. A , 10 ( 1997 ), pp. 93 -- 107 . Google Scholar

  • 19.  Z. J. Jurek and  W. Vervaat , An integral representation for self-decomposable Banach space valued random variables , Probab. Theory Related Fields , 62 ( 1983 ), pp. 247 -- 262 . CrossrefGoogle Scholar

  • 20.  J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Appl. Math. Sci. 160, Springer, New York, 2005.Google Scholar

  • 21.  S. Kotz, T. J. Kozubowski, and K. Podgorski, The Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance, Springer, New York, 2012.Google Scholar

  • 22.  A. Kumar and  B. Schreiber , Self-decomposable probability measures on Banach spaces , Studia Math. , 53 ( 1975 ), pp. 55 -- 71 . CrossrefISIGoogle Scholar

  • 23.  M. Lassas E. Saksman and  S. Siltanen , Discretization-invariant Bayesian inversion and Besov space priors , Inverse Probl. Imaging , 3 ( 2009 ), pp. 87 -- 122 . CrossrefISIGoogle Scholar

  • 24.  A. Lawrance , The innovation distribution of a gamma distributed autoregressive process , Scand. J. Stat., ( 1982 ), pp. 234 -- 236 . Google Scholar

  • 25.  P. A. Lewis E. McKenzie and  D. K. Hugus , Gamma processes , Stoch. Models , 5 ( 1989 ), pp. 1 -- 30 . CrossrefGoogle Scholar

  • 26.  F. Lucka and  Fast Markov Monte Carlo sampling for sparse Bayesian inference in high-dimensional inverse problems using L1-type priors , Inverse Problems , 28 ( 2012 ), 125012 . CrossrefISIGoogle Scholar

  • 27.  F. Lucka Gibbs sampling for high-dimensional Bayesian inversion , Inverse Problems , 32 ( 2016 ), 115019 . CrossrefISIGoogle Scholar

  • 28.  A. M. Mathai , On noncentral generalized Laplacianness of quadratic forms in normal variables , J. Multivariate Anal. , 45 ( 1993 ), pp. 239 -- 246 . CrossrefISIGoogle Scholar

  • 29.  D. F. Nicholls and B. G. Quinn, Random Coefficient Autoregressive Models: An Introduction, Lect. Notes Stat. 11, Springer, New York, 2012.Google Scholar

  • 30.  R. A. Norton and C. Fox, Tuning of MCMC with Langevin, Hamiltonian, and Other Stochastic Autoregressive Proposals, preprint. arXiv:1610.00781, 2016.Google Scholar

  • 31.  M. Ottobre N. S. Pillai F. J. Pinski and  A. . Stuart, A function space HMC algorithm with second order Langevin diffusion limit , Bernoulli , 22 ( 2016 ), pp. 60 -- 106 . CrossrefISIGoogle Scholar

  • 32.  K. Pearson G. Jeffery and  E. M. Elderton , On the distribution of the first product moment-coefficient, in samples drawn from an indefinitely large normal population , Biometrika , 21 ( 1929 ), pp. 164 -- 201 . CrossrefGoogle Scholar

  • 33.  F. W. Steutel and K. Van Harn, Infinite divisibility of probability distributions on the real line, Pure Appl. Math., Marcel Dekker, New York, 2003.Google Scholar

  • 34.  A. M. Stuart and  Inverse : A Bayesian perspective , Acta Numer. , 19 ( 2010 ), pp. 451 -- 559 . CrossrefGoogle Scholar

  • 35.  T. Sullivan Bayesian inverse problems and heavy-tailed stable quasi-Banach space priors , Inverse Probl. Imaging , 11 ( 2017 ), pp. 857 -- 874 . CrossrefISIGoogle Scholar

  • 36.  D. H. Thang , Random mappings on infinite dimensional spaces , Stochastics , 64 ( 1998 ), pp. 51 -- 73 . Google Scholar

  • 37.  L. Tierney , A note on Metropolis-Hastings kernels for general state spaces , Ann. Appl. Probab. , 8 ( 1998 ), pp. 1 -- 9 . CrossrefISIGoogle Scholar

  • 38.  K. Urbanik , Self-decomposable probability distributions on ${\reals}^{m}$ , Appl. Math. , 10 ( 1969 ), pp. 91 -- 97 . Google Scholar

  • 39.  C. R. Vogel, Computational Methods for Inverse Problems, Front. Appl. Math. 23, SIAM, Philadelphia, 2002.Google Scholar

  • 40.  S. G. Walker , A note on the innovation distribution of a gamma distributed autoregressive process , Scand. J. Stat. , 27 ( 2000 ), pp. 575 -- 576 . CrossrefISIGoogle Scholar

  • 41.  Z. Wang J. M. Bardsley A. Solonen T. Cui and  Y. M. Marzouk , Bayesian inverse problems with $l\_1$ priors: A randomize-then-optimize approach , SIAM J. Sci. Comput. , 39 ( 2017 ), pp. S140 -- S166 . LinkISIGoogle Scholar

  • 42.  G. Weiss , Time-reversibility of linear stochastic processes , J. Appl. Probab. , 12 ( 1975 ), pp. 831 -- 836 . CrossrefISIGoogle Scholar