High-Dimensional Gaussian Sampling: A Review and a Unifying Approach Based on a Stochastic Proximal Point Algorithm

Abstract

Efficient sampling from a high-dimensional Gaussian distribution is an old but high-stakes issue. Vanilla Cholesky samplers imply a computational cost and memory requirements that can rapidly become prohibitive in high dimensions. To tackle these issues, multiple methods have been proposed from different communities ranging from iterative numerical linear algebra to Markov chain Monte Carlo (MCMC) approaches. Surprisingly, no complete review and comparison of these methods has been conducted. This paper aims to review all these approaches by pointing out their differences, close relations, benefits, and limitations. In addition to reviewing the state of the art, this paper proposes a unifying Gaussian simulation framework by deriving a stochastic counterpart of the celebrated proximal point algorithm in optimization. This framework offers a novel and unifying revisiting of most of the existing MCMC approaches while also extending them. Guidelines to choosing the appropriate Gaussian simulation method for a given sampling problem in high dimensions are proposed and illustrated with numerical examples.

1. S. L. Adler , Over-relaxation method for the Monte Carlo evaluation of the partition function for multiquadratic actions , Phys. Rev. D , 23 ( 1981 ), pp. 2901 -- 2904 . Crossref ISI Google Scholar
2. M. V. Afonso, J. M. Bioucas-Dias and M. A. . Figueiredo, Fast image recovery using variable splitting and constrained optimization , IEEE Trans. Image Process. , 19 ( 2010 ), pp. 2345 -- 2356 . Crossref ISI Google Scholar
3. M. V. Afonso, J. M. Bioucas-Dias and M. A. . Figueiredo, An augmented Lagrangian approach to the constrained optimization formulation of imaging inverse problems , IEEE Trans. Image Process. , 20 ( 2011 ), pp. 681 -- 695 . Crossref ISI Google Scholar
4. E. Allen, J. Baglama and S. Boyd , Numerical approximation of the product of the square root of a matrix with a vector , Linear Algebra Appl. , 310 ( 2000 ), pp. 167 -- 181 , https://doi.org/10.1016/S0024-3795(00)00068-9. Crossref ISI Google Scholar
5. Y. Altmann, S. McLaughlin and N. Dobigeon , Sampling from a multivariate Gaussian distribution truncated on a simplex: A review , in Proc. IEEE-SP Workshop Stat. and Signal Process. (SSP) , Gold Coast, Australia , 2014 , pp. 113 -- 116 , https://doi.org/10.1109/SSP.2014.6884588. Google Scholar
6. S. A. Armstrong, J. E. Staunton, L. B. Silverman, R. Pieters, M. L. den Boer, M. D. Minden, S. E. Sallan, E. S. Lander, T. R. Golub and S. J. Korsmeyer , MLL translocations specify a distinct gene expression profile that distinguishes a unique leukemia , Nat. Genet. , 30 ( 2002 ), pp. 41 -- 47 , https://doi.org/10.1038/ng765. Crossref ISI Google Scholar
7. E. Aune, J. Eidsvik and Y. Pokern , Iterative numerical methods for sampling from high dimensional Gaussian distributions , Stat. Comput. , 23 ( 2013 ), pp. 501 -- 521 , https://doi.org/10.1007/s11222-012-9326-8. Crossref ISI Google Scholar
8. O. Axelsson , Iterative Solution Methods , Cambridge University Press , 1994 , https://doi.org/10.1017/CBO9780511624100. Google Scholar
9. A.-C. Barbos, F. Caron, J.-F. Giovannelli and A. Doucet , Clone MCMC: Parallel high-dimensional Gaussian Gibbs sampling , in Adv. in Neural Information Process. Systems , 2017 , pp. 5020 -- 5028 . Google Scholar
10. P. Barone and A. Frigessi , Improving stochastic relaxation for Gaussian random fields , Probab. Eng. Inform. Sci. , 4 ( 1990 ), pp. 369 -- 389 . Crossref Google Scholar
11. H. H. Bauschke and P. L. Combettes , Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2 nd ed., Springer , 2017 . Google Scholar
12. J. Besag and C. Kooperberg , On conditional and intrinsic autoregressions , Biometrika , 82 ( 1995 ), pp. 733 -- 746 , https://doi.org/10.2307/2337341. ISI Google Scholar
13. C. M. Bishop , Pattern Recognition and Machine Learning , Springer-Verlag , 2006 . Google Scholar
14. G. E. P. Box and G. M. Jenkins , Time Series Analysis: Forecasting and Control , 3 rd ed., Prentice Hall , 1994 . Google Scholar
15. S. Boyd, N. Parikh, E. Chu, B. Peleato and J. Eckstein , Distributed optimization and statistical learning via the alternating direction method of multipliers , Found. Trends Mach. Learn. , 3 ( 2011 ), pp. 1 -- 122 . Crossref Google Scholar
16. S. Brahim-Belhouari and A. Bermak , Gaussian process for nonstationary time series prediction , Comput. Statist. Data Anal. , 47 ( 2004 ), pp. 705 -- 712 , https://doi.org/10.1016/j.csda.2004.02.006. Crossref ISI Google Scholar
17. K. Bredies and H. Sun , A proximal point analysis of the preconditioned alternating direction method of multipliers , J. Optim. Theory Appl. , 173 ( 2017 ), pp. 878 -- 907 , https://doi.org/10.1007/s10957-017-1112-5. Crossref ISI Google Scholar
18. B. Carpenter, A. Gelman, M. Hoffman, D. Lee, B. Goodrich, M. Betancourt, M. Brubaker, J. Guo, P. Li and A. Riddell , Stan: A probabilistic programming language , J. Stat. Softw. , 76 ( 2017 ), pp. 1 -- 32 , https://doi.org/10.18637/jss.v076.i01. Crossref ISI Google Scholar
19. A. Chambolle and T. Pock , A first-order primal-dual algorithm for convex problems with applications to imaging , J. Math. Imaging Vision , 40 ( 2011 ), pp. 120 -- 145 , https://doi.org/10.1007/s10851-010-0251-1. Crossref ISI Google Scholar
20. A.-L. Cholesky , Sur la résolution numérique des systèmes d'équations linéaires , Bull. Soc. Amis Bibl. École Polytech. , 39 ( 1910 ), pp. 81 -- 95 . Google Scholar
21. E. Chow and Y. Saad , Preconditioned Krylov subspace methods for sampling multivariate Gaussian distributions , SIAM J. Sci. Comput. , 36 ( 2014 ), pp. A588 -- A608 , https://doi.org/10.1137/130920587. Link ISI Google Scholar
22. E. S. Coakley and V. Rokhlin , A fast divide-and-conquer algorithm for computing the spectra of real symmetric tridiagonal matrices , Appl. Comput. Harmonic Anal. , 34 ( 2013 ), pp. 379 -- 414 , https://doi.org/10.1016/j.acha.2012.06.003. Crossref ISI Google Scholar
23. N. A. C. Cressie , Statistics for Spatial Data , 2 nd ed., Wiley , 1993 . Google Scholar
24. M. W. Davis , Generating large stochastic simulations---the matrix polynomial approximation method , Math. Geosci. , 19 ( 1987 ), pp. 99 -- 107 , https://doi.org/10.1007/BF00898190. Google Scholar
25. O. Demir-Kavuk and H. Riedesel . Knapp, Exploring classification strategies with the CoEPrA 2006 contest , Bioinformatics , 26 ( 2010 ), pp. 603 -- 609 , https://doi.org/10.1093/bioinformatics/btq021. Crossref ISI Google Scholar
26. C. R. Dietrich and G. N. Newsam , Fast and exact simulation of stationary Gaussian processes through circulant embedding of the covariance matrix , SIAM J. Sci. Comput. , 18 ( 1997 ), pp. 1088 -- 1107 , https://doi.org/10.1137/S1064827592240555. Link ISI Google Scholar
27. S. Duane, A. Kennedy, B. J. Pendleton and D. Roweth , Hybrid Monte Carlo , Phys. Lett. B , 195 ( 1987 ), pp. 216 -- 222 . Crossref ISI Google Scholar
28. E. Esser, X. Zhang and T. F. Chan , A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science , SIAM J. Imaging Sci. , 3 ( 2010 ), pp. 1015 -- 1046 , https://doi.org/10.1137/09076934X. Link ISI Google Scholar
29. L. Fahrmeir and S. Lang , Bayesian semiparametric regression analysis of multicategorical time-space data , Ann. Inst. Statist. Math. , 53 ( 2001 ), pp. 11 -- 30 , https://doi.org/10.1023/A:1017904118167. Crossref ISI Google Scholar
30. O. Féron, F. Orieux and J. F. Giovannelli , Gradient scan Gibbs sampler: An efficient algorithm for high-dimensional Gaussian distributions , IEEE J. Sel. Topics Signal Process. , 10 ( 2016 ), pp. 343 -- 352 , https://doi.org/10.1109/JSTSP.2015.2510961. Crossref Google Scholar
31. C. Fox and A. Parker , Convergence in variance of Chebyshev accelerated Gibbs samplers , SIAM J. Sci. Comput. , 36 ( 2014 ), pp. A124 -- A147 , https://doi.org/10.1137/120900940. Link ISI Google Scholar
32. C. Fox and A. Parker , Accelerated Gibbs sampling of normal distributions using matrix splittings and polynomials , Bernoulli , 23 ( 2017 ), pp. 3711 -- 3743 , https://doi.org/10.3150/16-BEJ863. Crossref ISI Google Scholar
33. D. Gabay and B. Mercier , A dual algorithm for the solution of nonlinear variational problems via finite element approximation , Comput. Math. Appl. , 2 ( 1976 ), pp. 17 -- 40 , https://doi.org/10.1016/0898-1221(76)90003-1. Crossref Google Scholar
34. B. Galerne, Y. Gousseau and J. Morel , Random phase textures: Theory and synthesis , IEEE Trans. Image Process. , 20 ( 2011 ), pp. 257 -- 267 , https://doi.org/10.1109/TIP.2010.2052822. Crossref ISI Google Scholar
35. A. Gelman, C. Robert, N. Chopin and J. Rousseau , Bayesian Data Analysis , Chapman & Hall , 1995 . Google Scholar
36. S. Geman and D. Geman , Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , IEEE Trans. Pattern Anal. Mach. Intell. , 6 ( 1984 ), pp. 721 -- 741 . Crossref ISI Google Scholar
37. C. Gilavert, S. Moussaoui and J. Idier , Efficient Gaussian sampling for solving large-scale inverse problems using MCMC , IEEE Trans. Signal Process. , 63 ( 2015 ), pp. 70 -- 80 , https://doi.org/10.1109/TSP.2014.2367457. Crossref ISI Google Scholar
38. J.-F. Giovannelli and J. Idier , Regularization and Bayesian Methods for Inverse Problems in Signal and Image Processing , 1 st ed., Wiley-IEEE Press , 2015 . Google Scholar
39. P. Giudici and P. J. Green , Decomposable graphical Gaussian model determination , Biometrika , 86 ( 1999 ), pp. 785 -- 801 , https://doi.org/10.1093/biomet/86.4.785. Crossref ISI Google Scholar
40. R. Glowinski and A. Marroco , Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , RAIRO Anal. Numer. , 9 ( 1975 ), pp. 41 -- 76 , http://www.numdam.org/item/M2AN_1975__9_2_41_0. , http://www.numdam.org/item/M2AN_1975__9_2_41_0. Google Scholar
41. G. H. Golub and C. F. Van Loan , Matrix Computations , 2 nd ed., The Johns Hopkins University Press , 1989 . Google Scholar
42. J. Goodman and A. D. Sokal , Multigrid Monte Carlo method. Conceptual foundations , Phys. Rev. D , 40 ( 1989 ), pp. 2035 -- 2071 , https://doi.org/10.1103/PhysRevD.40.2035. Crossref ISI Google Scholar
43. P. J. Green and Reversible Monte Carlo computation and Bayesian model determination , Biometrika , 82 ( 1995 ), pp. 711 -- 732 , https://doi.org/10.1093/biomet/82.4.711. Crossref ISI Google Scholar
44. M. Gu and S. C. Eisenstat , A divide-and-conquer algorithm for the symmetric tridiagonal eigenproblem , SIAM J. Matrix Anal. Appl. , 16 ( 1995 ), pp. 172 -- 191 , https://doi.org/10.1137/S0895479892241287. Link ISI Google Scholar
45. N. Hale, N. J. Higham and L. N. Trefethen , Computing $A^\alpha, \log(a)$, and related matrix functions by contour integrals , SIAM J. Numer. Anal. , 46 ( 2008 ), pp. 2505 -- 2523 , https://doi.org/10.1137/070700607. Link ISI Google Scholar
46. J. Havil , Gamma: Exploring Euler's Constant , Princeton University Press , 2003 . Google Scholar
47. M. R. Hestenes and E. Stiefel , Methods of conjugate gradients for solving linear systems , J. Res. Natl. Inst. Stand. , 49 ( 1952 ), pp. 409 -- 436 . Crossref ISI Google Scholar
48. D. Higdon , A primer on space-time modeling from a Bayesian perspective , in Statistical Methods for Spatio-Temporal Systems, B. Finkenstadt, L. Held, and V. lsham , eds., Chapman & Hall/CRC , 2007 , pp. 217 -- 279 . Google Scholar
49. J. P. Hobert and G. L. Jones , Honest exploration of intractable probability distributions via Markov chain Monte Carlo , Stat. Sci. , 16 ( 2001 ), pp. 312 -- 334 , https://doi.org/10.1214/ss/1015346317. Crossref ISI Google Scholar
50. J. Idier , ed., Bayesian Approach to Inverse Problems , Wiley , 2008 , https://doi.org/10.1002/9780470611197. Google Scholar
51. M. Ilic, T. Pettitt and I. Turner , Bayesian computations and efficient algorithms for computing functions of large sparse matrices , ANZIAM J. , 45 ( 2004 ), pp. 504 -- 518 , https://eprints.qut.edu.au/22511/. Crossref Google Scholar
52. M. Ilic, I. W. Turner and D. P. Simpson , A restarted Lanczos approximation to functions of a symmetric matrix , IMA J. Numer. Anal. , 30 ( 2009 ), pp. 1044 -- 1061 , https://doi.org/10.1093/imanum/drp003. Crossref ISI Google Scholar
53. H. Jeffreys , An invariant form for the prior probability in estimation problems , Proc. R. Soc. A , 186 ( 1946 ), pp. 453 -- 461 . Crossref Google Scholar
54. M. Johnson, J. Saunderson and A. Willsky , Analyzing Hogwild parallel Gaussian Gibbs sampling , in Adv. in Neural Information Process. Systems , 2013 , pp. 2715 -- 2723 . Google Scholar
55. R. E. Kass, B. P. Carlin, A. Gelman and R. M. Neal , Markov chain Monte Carlo in practice: A roundtable discussion , Amer. Statist. , 52 ( 1998 ), pp. 93 -- 100 . ISI Google Scholar
56. J. Kittler and J. Föglein , Contextual classification of multispectral pixel data , Image Vis. Comput. , 2 ( 1984 ), pp. 13 -- 29 , https://doi.org/10.1016/0262-8856(84)90040-4. Crossref Google Scholar
57. Y. Le Cun, L. Bottou, Y. Bengio and P. Haffner , Gradient-based learning applied to document recognition , Proc. IEEE , 86 ( 1998 ), pp. 2278 -- 2324 , https://doi.org/10.1109/5.726791. Crossref ISI Google Scholar
58. M. Li and D. Sun . Toh, A majorized ADMM with indefinite proximal terms for linearly constrained convex composite optimization , SIAM J. Optim. , 26 ( 2016 ), pp. 922 -- 950 , https://doi.org/10.1137/140999025. Link ISI Google Scholar
59. S. Z. Li , Markov Random Field Modeling in Image Analysis , 3 rd ed., Springer , 2009 . Google Scholar
60. Y. Li and S. K. Ghosh , Efficient sampling methods for truncated multivariate normal and Student-t distributions subject to linear inequality constraints , J. Stat. Theory Pract. , 9 ( 2015 ), pp. 712 -- 732 , https://doi.org/10.1080/15598608.2014.996690. Crossref ISI Google Scholar
61. A. C. Likas and N. P. Galatsanos , A variational approach for Bayesian blind image deconvolution , IEEE Trans. Signal Process. , 52 ( 2004 ), pp. 2222 -- 2233 . Crossref ISI Google Scholar
62. J. S. Liu , Monte Carlo Strategies in Scientific Computing , Springer , 2001 . Google Scholar
63. D. J. C. MacKay , Information Theory, Inference and Learning Algorithms , Cambridge University Press , 2003 . Google Scholar
64. S. Mallat , A Wavelet Tour of Signal Processing: The Sparse Way , 3 rd ed., Academic Press , 2008 . Google Scholar
65. A. Mantoglou and J. L. Wilson , The turning bands method for simulation of random fields using line generation by a spectral method , Water Resour. Res. , 18 ( 1982 ), pp. 1379 -- 1394 , https://doi.org/10.1029/WR018i005p01379. Crossref ISI Google Scholar
66. Y. Marnissi, D. Abboud, E. Chouzenoux, J.-C. Pesquet, M. El-Badaoui and A. Benazza-Benyahia , A data augmentation approach for sampling Gaussian models in high dimension , in Proc. European Signal Process. Conf. (EUSIPCO) , Coruna, Spain , 2019 , pp. 1 -- 5 . Google Scholar
67. Y. Marnissi, E. Chouzenoux and A. Benazza-Benyahia . Pesquet, An auxiliary variable method for Markov chain Monte Carlo algorithms in high dimension , Entropy , 20 ( 2018 ), art. 110 , https://doi.org/10.3390/e20020110. Crossref ISI Google Scholar
68. B. Martinet and Brève Régularisation d'inéquations variationnelles par approximations successives , ESAIM Math. Model. Numer. Anal. , 4 ( 1970 ), pp. 154 -- 158 , http://www.numdam.org/item/M2AN_1970__4_3_154_0. , http://www.numdam.org/item/M2AN_1970__4_3_154_0. Google Scholar
69. B. Martinet and Determination point fixe dâune application pseudocontractante. Cas de l'application prox , C. R. Math. Acad. Sci. Paris , 274 ( 1972 ), pp. 163 -- 165 . Google Scholar
70. J. Mason and D. Handscomb , Chebyshev Polynomials , CRC Press , 2002 . Google Scholar
71. J. M. Mejía and I. Rodríguez-Iturbe , On the synthesis of random field sampling from the spectrum: An application to the generation of hydrologic spatial processes , Water Resour. Res. , 10 ( 1974 ), pp. 705 -- 711 , https://doi.org/10.1029/WR010i004p00705. Crossref ISI Google Scholar
72. G. J. Minty and Monotone , Duke Math. J. , 29 ( 1962 ), pp. 341 -- 346 , https://doi.org/10.1215/S0012-7094-62-02933-2. Crossref ISI Google Scholar
73. A. de Moivre , The Doctrine of Chances: Or, a Method of Calculating the Probability of Events in Play , 1 st ed., W. Pearson , 1718. Google Scholar
74. R. Molina, J. Mateos and A. K. Katsaggelos , Blind deconvolution using a variational approach to parameter, image, and blur estimation , IEEE Trans. Image Process. , 15 ( 2006 ), pp. 3715 -- 3727 . Crossref ISI Google Scholar
75. R. Molina and B. D. Ripley , Using spatial models as priors in astronomical image analysis , J. Appl. Stat. , 16 ( 1989 ), pp. 193 -- 206 . Crossref Google Scholar
76. J. J. Moreau , Proximité et dualité dans un espace hilbertien , Bull. de la Soc. Math. de France , 93 ( 1965 ), pp. 273 -- 299 , https://doi.org/10.24033/bsmf.1625. Crossref Google Scholar
77. F. Orieux, O. Feron and J. F. Giovannelli , Sampling high-dimensional Gaussian distributions for general linear inverse problems , IEEE Signal Process. Lett. , 19 ( 2012 ), pp. 251 -- 254 , https://doi.org/10.1109/LSP.2012.2189104. Crossref ISI Google Scholar
78. F. Orieux, J.-F. Giovannelli and T. Rodet , Bayesian estimation of regularization and point spread function parameters for Wiener--Hunt deconvolution , J. Opt. Soc. Am. A , 27 ( 2010 ), pp. 1593 -- 1607 , https://doi.org/10.1364/JOSAA.27.001593. Crossref Google Scholar
79. G. Papandreou and A. L. Yuille , Gaussian sampling by local perturbations , in Adv. in Neural Information Process. Systems , 2010 , pp. 1858 -- 1866 . Google Scholar
80. T. Park and G. Casella , The Bayesian Lasso , J. Amer. Stat. Assoc. , 103 ( 2008 ), pp. 681 -- 686 , https://doi.org/10.1198/016214508000000337. Crossref ISI Google Scholar
81. A. Parker and C. Fox , Sampling Gaussian distributions in Krylov spaces with conjugate gradients , SIAM J. Sci. Comput. , 34 ( 2012 ), pp. B312 -- B334 , https://doi.org/10.1137/110831404. Link ISI Google Scholar
82. M. Pereira and N. Desassis , Efficient simulation of Gaussian Markov random fields by Chebyshev polynomial approximation , Spat. Stat. , 31 ( 2019 ), art. 100359 , https://doi.org/10.1016/j.spasta.2019.100359. Crossref ISI Google Scholar
83. N. G. Polson, J. G. Scott and J. Windle , Bayesian inference for logistic models using Pólya-Gamma latent variables , J. Amer. Stat. Assoc. , 108 ( 2013 ), pp. 1339 -- 1349 , https://doi.org/10.1080/01621459.2013.829001. Crossref ISI Google Scholar
84. W. H. Press, S. A. Teukolsky, W. T. Vetterling and B. P. Flannery , Numerical Recipes: The Art of Scientific Computing , 3 rd ed., Cambridge University Press , 2007 . Google Scholar
85. C. E. Rasmussen , Gaussian processes to speed up hybrid Monte Carlo for expensive Bayesian integrals , in Bayesian Statistics 7 , Oxford University Press , 2003 , pp. 651 -- 660 . Google Scholar
86. C. P. Robert , The Bayesian Choice: A Decision-Theoretic Motivation , Springer , 2001 . Google Scholar
87. C. P. Robert and G. Casella , Monte Carlo Statistical Methods , Springer , 2004 . Google Scholar
88. G. O. Roberts and S. K. Sahu , Updating schemes, correlation structure, blocking and parameterization for the Gibbs sampler , J. Roy. Stat. Soc. Ser. B , 59 ( 1997 ), pp. 291 -- 317 , https://doi.org/10.1111/1467-9868.00070. Crossref Google Scholar
89. G. O. Roberts and R. L. Tweedie , Exponential convergence of Langevin distributions and their discrete approximations , Bernoulli , 2 ( 1996 ), pp. 341 -- 363 , https://doi.org/10.2307/3318418. Crossref Google Scholar
90. R. T. Rockafellar , Monotone operators and the proximal point algorithm , SIAM J. Control Optim. , 14 ( 1976 ), pp. 877 -- 898 , https://doi.org/10.1137/0314056. Link ISI Google Scholar
91. H. Rue , Fast sampling of Gaussian Markov random fields , J. Roy. Stat. Soc. Ser. B , 63 ( 2001 ), pp. 325 -- 338 , https://doi.org/10.1111/1467-9868.00288. Crossref Google Scholar
92. H. Rue and L. Held , Gaussian Markov Random Fields: Theory and Applications , Chapman & Hall/CRC , 2005 . Google Scholar
93. Y. Saad , Iterative Methods for Sparse Linear Systems , 2 nd ed., SIAM , 2003 , https://doi.org/10.1137/1.9780898718003. Google Scholar
94. E. M. Scheuer and D. S. Stoller , On the generation of normal random vectors , Technometrics , 4 ( 1962 ), pp. 278 -- 281 , https://doi.org/10.2307/1266625. Crossref ISI Google Scholar
95. M. K. Schneider and A. S. Willsky , A Krylov subspace method for covariance approximation and simulation of random processes and fields , Multidimens. Syst. Signal Process. , 14 ( 2003 ), pp. 295 -- 318 , https://doi.org/10.1023/A:1023530718764. Crossref ISI Google Scholar
96. J. Shi and J. Malik , Normalized cuts and image segmentation , IEEE Trans. Pattern Anal. Mach. Intell. , 22 ( 2000 ), pp. 888 -- 905 , https://doi.org/10.1109/34.868688. Crossref ISI Google Scholar
97. M. Shinozuka . Jan, Digital simulation of random processes and its applications , J. Sound Vib. , 25 ( 1972 ), pp. 111 -- 128 , https://doi.org/10.1016/0022-460X(72)90600-1. Crossref ISI Google Scholar
98. D. P. Simpson, I. W. Turner and A. N. Pettitt , Fast Sampling from a Gaussian Markov Random Field Using Krylov Subspace Approaches , 2008 , https://eprints.qut.edu.au/14376/. Google Scholar
99. D. P. Simpson, I. W. Turner, C. M. Strickland and A. N. Pettitt , Scalable Iterative Methods for Sampling from Massive Gaussian Random Vectors, preprint, https://arxiv.org/abs/1312.1476 , 2013 . , https://arxiv.org/abs/1312.1476. Google Scholar
100. G. W. Stewart, Matrix Algorithms and Volume : Eigensystems , SIAM, 2001 1805 8. , https://doi.org/10.1137/1.9780898718058. Google Scholar
101. D. B. Thomas, W. Luk, P. H. Leong and J. D. Villasenor , Gaussian random number generators , ACM Comput. Surv. , 39 ( 2007 ), art. 11 , https://doi.org/10.1145/1287620.1287622. Crossref ISI Google Scholar
102. W. F. Trench , An algorithm for the inversion of finite Toeplitz matrices , SIAM J. Appl. Math. , 12 ( 1964 ), pp. 512 -- 522 , https://doi.org/10.1137/0112045. Link Google Scholar
103. D. W. Vasco, L. R. Johnson and O. Marques , Global Earth structure: Inference and assessment , Geophys. J. Int. , 137 ( 1999 ), pp. 381 -- 407 , https://doi.org/10.1046/j.1365-246X.1999.00823.x. Crossref ISI Google Scholar
104. M. Vono, N. Dobigeon and P. Chainais , Split-and-augmented Gibbs sampler: Application to large-scale inference problems , IEEE Trans. Signal Process. , 67 ( 2019 ), pp. 1648 -- 1661 , https://doi.org/10.1109/TSP.2019.2894825. Crossref ISI Google Scholar
105. M. Vono, N. Dobigeon and P. Chainais , Asymptotically exact data augmentation: Models, properties, and algorithms , J. Comput. Graph. Statist. , 30 ( 2021 ), pp. 335 -- 348 Crossref ISI Google Scholar
106. M. Welling and Y. W. Teh , Bayesian learning via stochastic gradient Langevin dynamics , in Proc. Int. Conf. Machine Learning (ICML) , 2011 , pp. 681 -- 688 . Google Scholar
107. S. Wilhelm and B. Manjunath , tmvtnorm: A package for the truncated multivariate normal distribution , The R Journal , 2 ( 2010 ), pp. 25 -- 29 , https://doi.org/10.32614/RJ-2010-005. Crossref ISI Google Scholar
108. C. K. I. Williams and C. E. Rasmussen , Gaussian Processes for Machine Learning , MIT Press , 2006 . Google Scholar
109. A. T. A. Wood and G. Chan , Simulation of stationary Gaussian processes in $[0, 1]^d$ , J. Comput. Graph. Stat. , 3 ( 1994 ), pp. 409 -- 432 , https://doi.org/10.1080/10618600.1994.10474655. Google Scholar
110. X. Zhang, M. Burger and S. Osher , A unified primal-dual algorithm framework based on Bregman iteration , J. Sci. Comput. , 46 ( 2011 ), pp. 20 -- 46 , https://doi.org/10.1007/s10915-010-9408-8. Crossref ISI Google Scholar

Keywords

- Synthetic Data Generation for Condition Monitoring of Railway Switches
  5 September 2022
  Cross Ref
- Bayesian Sparse Blind Deconvolution Using MCMC Methods Based on Normal-Inverse-Gamma Prior
  IEEE Transactions on Signal Processing, Vol. 70
  Cross Ref

Volume 64, Issue 1| 2022

SIAM Review

History

Submitted:02 October 2020
Accepted:09 August 2021
Published online:03 February 2022

Information

Keywords

MSC codes

PDF Download

Article & Publication Data

Article DOI:10.1137/20M1371026

Article page range:pp. 3-56

ISSN (print):0036-1445

ISSN (online):1095-7200

Publisher:Society for Industrial and Applied Mathematics

High-Dimensional Gaussian Sampling: A Review and a Unifying Approach Based on a Stochastic Proximal Point Algorithm

Abstract

Keywords

SIAM Review

History

Information

Browse

Browse

Alerts

Alerts

Information

Information

About

About

Verify Phone

Congrats!

High-Dimensional Gaussian Sampling: A Review and a Unifying Approach Based on a Stochastic Proximal Point Algorithm

Abstract

Keywords

SIAM Review

History

Information

Browse

Alerts

Information

About

Request Username

Register

Change Password

Password Changed Successfully

Login

Verify Phone

Congrats!