Methods and Algorithms for Scientific Computing

Randomize-Then-Optimize: A Method for Sampling from Posterior Distributions in Nonlinear Inverse Problems


High-dimensional inverse problems present a challenge for Markov chain Monte Carlo (MCMC)-type sampling schemes. Typically, they rely on finding an efficient proposal distribution, which can be difficult for large-scale problems, even with adaptive approaches. Moreover, the autocorrelations of the samples typically increase with dimension, which leads to the need for long sample chains. We present an alternative method for sampling from posterior distributions in nonlinear inverse problems, when the measurement error and prior are both Gaussian. The approach computes a candidate sample by solving a stochastic optimization problem. In the linear case, these samples are directly from the posterior density, but this is not so in the nonlinear case. We derive the form of the sample density in the nonlinear case, and then show how to use it within both a Metropolis--Hastings and importance sampling framework to obtain samples from the posterior distribution of the parameters. We demonstrate, with various small- and medium-scale problems, that randomize-then-optimize can be efficient compared to standard adaptive MCMC algorithms.


  1. nonlinear inverse problems
  2. Bayesian methods
  3. uncertainty quantification
  4. computational statistics
  5. sampling methods

MSC codes

  1. 15A29
  2. 65C05
  3. 65C60

Get full access to this article

View all available purchase options and get full access to this article.


J. M. Bardsley, MCMC-based image reconstruction with uncertainty quantification, SIAM J. Sci. Comput., 34 (2012), pp. A1316--A1332.
Y. Chen and D. Oliver, Ensemble randomized maximum likelihood method as an iterative ensemble smoother, Math Geosci, 44 (2012), pp. 1--26.
T. Cui, J. Martin, Y. Marzouk, A. Solonen, and A. Spantini, Likelihood-Informed Dimension Reduction for Nonlinear Inverse Problems, Inverse Problems, to appear.
A. Doucet, N. D. Freitas, and N. Gordon, Sequential Monte Carlo methods in practice, Stat. Eng. Inf. Sci., Springer, New York, 2001.
B. Efron and R. J. Tibshirani, An Introduction to the Bootstrap, Chapman and Hall/CRC, New York, 1993.
D. Gamerman and H. F. Lopes, Markov Chain Monte Carlo -- Stochastic Simulation for Bayesian Inference. 2nd ed., Chapman and Hall/CRC, Boca Raton, FL, 2006.
H., Haario, M. Laine, A. Mira, and E. Saksman, DRAM: Efficient adaptive MCMC, Statist. Comput., 16 (2006), pp. 339--354.
H. Haario, E. Saksman, and J. Tamminen, An adaptive Metropolis algorithm, Bernoulli, 7 (2001), pp. 223--242.
H. Haario, M. Laine, M. Lehtinen, E. Saksman, and J. Tamminen, Markov chain Monte Carlo methods for high dimensional inversion in remote sensing, J. R. Stat. Soc. Ser. B Stat. Methodol., 66 (2004), pp. 591--607.
J. P. Kaipio, V. Kolehmainen, E. Somersalo, and M. Vauhkonen, Statistical inversion and Monte Carlo sampling methods in electrical impedance tomography, Inverse Problems, 16 (2000), pp. 1487--1522.
J. P. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems, Springer, New York, 2005.
J. Martin, L. C. Wilcox, C. Burstedde, and O. Ghattas, A stochastic Newton MCMC method for large-scale statistical inverse problems with application to seismic inversion, SIAM J. Sci. Comput., 34 (2012), pp. A1460--A1487.
G. Nicholls and C. Fox, Prior modelling and posterior sampling in impedance imaging, in Bayesian Inference for Inverse Problems, Proc. SPIE, 3459, SPIE, Bellingham, WA, 1998, pp. 116--127.
Y. Qi and T. P. Minka, Hessian-based Markov chain Monte-Carlo algorithms, in First Cape Cod Workshop on Monte Carlo Methods, Cape Cod, MA, September, 2002.
C. Robert and G. Casella, Monte Carlo Statistical Methods, 2nd ed., Springer, New York, 2004.
C. R. Vogel, Computational Methods for Inverse Problems, Front. Appl. Math. 23, SIAM, Philadelphia, 2002.
D. Watzenig and C. Fox, A review of statistical modeling and inference for electrical capacitance tomography, Meas. Sci. Tech., 20 (2009), 052002.

Information & Authors


Published In

cover image SIAM Journal on Scientific Computing
SIAM Journal on Scientific Computing
Pages: A1895 - A1910
ISSN (online): 1095-7197


Submitted: 8 April 2014
Accepted: 9 June 2014
Published online: 14 August 2014


  1. nonlinear inverse problems
  2. Bayesian methods
  3. uncertainty quantification
  4. computational statistics
  5. sampling methods

MSC codes

  1. 15A29
  2. 65C05
  3. 65C60



Metrics & Citations



If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By







Copy the content Link

Share with email

Email a colleague

Share on social media

The SIAM Publications Library now uses SIAM Single Sign-On for individuals. If you do not have existing SIAM credentials, create your SIAM account