A Multiscale Strategy for Bayesian Inference Using Transport Maps

In many inverse problems, model parameters cannot be precisely determined from observational data. Bayesian inference provides a mechanism for capturing the resulting parameter uncertainty, but typically at a high computational cost. This work introduces a multiscale decomposition that exploits conditional independence across scales, when present in certain classes of inverse problems, to decouple Bayesian inference into two stages: (1) a computationally tractable coarse-scale inference problem, and (2) a mapping of the low-dimensional coarse-scale posterior distribution into the original high-dimensional parameter space. This decomposition relies on a characterization of the non-Gaussian joint distribution of coarse- and fine-scale quantities via optimal transport maps. We demonstrate our approach on a sequence of inverse problems arising in subsurface flow, using the multiscale finite element method to discretize the steady state pressure equation. We compare the multiscale strategy with full-dimensional Markov chain Monte Carlo on a problem of moderate dimension (100 parameters) and then use it to infer a conductivity field described by over 10000 parameters.

  • 1.  J. E. Aarnes and  Y. Efendiev , Mixed multiscale finite element methods for stochastic porous media flows , SIAM J. Sci. Comput. , 30 ( 2008 ), pp. 2319 -- 2339 LinkISIGoogle Scholar

  • 2.  R. Andreani E. G. Birgin J. M. Martínez and  M. L. Schuverdt , On augmented Lagrangian methods with general lower-level constraints , SIAM J. Optim. , 18 ( 2007 ), pp. 1286 -- 1309 LinkISIGoogle Scholar

  • 3.  T. Arbogast , Numerical subgrid upscaling of two-phase flow in porous media , in Numerical Treatment of Multiphase Flows in Porous Media , Lecture Notes in Phys. 552 , Springer , Berlin , 2000 , pp. 35 -- 49 . Google Scholar

  • 4.  R. C. Aster B. Borchers and  C. H. Thurber , Parameter Estimation and Inverse Problems , Academic Press , 2011 . Google Scholar

  • 5.  J. Bear , Dynamics of Fluids in Porous Media , Dover , 1988 . Google Scholar

  • 6.  L. Biegler G. Biros O. Ghattas M. Heinkenschloss D. Keyes B. Mallick L. Tenorio B. van Bloemen Waanders K. Willcox and  Y. Marzouk , Large-Scale Inverse Problems and Quantification of Uncertainty , Wiley Ser. Comput. Stat. 712, John Wiley and Sons , Chichester, UK, 2011 . Google Scholar

  • 7.  S. Boyd N. Parikh E. Chu B. Peleato and  J. Eckstein , Distributed optimization and statistical learning via the alternating direction method of multipliers , Found. Trends Mach. Learn. , 3 ( 2011 ), pp. 1 -- 122 . CrossrefGoogle Scholar

  • 8.  Y. Brenier , Polar factorization and monotone rearrangement of vector-valued functions , Comm. Pure Appl. Math. , 44 ( 1991 ), pp. 375 -- 417 . CrossrefISIGoogle Scholar

  • 9.  S. Brooks A. Gelman G. Jones and  X.-L. Meng , eds., Handbook of Markov Chain Monte Carlo, Chapman and Hall , Boca Raton, FL , 2011 . Google Scholar

  • 10.  T. Bui-Thanh and  O. Ghattas , Analysis of the Hessian for inverse scattering problems: I. Inverse shape scattering of acoustic waves , Inverse Problems , 28 ( 2012 ), 055001 . CrossrefISIGoogle Scholar

  • 11.  J. A. Christen and  C. Fox , MCMC using an approximation , J. Comput. Graph. Statist. , 14 ( 2005 ), pp. 795 -- 810 . CrossrefISIGoogle Scholar

  • 12.  T. Cui, K. J. H. Law, and Y. M. Marzouk, Dimension-Independent Likelihood-Informed MCMC, preprint, http://arxiv.org/abs/1411.3688 arXiv:1411.3688 [stat.CO], 2014.Google Scholar

  • 13.  T. Cui J. Martin Y. M. Marzouk A. Solonen and  A. Spantini , Likelihood-informed dimension reduction for nonlinear inverse problems , Inverse Problems , 30 ( 2014 ), 114015 . CrossrefISIGoogle Scholar

  • 14.  J. Doherty L. Brebber and  P. Whyte , PEST: Model-Independent Parameter Estimation , Watermark Computing , Corinda, Australia , 1994 . Google Scholar

  • 15.  P. Dostert Y. Efendiev T. Y. Hou and  W. Luo , Coarse-gradient Langevin algorithms for dynamic data integration and uncertainty quantification , J. Comput. Phys. , 217 ( 2006 ), pp. 123 -- 142 . CrossrefISIGoogle Scholar

  • 16.  A. Doucet N. de Freitas N. Gordon and  A. Smith , Sequential Monte Carlo in Practice , Springer , New York , 2001 . Google Scholar

  • 17.  W. E, B. Engquist X. Li W. Ren and  E. Vanden-Eijnden , Heterogeneous multiscale methods: A review , Comm. Comput. Phys. , 2 ( 2007 ), pp. 367 -- 450 . ISIGoogle Scholar

  • 18.  Y. Efendiev T. Hou and  W. Luo , Preconditioning Markov chain Monte Carlo simulations using coarse-scale models , SIAM J. Sci. Comput. , 28 ( 2006 ), pp. 776 -- 803 LinkISIGoogle Scholar

  • 19.  I. Epanomeritakis V. Akçelik O. Ghattas and  J. Bielak , A Newton-CG method for large-scale three-dimensional elastic full-waveform seismic inversion , Inverse Problems , 24 ( 2008 ), 034015 . CrossrefISIGoogle Scholar

  • 20.  G. Evensen , Data Assimilation: The Ensemble Kalman Filter , Springer , New York , 2009 . Google Scholar

  • 21.  R. A. Freeze J. Massmann L. Smith T. Sperling and  B. James , Hydrogeological decision analysis: 1 A framework , Groundwater , 28 ( 1990 ), pp. 738 -- 766 . CrossrefISIGoogle Scholar

  • 22.  A. Gelman J. B. Carlin H. S. Stern and  D. B. Rubin , Bayesian Data Analysis , Vol. 2 , Taylor & Francis , 2014 . Google Scholar

  • 23.  I. Graf and J. M. Stockie, Homogenization of the Stefan Problem, with Application to Maple SAP Exudation, preprint, http://arxiv.org/abs/1411.3039 arXiv:1411.3039 [math.AP], 2014.Google Scholar

  • 24.  H. Haario M. Laine A. Mira and  E. Saksman , DRAM: Efficient adaptive MCMC , Statist. Comput. , 16 ( 2006 ), pp. 339 -- 354 . CrossrefISIGoogle Scholar

  • 25.  C. He and  L. J. Durlofsky , Structured flow-based gridding and upscaling for modeling subsurface flow , Adv. Water Resources , 29 ( 2006 ), pp. 1876 -- 1892 . CrossrefISIGoogle Scholar

  • 26.  D. Higdon and  H. Lee . Bi, A Bayesian approach to characterizing uncertainty in inverse problems using coarse and fine-scale information , IEEE Trans. Signal Process. , 50 ( 2002 ), pp. 389 -- 399 . CrossrefISIGoogle Scholar

  • 27.  C. H. Holloman H. K. H. Lee and  D. M. Higdon , Multiresolution genetic algorithms and Markov chain Monte Carlo , J. Comput. Graph. Statist. , 15 ( 2006 ), pp. 861 -- 879 . CrossrefISIGoogle Scholar

  • 28.  T. Hou and  Y. Efendiev , Multiscale Finite Element Methods: Theory and Applications , Springer , New York , 2009 . Google Scholar

  • 29.  T. J. R. Hughes G. R. Feijoo L. Mazzei and  Baptiste Quincy , The variational multiscale method---a paradigm for computational mechanics , Comput. Methods Appl. Mech. Engrg. , 166 ( 1998 ), pp. 3 -- 34 . CrossrefISIGoogle Scholar

  • 30.  T. J. R. Hughes and  G. Sangalli , Variational multiscale analysis: The fine-scale Green's function, projection, optimization, localization, and stabilized methods , SIAM J. Numer. Anal. , 45 ( 2007 ), pp. 539 -- 557 LinkISIGoogle Scholar

  • 31.  J. Jagalur Mohan O. Sahni A. Doostan and  A. A. Oberai , Variational multiscale analysis: The fine-scale Green's function for stochastic partial differential equations , SIAM/ASA J. Uncertain. Quantif. , 2 ( 2014 ), pp. 397 -- 422 LinkGoogle Scholar

  • 32.  E. T. Jaynes , Probability Theory: The Logic of Science , Cambridge University Press , Cambridge, UK , 2003 . Google Scholar

  • 33.  H. Jeffreys , Scientific Inference , Cambridge University Press , Cambridge, UK , 1957 . Google Scholar

  • 34.  P. Jenny S. H. Lee and  H. A. Tchelepi , Multi-scale finite-volume method for elliptic problems in subsurface flow simulation , J. Comput. Phys. , 187 ( 2003 ), pp. 47 -- 67 . CrossrefISIGoogle Scholar

  • 35.  P. Jenny S. H. Lee and  H. A. Tchelepi , Adaptive fully implicit multi-scale finite-volume method for multi-phase flow and transport in heterogeneous porous media , J. Comput. Phys. , 217 ( 2006 ), pp. 627 -- 641 . CrossrefISIGoogle Scholar

  • 36.  R. Juanes . Dub, A locally conservative variational multiscale method for the simulation of porous media flow with multiscale source terms , Comput. Geosci. , 12 ( 2008 ), pp. 273 -- 295 . CrossrefISIGoogle Scholar

  • 37.  J. Kaipio and  E. Somersalo , Statistical and Computational Inverse Problems , Springer-Verlag , New York , 2004 . Google Scholar

  • 38.  R. M. Lewis and  V. Torczon , A Direct Search Approach to Nonlinear Programming Problems using an Augmented Lagrangian Method with Explicit Treatment of Linear Constraints , Tech. report WM-CS-2010-01 , College of William & Mary , 2010 . Google Scholar

  • 39.  W. Li and  O. A. Cirpka , Efficient geostatistical inverse methods for structured and unstructured grids , Water Resources Res. , 42 ( 2006 ), W06402 . CrossrefISIGoogle Scholar

  • 40.  J. S. Liu , Monte Carlo Strategies in Scientific Computing , Springer , New York , 2004 . Google Scholar

  • 41.  Y. M. Marzouk T. Moselhy M. Parno and  A. Spantini , An introduction to sampling via measure transport , in Handbook of Uncertainty Quantification , R. Ghanem, D. Higdon, and H. Owhadi, eds., Springer , New York , 2016 . Google Scholar

  • 42.  Y. M. Marzouk and  H. N. Najm , Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems , J. Comput. Phys. , 228 ( 2009 ), pp. 1862 -- 1902 . CrossrefISIGoogle Scholar

  • 43.  R. . McCann, Existence and uniqueness of monotone measure-preserving maps , Duke Math. J. , 80 ( 1995 ), pp. 309 -- 323 . CrossrefISIGoogle Scholar

  • 44.  R. E. Miller and  E. B. Tadmor , A unified framework and performance benchmark of fourteen multiscale atomistic/continuum coupling methods , Model. Simul. Mat. Sci. Engrg. , 17 ( 2009 ), 053001 . CrossrefISIGoogle Scholar

  • 45.  J. J. Moré , Generalizations of the trust region problem , Optim. Methods Softw. , 2 ( 1993 ), pp. 189 -- 209 . CrossrefGoogle Scholar

  • 46.  T. A. Moselhy and  Y. M. Marzouk , Bayesian inference with optimal maps , J. Comput. Phys. , 231 ( 2012 ), pp. 7815 -- 7850 . CrossrefISIGoogle Scholar

  • 47.  J. Nolen and  G. Papanicolaou , Fine scale uncertainty in parameter estimation for elliptic equations , Inverse Problems , 25 ( 2009 ), 115021 . CrossrefISIGoogle Scholar

  • 48.  J. Nolen G. A. Pavliotis and  A. M. Stuart , Multiscale modelling and inverse problems , in Numerical Analysis of Multiscale Problems , I. G. Graham, T. Y. Hou, O. Lakkis, and R. Scheichl, eds., Springer , Berlin, Heidelberg , 2012 , pp. 1 -- 34 . Google Scholar

  • 49.  M. Parno , Transport Maps for Accelerated Bayesian Computation , Ph.D. thesis, Massachusetts Institute of Technology , Cambridge, MA , 2014 . Google Scholar

  • 50.  M. Parno A. Davis and  P. Conrad , MIT Uncertainty Quantification (MUQ) library , 2014 Google Scholar

  • 51.  M. Parno and Y. Marzouk, Transport Map Accelerated Markov Chain Monte Carlo, preprint, http://arxiv.org/abs/1412.5492 arXiv:1412.5492 [stat.CO], 2014.Google Scholar

  • 52.  G. A. Pavliotis and  A. M. Stuart , Multiscale Methods: Averaging and Homogenization , Springer , New York , 2008 . Google Scholar

  • 53.  C. P. Robert and  G. Casella , Monte Carlo Statistical Methods , Springer , New York , 2004 . Google Scholar

  • 54.  G. O. Roberts and  O. Stramer , Langevin diffusions and Metropolis-Hastings algorithms , Methodol. Comput. Appl. Probab. , 4 ( 2002 ), pp. 337 -- 357 . CrossrefGoogle Scholar

  • 55.  A. Spantini A. Solonen T. Cui J. Martin L. Tenorio and  Y. Marzouk , Optimal low-rank approximations of Bayesian linear inverse problems , SIAM J. Sci. Comput. , 37 ( 2015 ), pp. A2451 -- A2487 LinkISIGoogle Scholar

  • 56.  F. Stavropoulou and  J. Muller , A polynomial chaos based Bayesian approach for online parameter estimation and control , in Proceedings of the 2011 IEEE International Symposium on Computer-Aided Control System Design (CACSD) , 2011 , pp. 1391 -- 1396 . Google Scholar

  • 57.  A. M. Stuart and  Inverse : A Bayesian perspective , Acta Numer. , 19 ( 2010 ), pp. 451 -- 559 . CrossrefGoogle Scholar

  • 58.  E. G. Tabak and  C. V. Turner , A family of nonparametric density estimation algorithms , Comm. Pure Appl. Math. , 66 ( 2013 ), p. 145 -- 164 . CrossrefISIGoogle Scholar

  • 59.  E. G. Tabak and  E. Vanden-Eijnden , Density estimation by dual ascent of the log-likelihood , Commun. Math. Sci. , 8 ( 2010 ), pp. 217 -- 233 . CrossrefISIGoogle Scholar

  • 60.  A. Tarantola , Inverse Problem Theory and Methods for Model Parameter Estimation , SIAM , Philadelphia , 2005 . Google Scholar

  • 61.  U.S. , Contaminated Sediment Remediation Guidance for Hazardous Waste Sites, tech. report , United States Environmental Protection Agency , Washington, D.C. , 2005 . Google Scholar

  • 62.  V. V. Vesselinov D. O. Malley and  D. Katzman , Robust decision analysis for environmental management of groundwater contamination sites , Vulnerability Uncertainty Risk , 2014 , pp. 1970 -- 1979 CrossrefGoogle Scholar

  • 63.  C. Villani , Topics in Optimal Transportation , AMS , Providence , RI , 2003 . Google Scholar

  • 64.  C. Villani , Optimal Transport: Old and New , Springer-Verlag , New York , 2009 . Google Scholar

  • 65.  C. R. Vogel , Computational Methods for Inverse Problems , Frontiers Appl. Math. 23 , SIAM , Philadelphia , 2002 . Google Scholar

  • 66.  A. Wächter and  L. T. Biegler , On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , Math. Program. , 106 ( 2006 ), pp. 25 -- 57 . CrossrefISIGoogle Scholar

  • 67.  B. J. Wagner and  S. M. Gorelick , Reliable aquifer remediation in the presence of spatially variable hydraulic conductivity: From data to design , Water Resources Res. , 25 ( 1989 ), pp. 2211 -- 2225 . CrossrefISIGoogle Scholar

  • 68.  J. Wan and  N. Zabaras , A Bayesian approach to multiscale inverse problems using the sequential Monte Carlo method , Inverse Problems , 27 ( 2011 ), 105004 . CrossrefISIGoogle Scholar

  • 69.  U. Wolff Carlo errors with less errors , Comput. Phys. Commun. , 156 ( 2004 ), pp. 143 -- 153 . CrossrefISIGoogle Scholar