An MCMC Algorithm for Parameter Estimation in Signals with Hidden Intermittent Instability
Abstract
Prediction of extreme events is a highly important and challenging problem in science, engineering, finance, and many other areas. The observed extreme events in these areas are often associated with complex nonlinear dynamics with intermittent instability. However, due to lack of resolution or incomplete knowledge of the dynamics of nature, these instabilities are typically hidden. To describe nature with hidden instability, a stochastic parameterized model is used as the low-order reduced model. Bayesian inference incorporating data augmentation, regarding the missing path of the hidden processes as the augmented variables, is adopted in a Markov chain Monte Carlo (MCMC) algorithm to estimate the parameters in this reduced model from the partially observed signal. Howerver, direct application of this algorithm leads to an extremely low acceptance rate of the missing path. To overcome this shortcoming, an efficient MCMC algorithm which includes a pre-estimation of hidden processes is developed. This algorithm greatly increases the acceptance rate and provides the low-order reduced model with a high skill in capturing the extreme events due to intermittency.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
A. Beskos and G. Roberts Exact simulation of diffusions, Ann. Appl. Probab., 15 (2005), pp. 2422--2444.
2.
M. Bladt and M. Sorensen, Simple Simulation of Diffusion Ridges with Application to Likelihood Inference for Diffusions, Working paper, Centre for Analytical Finance, University of Copenhagen, Copenhagen, 2007.
3.
M. Branicki, B. Gershgorin, and A. J. Majda, Filtering skill for turbulent signals for a suite of nonlinear and linear extended Kalman filters, J. Comput. Phys., 231 (2012), pp. 1462--1498.
4.
M. Branicki, N. Chen, and A. J. Majda, Non-Gaussian test models for prediction and state estimation with model errors, Chinese Ann. Math. Ser. B, 34 (2013), pp. 29--64.
5.
M. Branicki and A. J. Majda, Quantifying uncertainty for predictions with model error in non-Guassian systems with intermittency, Nonlinearity, 25 (2012), pp. 2543--2578.
6.
M. Branicki and A. J. Majda, Dynamic stochastic superresolution of sparseley observed turbulent systems, J. Comput. Phys., 241 (2013), pp. 333--363.
7.
K. Cochrane, C. De Young, D. Soto, and T. Bahri, Climate Change Implications for Fisheries and Aquaculture, FAO Fisheries and Aquaculture technical report 530, Food and Agriculture Organization of the United Nations, Rome, 2009.
8.
S. Cotter, G. Roberts, A. Stuart, and D. White, MCMC methods for functions: Modifying old algorithms to make them faster, Statist. Sci., 28 (2013), pp. 424--446.
9.
S. Chib, M. K. Pitt, and N. Shephard, Likelihood based inference for diffusion driven models, Working paper, Nuffield College, University of Oxford, Oxford, 2004.
10.
S. Chib, M. K. Pitt, and N. Shephard, Likelihood based inference for diffusion driven models, Working paper, Nuffield College, University of Oxford, Oxford, 2010.
11.
O. Elerian, S. Chib, and N. Shephard, Likelihood inference for discretely observed non-linear diffusions, Econometrica, 69 (2001), pp. 959--993.
12.
B. Eraker, Markov chain Monte Carlo analysis of diffusion models with application to finance, J. Bus. Econom. Statist., 19 (2001), pp. 177--191.
13.
C. Fuchs, Inference for Diffusion Processes with Applications in Life Sciences, Springer, Heidelberg, 2013.
14.
B. Gershgorin, J. Harlim, and A. J. Majda, Test models for improving filtering with model errors through stochastic parameter estimation, J. Comput. Phys., 229 (2010), pp. 1--31.
15.
B. Gershgorin, J. Harlim, and A. J. Majda, Improving filtering and prediction of spatially extended turbulent systems with model errors through stochastic parameter estimation, J. Comput. Phys., 229 (2010), pp. 32--57.
16.
B. Gershgorin and A. J. Majda, Filtering a statistically exactly solvable test model for turbulent tracers from partial observations, J. Comput. Phys., 230 (2011), pp. 1602--1638.
17.
D. Giannakis, A. J. Majda, and I. Horenko, Information theory, model error, and predictive skill of stochastic models for complex nonlinear systems, Phys. D, 241 (2012), pp. 1735--1752.
18.
A. Golightly and D. J. Wilkinson, Bayesian inference for stochastic kinetic models using a diffusion approximation, Biometrics, 61 (2005), pp. 781--788.
19.
A. Golightly and D. J. Wilkinson, Bayesian sequential inference for nonlinear multivariate diffusions, Stat. Comput., 16 (2006), pp. 323--338.
20.
A. Golightly and D. J. Wilkinson, Bayesian sequential inference for stochastic kinetic biochemical network models, J. Comput. Biol., 13 (2006), pp. 838--851.
21.
A. Golightly and D. J. Wilkinson, Bayesian inference for nonlinear multivariate diffusion models observed with error, Comput. Statist. Data Anal., 52 (2008), pp. 1674--1693.
22.
A. Golightly and D. J. Wilkinson, Bayesian parameter inference for stochastic biochemical network models using particle MCMC, Interface Focus, 1 (2011), pp. 807--820.
23.
J. Harlim and A. J. Majda, Filtering turbulent sparsely observed geophysical flows, Monthly Weather Rev., 138 (2010), pp. 1050--1083.
24.
S. E. Hills and A. F. M. Smith, Parameterization Issues in Bayesian Inference, in Bayesian Statistics 4, J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith, eds., Oxford University Press, Oxford, 1992.
25.
K. Kalogeropoulos, Likelihood-based inference for a class of multivariate diffusions with unobserved paths, J. Statistical Planning and Inference, 137 (2007), pp. 3092--3102.
26.
A. J. Majda and M. Branicki, Lessons in uncertainty quantification for turbulent dynamical system, Discrete Contin. Dyn. Syst., 32 (2013), pp. 3133--3221.
27.
A. J. Majda, C. Franzke, and D. Crommelin, Normal forms for reduced stochastic climate models, Proc. Natl. Acad. Sci., 16 (2009), pp. 3649--3653.
28.
A. J. Majda and J. Harlim, Filtering Complex Turbulent Systems, Cambridge University Press, Cambridge, 2012.
29.
A. J. Majda and B. Gershgorin, Link between statistical equilibrium fidelity and forecasting skill for complex systems with model error, Proc. Natl. Acad. Sci., 108 (2011), pp. 12599--12604.
30.
A. J. Majda, J. Harlim, and B. Gershgorin, Mathematical strategies for filtering turbulent dynamical systems, Discrete Contin. Dyn. Syst., 27 (2010), pp. 441--486.
31.
G. A. Meehl and C. Tebaldi, More intense, more frequent, and longer lasting heat waves in the $21$st century, Science, 305 (2004), pp. 994--997.
32.
O. Papaspiliopoulos, G. O. Roberts, and O. Stramer, Data augmentation for diffusions, J. Comput. Graph. Statist., 22 (2013), pp. 665--688.
33.
B. Ø ksendal, Stochastic Differential Equations: An Introduction with Applications, 6th ed., Springer, Heidelberg, 2003.
34.
G. Roberts and O. Stramer, On inference for partial observations from partial observed nonlinear diffusion model using Metropolis--Hastings algorithms, Biometrika 88 (2001), pp. 603--621.
35.
G. Sermaidis, O. Papaspiliopoulos, G. O. Roberts, A. Beskos, and P. Fearnhead, Markov chain Monte Carlo for exact inference for diffusions, Scand. J. Statist., 40 (2013), pp. 294--321.
36.
H. Sorensen, Parametric inference for diffusion processes observed at discrete points in time: A survey, Int. Statist. Rev., 72 (2004), pp. 337--354.
37.
O. Stramer and M. Bognar, Bayesian inference for irreducible diffusion processes using the pseudo-marginal approach, Bayesian Anal., 6 (2011), pp. 231--258.
38.
N. N. Taleb, The Fourth Quadrant: A Map of the Limits of Statistics, Edge, 2008.
39.
M. A. Tanner and W. H. Wong, The calculation of posterior distributions by data augmentation, J. Amer. Statist. Assoc., 82 (1987), pp. 528--540.
Information & Authors
Information
Published In
SIAM/ASA Journal on Uncertainty Quantification
Pages: 647 - 669
DOI: 10.1137/130944977
ISSN (online): 2166-2525
Copyright
© 2014, Society for Industrial and Applied Mathematics.
History
Submitted: 12 November 2013
Accepted: 2 September 2014
Published online: 11 November 2014
Keywords
MSC codes
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- Estimating the time-evolving refractivity of a turbulent medium using optical beam measurements: a data assimilation approachJournal of the Optical Society of America A | 15 April 2024
- Augmented pseudo-marginal Metropolis–Hastings for partially observed diffusion processesStatistics and Computing, Vol. 32, No. 1 | 15 February 2022
- Learning forecasts of rare stratospheric transitions from short simulationsMonthly Weather Review, Vol. 20 | 31 Aug 2021
- Parameter Estimation of Partially Observed Turbulent Systems Using Conditional Gaussian Path-Wise SamplerComputation, Vol. 9, No. 8 | 13 August 2021
- Improving the prediction of complex nonlinear turbulent dynamical systems using nonlinear filter, smoother and backward sampling techniquesResearch in the Mathematical Sciences, Vol. 7, No. 3 | 9 July 2020
- A study of the data augmentation strategy for stochastic differential equationsJournal of Statistical Computation and Simulation, Vol. 90, No. 10 | 2 April 2020
- Predicting observed and hidden extreme events in complex nonlinear dynamical systems with partial observations and short training time seriesChaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 30, No. 3 | 2 March 2020
- A new efficient parameter estimation algorithm for high-dimensional complex nonlinear turbulent dynamical systems with partial observationsJournal of Computational Physics, Vol. 397 | 1 Nov 2019
- Simple nonlinear models with rigorous extreme events and heavy tailsNonlinearity, Vol. 32, No. 5 | 12 April 2019
- Temporal Filtering Enhances the Skewness of Sea Surface WindsJournal of Climate, Vol. 31, No. 14 | 1 Jul 2018
- Conditional Gaussian Systems for Multiscale Nonlinear Stochastic Systems: Prediction, State Estimation and Uncertainty QuantificationEntropy, Vol. 20, No. 7 | 4 July 2018
- Rigorous Analysis for Efficient Statistically Accurate Algorithms for Solving Fokker--Planck Equations in Large DimensionsSIAM/ASA Journal on Uncertainty Quantification, Vol. 6, No. 3 | 13 September 2018
- Filtering Nonlinear Turbulent Dynamical Systems through Conditional Gaussian StatisticsMonthly Weather Review, Vol. 144, No. 12 | 1 December 2016
- Filtering the Stochastic Skeleton Model for the Madden–Julian OscillationMonthly Weather Review, Vol. 144, No. 2 | 29 January 2016
- Bayesian sparse solutions to linear inverse problems with non-stationary noise with Student-t priorsDigital Signal Processing, Vol. 47 | 1 Dec 2015
- Predicting the Real-Time Multivariate Madden–Julian Oscillation Index through a Low-Order Nonlinear Stochastic ModelMonthly Weather Review, Vol. 143, No. 6 | 28 May 2015
View Options
Get Access
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].