The Mean-Field Ensemble Kalman Filter: Near-Gaussian Setting
Abstract.
The ensemble Kalman filter is widely used in applications because, for high-dimensional filtering problems, it has a robustness that is not shared, for example, by the particle filter; in particular, it does not suffer from weight collapse. However, there is no theory which quantifies its accuracy as an approximation of the true filtering distribution, except in the Gaussian setting. To address this issue, we provide the first analysis of the accuracy of the ensemble Kalman filter beyond the Gaussian setting. We prove two types of results: The first type comprises a stability estimate controlling the error made by the ensemble Kalman filter in terms of the difference between the true filtering distribution and a nearby Gaussian, and the second type uses this stability result to show that, in a neighborhood of Gaussian problems, the ensemble Kalman filter makes a small error in comparison with the true filtering distribution. Our analysis is developed for the mean-field ensemble Kalman filter. We rewrite the update equations for this filter and for the true filtering distribution in terms of maps on probability measures. We introduce a weighted total variation metric to estimate the distance between the two filters, and we prove various stability estimates for the maps defining the evolution of the two filters in this metric. Using these stability estimates, we prove results of the first and second types in the weighted total variation metric. We also provide a generalization of these results to the Gaussian projected filter, which can be viewed as a mean-field description of the unscented Kalman filter.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments.
We are grateful to the reviewers for very useful comments on a previous version of this paper. In particular, we thank the reviewers for pointing out the elementary inequality in Lemma A.3 and for showing us how this inequality could be used to significantly improve the proofs of Lemmas A.4, A.6, and C.1. The constructive proof of Lemma A.4, in particular, was proposed by a reviewer.
References
1.
A. Apte, C. K. Jones, and A. M. Stuart, A Bayesian approach to Lagrangian data assimilation, Tellus A, 60 (2008), pp. 336–347.
2.
A. Arnold, J. A. Carrillo, and C. Manzini, Refined long-time asymptotics for some polymeric fluid flow models, Commun. Math. Sci., 8 (2010), pp. 763–782.
3.
M. Asch, M. Bocquet, and M. Nodet, Data Assimilation, Fundamentals of Algorithms 11, SIAM, Philadelphia, 2016, https://doi.org/10.1137/1.9781611974546.pt1.
4.
P. Bickel, B. Li, and T. Bengtsson, Sharp failure rates for the bootstrap particle filter in high dimensions, in Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh, Institute of Mathematical Statistics Collections, Vol. 3, B. Clarke and S. Ghosal, eds., Institute of Mathematical Statistics, Waite Hill, OH, 2008, pp. 318–329, https://doi.org/10.1214/074921708000000228.
5.
C. M. Bishop, Pattern Recognition and Machine Learning, Information Science and Statistics, Springer-Verlag, Berlin, 2006, https://doi.org/10.1007/978-0-387-45528-0.
6.
A. Biswas and M. Branicki, A Unified Framework for the Analysis of Accuracy and Stability of a Class of Approximate Gaussian Filters for the Navier-Stokes Equations, preprint, arXiv:2402.14078, 2024.
7.
F. Bolley, J. A. Cañizo, and J. A. Carrillo, Stochastic mean-field limit: Non-Lipschitz forces and swarming, Math. Models Methods Appl. Sci., 21 (2011), pp. 2179–2210, https://doi.org/10.1142/S0218202511005702.
8.
E. Caglioti, F. Golse, and M. Iacobelli, Quantization of probability distributions and gradient flows in space dimension 2, Ann. Inst. H. Poincaré Anal. Non Linéaire, 35 (2018), pp. 1531–1555, https://doi.org/10.1016/j.anihpc.2017.12.003.
9.
E. Calvello, S. Reich, and A. M. Stuart, Ensemble Kalman Methods: A Mean Field Perspective, preprint, arXiv:2209.11371, 2024, (to appear 2025, Acta Numerica).
10.
J. A. Carrillo and U. Vaes, Wasserstein stability estimates for covariance-preconditioned Fokker-Planck equations, Nonlinearity, 34 (2021), pp. 2275–2295.
11.
D. Crişan, P. Del Moral, and T. J. Lyons, Interacting particle systems approximations of the Kushner-Stratonovitch equation, Adv. Appl. Probab., 31 (1999), pp. 819–838, https://doi.org/10.1239/aap/1029955206.
12.
D. Crisan, A. López-Yela, and J. Miguez, Stable approximation schemes for optimal filters, SIAM/ASA J. Uncertain. Quantif., 8 (2020), pp. 483–509, https://doi.org/10.1137/19M1255410.
13.
J. De Wiljes, S. Reich, and W. Stannat, Long-time stability and accuracy of the ensemble Kalman–Bucy filter for fully observed processes and small measurement noise, SIAM J. Appl. Dyn. Syst., 17 (2018), pp. 1152–1181.
14.
P. Del Moral, Nonlinear filtering: Interacting particle resolution, C. R. Acad. Sci. Paris Sér. I Math., 325 (1997), pp. 653–658, https://doi.org/10.1016/S0764-4442(97)84778-7.
15.
P. Del Moral and A. Guionnet, On the stability of interacting processes with applications to filtering and genetic algorithms, Ann. Inst. Henri Poincaré Probab. Stat., 37 (2001), pp. 155–194, https://doi.org/10.1016/S0246-0203(00)01064-5.
16.
P. Del Moral and J. Tugaut, On the stability and the uniform propagation of chaos properties of ensemble Kalman-Bucy filters, Ann. Appl. Probab., 28 (2018), pp. 790–850, https://doi.org/10.1214/17-AAP1317.
17.
L. Devroye, A. Mehrabian, and T. Reddad, The Total Variation Distance between High-Dimensional Gaussians, preprint, arXiv:1810.08693, 2018.
18.
Z. Ding and Q. Li, Ensemble Kalman inversion: Mean-field limit and convergence analysis, Stat. Comput., 31 (2021), 9, https://doi.org/10.1007/s11222-020-09976-0.
19.
Z. Ding and Q. Li, Ensemble Kalman sampler: Mean-field limit and convergence analysis, SIAM J. Math. Anal., 53 (2021), pp. 1546–1578, https://doi.org/10.1137/20M1339507.
20.
Z. Ding, Q. Li, and J. Lu, Ensemble Kalman inversion for nonlinear problems: Weights, consistency, and variance bounds, Found. Data Sci., 3 (2021), pp. 371–411, https://doi.org/10.3934/fods.2020018.
21.
A. Doucet, N. de Freitas, and N. Gordon, Sequential Monte Carlo Methods in Practice, Statistics for Engineering and Information Science, Springer-Verlag, Berlin, 2001, https://doi.org/10.1007/978-1-4757-3437-9.
22.
A. Durmus, A. Eberle, A. Guillin, and R. Zimmer, An elementary approach to uniform in time propagation of chaos, Proc. Amer. Math. Soc., 148 (2020), pp. 5387–5398, https://doi.org/10.1090/proc/14612.
23.
O. G. Ernst, B. Sprungk, and H. Starkloff, Analysis of the ensemble and polynomial chaos Kalman filters in Bayesian inverse problems, SIAM/ASA J. Uncertain. Quantif., 3 (2015), pp. 823–851, https://doi.org/10.1137/140981319.
24.
S. Ertel and W. Stannat, Analysis of the ensemble Kalman–Bucy filter for correlated observation noise, Ann. Appl. Probab., 34 (2024), pp. 1072–1107.
25.
G. Evensen, Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics, J. Geophys. Res. Oceans, 99 (1994), pp. 10143–10162.
26.
G. Evensen, Data Assimilation: The Ensemble Kalman Filter, 2nd ed., Springer-Verlag, Berlin, 2009, https://doi.org/10.1007/978-3-642-03711-5.
27.
G. Evensen, F. C. Vossepoel, and P. J. van Leeuwen, Data Assimilation Fundamentals: A Unified Formulation of the State and Parameter Estimation Problem, Springer Textbooks in Earth Sciences, Geography and Environment, Springer-Verlag, Berlin, 2022, https://doi.org/10.1007/978-3-030-96709-3.
28.
A. Garbuno-Inigo, F. Hoffmann, W. Li, and A. M. Stuart, Interacting Langevin diffusions: Gradient structure and ensemble Kalman sampler, SIAM J. Appl. Dyn. Syst., 19 (2020), pp. 412–441, https://doi.org/10.1137/19M1251655.
29.
A. Garbuno-Inigo, N. Nüsken, and S. Reich, Affine invariant interacting Langevin dynamics for Bayesian inference, SIAM J. Appl. Dyn. Syst., 19 (2020), pp. 1633–1658.
30.
M. Ghil, S. Cohn, J. Tavantzis, K. Bube, and E. Isaacson, Application of estimation theory to numerical weather prediction, in Seminar on Data Assimilation Methods, European Centre for Medium-Range Weather Forecasts, Shinfield Park, UK, 1980, pp. 139–224.
31.
G. A. Gottwald and A. J. Majda, A mechanism for catastrophic filter divergence in data assimilation for sparse observation networks, Nonlinear Process. Geophys., 20 (2013), pp. 705–712.
32.
S. Graf and H. Luschgy, Foundations of Quantization for Probability Distributions, Lecture Notes in Mathematics 1730, Springer-Verlag, Berlin, 2000.
33.
M. Hairer and J. C. Mattingly, Yet another look at Harris’ ergodic theorem for Markov chains, in Seminar on Stochastic Analysis, Random Fields and Applications VI, Progress in Probability 63, Birkhäuser Basel, Basel, Switzerland, 2011, pp. 109–117, https://doi.org/10.1007/978-3-0348-0021-1.
34.
D. Z. Huang, J. Huang, S. Reich, and A. M. Stuart, Efficient derivative-free Bayesian inference for large-scale inverse problems, Inverse Problems, 38 (2022), 125006, https://doi.org/10.1088/1361-6420/ac99fa.
35.
D. Z. Huang, T. Schneider, and A. M. Stuart, Iterated Kalman methodology for inverse problems, J. Comput. Phys., 463 (2022), 111262, https://doi.org/10.1016/j.jcp.2022.111262.
36.
M. A. Iglesias, K. J. H. Law, and A. M. Stuart, Ensemble Kalman methods for inverse problems, Inverse Problems, 29 (2013), 045001, https://doi.org/10.1088/0266-5611/29/4/045001.
37.
A. H. Jazwinski, Stochastic Processes and Filtering Theory, Dover, New York, 2007.
38.
S. J. Julier and J. K. Uhlmann, New extension of the Kalman filter to nonlinear systems, in Signal Processing, Sensor Fusion, and Target Recognition VI, Vol. 3068, SPIE, Bellingham, WA, 1997, pp. 182–193.
39.
R. E. Kalman, A new approach to linear filtering and prediction problems, Trans. ASME Ser. D. J. Basic Engrg., 82 (1960), pp. 35–45.
40.
D. Kelly, A. J. Majda, and X. T. Tong, Concrete ensemble Kalman filters with rigorous catastrophic filter divergence, Proc. Natl. Acad. Sci. USA, 112 (2015), pp. 10589–10594.
41.
D. T. B. Kelly, K. J. H. Law, and A. M. Stuart, Well-posedness and accuracy of the ensemble Kalman filter in discrete and continuous time, Nonlinearity, 27 (2014), pp. 2579–2604, https://doi.org/10.1088/0951-7715/27/10/2579.
42.
T. Lange and W. Stannat, Mean field limit of ensemble square root filters—discrete and continuous time, Found. Data Sci., 3 (2021), pp. 563–588.
43.
T. Lange and W. Stannat, On the continuous time limit of the ensemble kalman filter, Math. Comp., 90 (2021), pp. 233–265.
44.
K. Law, A. Stuart, and K. Zygalakis, Data Assimilation: A Mathematical Introduction, Texts in Applied Mathematics 62, Springer-Verlag, Berlin, 2015, https://doi.org/10.1007/978-3-319-20325-6.
45.
F. Le Gland, V. Monbet, and V.-D. Tran, Large sample asymptotics for the ensemble Kalman filter, in The Oxford Handbook of Nonlinear Filtering, Oxford University Press, Oxford, 2011, pp. 598–631.
46.
A. J. Majda and X. T. Tong, Performance of ensemble Kalman filters in large dimensions, Comm. Pure Appl. Math., 71 (2018), pp. 892–937, https://doi.org/10.1002/cpa.21722.
47.
J. Mandel, L. Cobb, and J. D. Beezley, On the convergence of the ensemble Kalman filter, Appl. Math., 56 (2011), pp. 533–541, https://doi.org/10.1007/s10492-011-0031-2.
48.
F. Medina-Aguayo, D. Rudolf, and N. Schweizer, Perturbation bounds for Monte Carlo within Metropolis via restricted approximations, Stochastic Process. Appl., 130 (2020), pp. 2200–2227, https://doi.org/10.1016/j.spa.2019.06.015.
49.
S. P. Meyn and R. L. Tweedie, Stability of Markovian processes I. Criteria for discrete-time chains, Adv. Appl. Probab., 24 (1992), pp. 542–574, https://doi.org/10.2307/1427479.
50.
R. Nickl and E. S. Titi, On posterior consistency of data assimilation with Gaussian process priors: The 2D Navier-Stokes equations, Ann. Statist., 52 (2024), pp. 1825–1844.
51.
D. Ocone and E. Pardoux, Asymptotic stability of the optimal filter with respect to its initial condition, SIAM J. Control Optim., 34 (1996), pp. 226–243, https://doi.org/10.1137/S0363012993256617.
52.
P. Rebeschini and R. van Handel, Can local particle filters beat the curse of dimensionality?, Ann. Appl. Probab., 25 (2015), pp. 2809–2866, https://doi.org/10.1214/14-AAP1061.
53.
S. Reich and C. Cotter, Probabilistic Forecasting and Bayesian Data Assimilation, Cambridge University Press, Cambridge, 2015, https://doi.org/10.1017/CBO9781107706804.
54.
D. Sanz-Alonso and A. M. Stuart, Long-time asymptotics of the filtering distribution for partially observed chaotic dynamical systems, SIAM/ASA J. Uncertain. Quantif., 3 (2015), pp. 1200–1220, https://doi.org/10.1137/140997336.
55.
D. Sanz-Alonso, A. M. Stuart, and A. Taeb, Inverse problems and data assimilation, London Math. Soc. Stud. Texts 107, Cambridge University Press, 2023.
56.
C. Schillings and A. M. Stuart, Analysis of the ensemble Kalman filter for inverse problems, SIAM J. Numer. Anal., 55 (2017), pp. 1264–1290, https://doi.org/10.1137/16M105959X.
57.
C. Schillings and A. M. Stuart, Convergence analysis of ensemble Kalman inversion: The linear, noisy case, Appl. Anal., 97 (2018), pp. 107–123, https://doi.org/10.1080/00036811.2017.1386784.
58.
C. Snyder, T. Bengtsson, P. Bickel, and J. Anderson, Obstacles to high-dimensional particle filtering, Monthly Weather Rev., 136 (2008), pp. 4629–4640.
59.
C. Snyder, T. Bengtsson, and M. Morzfeld, Performance bounds for particle filters using the optimal proposal, Monthly Weather Rev., 143 (2015), pp. 4750–4761.
60.
A. Stuart and A. R. Humphries, Dynamical Systems and Numerical Analysis, Vol. 2, Cambridge University Press, Cambridge, 1998.
61.
I. O. Tolstikhin, B. K. Sriperumbudur, and B. Schölkopf, Minimax estimation of maximum mean discrepancy with radial kernels, Adv. Neural Inf. Process. Syst., 29 (2016).
62.
X. T. Tong, Performance analysis of local ensemble Kalman filter, J. Nonlinear Sci., 28 (2018), pp. 1397–1442, https://doi.org/10.1007/s00332-018-9453-2.
63.
X. T. Tong, A. J. Majda, and D. Kelly, Nonlinear stability of the ensemble Kalman filter with adaptive covariance inflation, Commun. Math. Sci., 14 (2016), pp. 1283–1313, https://doi.org/10.4310/CMS.2016.v14.n5.a5.
64.
X. T. Tong and M. Morzfeld, Localized ensemble Kalman inversion, Inverse Problems, 39 (2023), 064002.
65.
X. T. Tong and R. van Handel, Ergodicity and stability of the conditional distributions of nondegenerate Markov chains, Ann. Appl. Probab., 22 (2012), pp. 1495–1540, https://doi.org/10.1214/11-AAP800.
66.
A. W. van der Vaart, Asymptotic Statistics, Cambridge Series in Statistical and Probabilistic Mathematics 3, Cambridge University Press, Cambridge, 1998, https://doi.org/10.1017/CBO9780511802256.
67.
R. van Handel, Uniform observability of hidden Markov models and filter stability for unstable signals, Ann. Appl. Probab., 19 (2009), pp. 1172–1199, https://doi.org/10.1214/08-AAP576.
68.
C. Villani, Optimal Transport: Old and New, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] 338, Springer-Verlag, Berlin, 2009, https://doi.org/10.1007/978-3-540-71050-9.
Information & Authors
Information
Published In
Copyright
© 2024 Society for Industrial and Applied Mathematics.
History
Submitted: 2 January 2024
Accepted: 26 August 2024
Published online: 15 November 2024
Keywords
MSC codes
Authors
Funding Information
Agence Nationale de la Recherche (ANR): ANR-21-CE40-0006 (SINEQ), ANR-23-CE40-0027 (IPSO)
Office of Naval Research (ONR): N00014-23-1-272
Deutsche Forschungsgemeinschaft (DFG): 390685813 - GZ 2047/1 - HCM
NSF CAREER: 2340762
Engineering and Physical Sciences Research Council (EPSRC): EP/T022132/1, EP/V051121/1
Funding: The first author was supported by the Advanced Grant Nonlocal-CPD (Nonlocal PDEs for Complex Particle Dynamics: Phase Transitions, Patterns, and Synchronization) of the European Research Council (ERC) Executive Agency under the European Union’s Horizon 2020 research and innovation program (grant agreement 883363). The first author was also partially supported by the Engineering and Physical Sciences Research Council (EPSRC) under grants EP/T022132/1 and EP/V051121/1. The second author was supported by start-up funds at the California Institute of Technology, by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) via project 390685813-GZ 2047/1-HCM, and also by NSF CAREER Award 2340762. The work of the third author was supported by a Department of Defense Vannevar Bush Faculty Fellowship and by the SciAI Center, funded by the Office of Naval Research (ONR), under grant N00014-23-1-2729. The fourth author was partially supported by the ERC under the European Union’s Horizon 2020 research and innovation program (grant agreement 810367) and by the Agence Nationale de la Recherche under grants ANR-21-CE40-0006 (SINEQ) and ANR-23-CE40-0027 (IPSO).
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
There are no citations for this item
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].