Kernel Analog Forecasting: Multiscale Test Problems
Abstract
Data-driven prediction is becoming increasingly widespread as the volume of data available grows and as algorithmic development matches this growth. The nature of the predictions made and the manner in which they should be interpreted depend crucially on the extent to which the variables chosen for prediction are Markovian or approximately Markovian. Multiscale systems provide a framework in which this issue can be analyzed. In this work kernel analog forecasting methods are studied from the perspective of data generated by multiscale dynamical systems. The problems chosen exhibit a variety of different Markovian closures, using both averaging and homogenization; furthermore, settings where scale separation is not present and the predicted variables are non-Markovian are also considered. The studies provide guidance for the interpretation of data-driven prediction methods when used in practice.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
R. Alexander and D. Giannakis, Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques, Phys. D, 409 (2020), 132520, https://doi.org/10.1016/j.physd.2020.132520.
2.
H. Arbabi and I. Mezic, Ergodic theory, dynamic mode decomposition, and computation of spectral properties of the koopman operator, SIAM J. Appl. Dyn. Syst., 16 (2017), pp. 2096--2126, https://doi.org/10.1137/17M1125236.
3.
N. Aronszajn, Theory of reproducing kernels, Trans. Amer. Math. Soc., 68 (1950), pp. 337--404.
4.
M. Belkin and P. Niyogi, Laplacian eigenmaps for dimensionality reduction and data representation, Neural Comput., 15 (2003), pp. 1373--1396, https://doi.org/10.1162/089976603321780317.
5.
A. Bensoussan, J.-L. Lions, and G. Papanicolaou, Asymptotic Analysis for Periodic Structures, Vol. 374, American Mathematical Society, Providence, RI, 2011.
6.
T. Berry, D. Giannakis, and J. Harlim, Nonparametric forecasting of low-dimensional dynamical systems, Phys. Rev. E, 91 (2015), 032915, https://doi.org/10.1103/PhysRevE.91.032915.
7.
T. Berry and J. Harlim, Variable bandwidth diffusion kernels, Appl. Comput. Harmon. Anal., 40 (2016), pp. 68--96, https://doi.org/10.1016/j.acha.2015.01.001.
8.
A. Chattopadhyay, E. Nabizadeh, and P. Hassanzadeh, Analog forecasting of extreme-causing weather patterns using deep learning, J. Adv. Model. Earth Syst., 12 (2020), e2019MS001958, https://doi.org/10.1029/2019MS001958.
9.
R. Coifman and M. Hirn, Bi-stochastic kernels via asymmetric affinity functions, Appl. Comput. Harmon. Anal., 35 (2013), pp. 177--180, https://doi.org/10.1016/j.acha.2013.01.001.
10.
R. R. Coifman and S. Lafon, Diffusion maps, Appl. Comput. Harmon. Anal., 21 (2006), pp. 5--30, https://doi.org/10.1016/j.acha.2006.04.006.
11.
R. R. Coifman and S. Lafon, Geometric harmonics: A novel tool for multiscale out-of-sample extension of empirical functions, Appl. Comput. Harmon. Anal., 21 (2006), pp. 31--52, https://doi.org/10.1016/j.acha.2005.07.005.
12.
S. Das and D. Giannakis, Delay-coordinate maps and the spectra of Koopman operators, J. Stat. Phys., 175 (2019), pp. 1107--1145, https://doi.org/10.1007/s10955-019-02272-w.
13.
S. Das, D. Giannakis, and J. Slawinska, Reproducing kernel Hilbert space compactification of unitary evolution groups, Appl. Comput. Harmon. Anal., 54 (2021), pp. 75--136, https://doi.org/10.1016/j.acha.2021.02.004.
14.
L. Delle Monache, F. A. Eckel, D. L. Rife, B. Nagarajan, and K. Searight, Probabilistic weather prediction with an analog ensemble, Monthly Weather Rev., 141 (2013), pp. 3498--3516.
15.
M. Dellnitz and O. Junge, On the approximation of complicated dynamical behavior, SIAM J. Numer. Anal., 36 (1999), pp. 491--515, https://doi.org/10.1137/S0036142996313002.
16.
I. Fatkullin and E. Vanden-Eijnden, A computational strategy for multiscale systems with applications to Lorenz 96 model, J. Comput. Phys., 200 (2004), pp. 605--638, https://doi.org/10.1016/j.jcp.2004.04.013.
17.
D. Giannakis, Data-driven spectral decomposition and forecasting of ergodic dynamical systems, Appl. Comput. Harmon. Anal., 62 (2019), pp. 338--396, https://doi.org/10.1016/j.acha.2017.09.001.
18.
D. Giannakis, A. Kolchinskaya, D. Krasnov, and J. Schumacher, Koopman analysis of the long-term evolution in a turbulent convection cell, J. Fluid Mech., 847 (2018), pp. 735--767, https://doi.org/10.1017/jfm.2018.297.
19.
D. Givon, R. Kupferman, and A. Stuart, Extracting macroscopic dynamics: Model problems and algorithms, Nonlinearity, 17 (2004), R55, https://doi.org/10.1088/0951-7715/17/6/R01.
20.
F. Hamilton, T. Berry, and T. Sauer, Ensemble Kalman filtering without a model, Phys. Rev. X, 6 (2016), 011021, https://doi.org/10.1103/PhysRevX.6.011021.
21.
J. Harlim, S. W. Jiang, S. Liang, and H. Yang, Machine learning for prediction with missing dynamics, J. Comput. Phys., 428 (2021), 109922, https://doi.org/10.1016/j.jcp.2020.109922.
22.
S. W. Jiang and J. Harlim, Modeling of Missing Dynamical Systems: Deriving Parametric Models Using a Nonparametric Framework, Res. Math. Sci., 7 (2020), pp. 1--25, https://doi.org/10.1007/s40687-020-00217-4.
23.
D. Kelly and I. Melbourne, Deterministic homogenization for fast--slow systems with chaotic noise, J. Funct. Anal., 272 (2017), pp. 4063--4102, https://doi.org/10.1016/j.jfa.2017.01.015.
24.
D. Kelly and I. Melbourne, Smooth approximation of stochastic differential equations, Ann. Probab., 44 (2016), pp. 479--520, https://doi.org/10.1214/14-AOP979.
25.
M. A. Khodkar, P. Hassanzadeh, and A. Antoulas, A Koopman-Based Framework for Forecasting the Spatiotemporal Evolution of Chaotic Dynamics with Nonlinearities Modeled as Exogenous Forcings, preprint, arXiv:1909.00076, 2019.
26.
S. Klus, F. Nüske, P. Koltai, H. Wu, I. Kevrekidis, C. Schütte, and F. Noé, Data-driven model reduction and transfer operator approximation, J. Nonlinear Sci., 28 (2018), pp. 985--1010, https://doi.org/10.1007/s00332-017-9437-7.
27.
B. O. Koopman, Hamiltonian systems and transformation in Hilbert space, Proc. Natl. Acad. Sci. USA, 17 (1931), p. 315.
28.
M. Korda, M. Putinar, and I. Mezić, Data-driven spectral analysis of the Koopman operator, Appl. Comput. Harmon. Anal., 48 (2020), pp. 599--629, https://doi.org/10.1016/j.acha.2018.08.002, in press.
29.
E. N. Lorenz, Deterministic nonperiodic flow, J. Atmos. Sci., 20 (1963), pp. 130--141, https://doi.org/10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2.
30.
E. N. Lorenz, Atmospheric predictability as revealed by naturally occurring analogues, J. Atmos. Sci., 26 (1969), pp. 636--646, https://doi.org/10.1175/1520-0469(1969)26<636:APARBN>2.0.CO;2.
31.
E. N. Lorenz, Predictability: A problem partly solved, in Proceedings of the Seminar on Predictability, Vol. 1, ECMWF, 1996.
32.
I. Melbourne and M. Nicol, Almost sure invariance principle for nonuniformly hyperbolic systems, Commun. Math. Phys., 260 (2005), pp. 131--146, https://doi.org/10.1007/s00220-005-1407-5.
33.
I. Melbourne and A. Stuart, A note on diffusion limits of chaotic skew-product flows, Nonlinearity, 24 (2011), pp. 1361--1367, https://doi.org/10.1088/0951-7715/24/4/018.
34.
I. Mezić, Spectral properties of dynamical systems, model reduction and decompositions, Nonlinear Dyn., 41 (2005), pp. 309--325, https://doi.org/10.1007/s11071-005-2824-x.
35.
I. Mezić, Analysis of fluid flows via spectral properties of the koopman operator, Annu. Rev. Fluid Mech., 45 (2013), pp. 357--378, https://doi.org/10.1146/annurev-fluid-011212-140652.
36.
B. Nadler, S. Lafon, R. R. Coifman, and I. G. Kevrekidis, Diffusion maps, spectral clustering and reaction coordinates of dynamical systems, Appl. Comput. Harmon. Anal., 21 (2006), pp. 113--127, https://doi.org/10.1016/j.acha.2005.07.004.
37.
G. C. Papanicolaou and W. Kohler, Asymptotic theory of mixing stochastic ordinary differential equations, Commun. Pure Appl. Math., 27 (1974), pp. 641--668, https://doi.org/10.1002/cpa.3160270503.
38.
G. Pavliotis and A. Stuart, Multiscale Methods: Averaging and Homogenization, Springer-Verlag, Berlin, 2008, https://doi.org/10.1007/978-0-387-73829-1.
39.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, Scikit-learn: Machine learning in Python, J. Mach. Learn. Res., 12 (2011), pp. 2825--2830.
40.
C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning), MIT Press, Cambridge, MA, 2005.
41.
C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter, and D. S. Henningson, Spectral analysis of nonlinear flows, J. Fluid Mech., 641 (2009), pp. 115--127, https://doi.org/10.1017/S0022112009992059.
42.
P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech., 656 (2010), pp. 5--28, https://doi.org/10.1017/S0022112010001217.
43.
B. Schölkopf, A. Smola, and K. Müller, Nonlinear component analysis as a kernel eigenvalue problem, Neural Comput., 10 (1998), pp. 1299--1319, https://doi.org/10.1162/089976698300017467.
44.
B. Schölkopf, A. J. Smola, and F. Bach, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, MIT Press, Cambridge, MA, 2002.
45.
A. Singer, From graph to manifold laplacian: The convergence rate, Appl. Comput. Harmon. Anal., 21 (2006), pp. 128--134, https://doi.org/10.1016/j.acha.2006.03.004.
46.
J. Slawinska and D. Giannakis, Indo-Pacific variability on seasonal to multidecadal time scales. Part I: Intrinsic SST modes in models and observations, J. Climate, 30 (2017), pp. 5265--5294, https://doi.org/10.1175/JCLI-D-16-0176.1.
47.
N. G. Trillos, M. Gerlach, M. Hein, and D. Slepčev, Error estimates for spectral convergence of the graph Laplacian on random geometric graphs towards the Laplace--Beltrami operator, Found. Comput. Math., 20 (2020), pp. 827--887, https://doi.org/10.1007/s10208-019-09436-w.
48.
W. Tucker, The Lorenz attractor exists, C. R. Acad. Sci. Ser. I Math., 328 (1999), pp. 1197--1202, https://doi.org/10.1016/S0764-4442(99)80439-X.
49.
H. Van den Dool, A new look at weather forecasting through analogues, Monthly Weather Rev., 117 (1989), pp. 2230--2247, https://doi.org/10.1175/1520-0493(1989)117<2230:ANLAWF>2.0.CO;2.
50.
U. Von Luxburg, M. Belkin, and O. Bousquet, Consistency of spectral clustering, Ann. Statist., (2008), pp. 555--586, http://doi.org/10.1214/009053607000000640.
51.
G. Wahba, Spline Models for Observational Data, Vol. 59, SIAM, Philadelphia, 1990.
52.
X. Wang, J. Slawinska, and D. Giannakis, Extended-range statistical ENSO prediction through operator-theoretic techniques for nonlinear dynamics, Sci. Rep., 10 (2020), 2636, https://doi.org/10.1038/s41598-020-59128-7.
53.
E. Weinan, Principles of Multiscale Modeling, Cambridge University Press, Cambridge, 2011.
54.
D. S. Wilks, Effects of stochastic parametrizations in the Lorenz \textup'96 system, Quart. J. Roy. Meteorol. Soc., 131 (2005), pp. 389--407, https://doi.org/10.1256/qj.04.03.
55.
Z. Zhao and D. Giannakis, Analog forecasting with dynamics-adapted kernels, Nonlinearity, 29 (2016), 2888, https://doi.org/10.1088/0951-7715/29/9/2888.
Information & Authors
Information
Published In
Copyright
© 2021, Society for Industrial and Applied Mathematics.
History
Submitted: 18 May 2020
Accepted: 2 February 2021
Published online: 16 June 2021
Keywords
MSC codes
Authors
Funding Information
Schmidt Futures Program
Earthrise Allicance
Mountain Philanthropies
Office of Naval Research https://doi.org/10.13039/100000006 : N00014-17-1-2079
Office of Naval Research https://doi.org/10.13039/100000006 : N00014-16-1-2649, N00014-19-1-242
National Science Foundation https://doi.org/10.13039/100000001 : AGS1835860
National Science Foundation https://doi.org/10.13039/100000001 : 1842538, DMS-1854383
National Science Foundation https://doi.org/10.13039/100000001 : 1803663
National Science Foundation https://doi.org/10.13039/100000001 : DMS-1818977
Paul G. Allen Family Foundation https://doi.org/10.13039/100000952
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- FlightKoopman: Deep Koopman for Multi-Dimensional Flight Trajectory PredictionInternational Journal of Computational Intelligence and Applications, Vol. 24 | 19 February 2025
- The Effects of Unresolved Scales on Analogue Forecasting EnsemblesStochastic Transport in Upper Ocean Dynamics III | 9 August 2024
- Deep Embedding Koopman Neural Operator-Based Nonlinear Flight Training Trajectory Prediction ApproachMathematics, Vol. 12, No. 14 | 10 July 2024
- Learning nonparametric ordinary differential equations from noisy dataJournal of Computational Physics, Vol. 507 | 1 Jun 2024
- Physics-informed and data-driven discovery of governing equations for complex phenomena in heterogeneous mediaPhysical Review E, Vol. 109, No. 4 | 4 April 2024
- Quantum Mechanics for Closure of Dynamical SystemsMultiscale Modeling & Simulation, Vol. 22, No. 1 | 5 February 2024
- Ensemble forecasts in reproducing kernel Hilbert space familyPhysica D: Nonlinear Phenomena, Vol. 459 | 1 Mar 2024
- Rigorous data‐driven computation of spectral properties of Koopman operators for dynamical systemsCommunications on Pure and Applied Mathematics, Vol. 77, No. 1 | 27 July 2023
- The multiverse of dynamic mode decomposition algorithmsNumerical Analysis Meets Machine Learning | 1 Jan 2024
- Operator inference with roll outs for learning reduced models from scarce and low-quality dataComputers & Mathematics with Applications, Vol. 145 | 1 Sep 2023
- Active Operator Inference for Learning Low-Dimensional Dynamical-System Models from Noisy DataSIAM Journal on Scientific Computing, Vol. 45, No. 4 | 5 July 2023
- Multiscale extensions for enhancing coarse grid computationsJournal of Computational and Applied Mathematics, Vol. 427 | 1 Aug 2023
- Learning to Forecast Dynamical Systems from Streaming DataSIAM Journal on Applied Dynamical Systems, Vol. 22, No. 2 | 5 May 2023
- The mpEDMD Algorithm for Data-Driven Computations of Measure-Preserving Dynamical SystemsSIAM Journal on Numerical Analysis, Vol. 61, No. 3 | 16 June 2023
- Data assimilation in operator algebrasProceedings of the National Academy of Sciences, Vol. 120, No. 8 | 17 February 2023
- Ensemble Kalman inversion for sparse learning of dynamical systems from time-averaged dataJournal of Computational Physics, Vol. 470 | 1 Dec 2022
- A framework for machine learning of model error in dynamical systemsCommunications of the American Mathematical Society, Vol. 2, No. 7 | 14 October 2022
- Modern Koopman Theory for Dynamical SystemsSIAM Review, Vol. 64, No. 2 | 5 May 2022
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].