Abstract

Data-driven prediction is becoming increasingly widespread as the volume of data available grows and as algorithmic development matches this growth. The nature of the predictions made and the manner in which they should be interpreted depend crucially on the extent to which the variables chosen for prediction are Markovian or approximately Markovian. Multiscale systems provide a framework in which this issue can be analyzed. In this work kernel analog forecasting methods are studied from the perspective of data generated by multiscale dynamical systems. The problems chosen exhibit a variety of different Markovian closures, using both averaging and homogenization; furthermore, settings where scale separation is not present and the predicted variables are non-Markovian are also considered. The studies provide guidance for the interpretation of data-driven prediction methods when used in practice.

Keywords

  1. data-driven prediction
  2. multiscale systems
  3. kernel methods
  4. analog forecasting
  5. averaging
  6. homogenization

MSC codes

  1. 37M10
  2. 34E13
  3. 58J65

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
R. Alexander and D. Giannakis, Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques, Phys. D, 409 (2020), 132520, https://doi.org/10.1016/j.physd.2020.132520.
2.
H. Arbabi and I. Mezic, Ergodic theory, dynamic mode decomposition, and computation of spectral properties of the koopman operator, SIAM J. Appl. Dyn. Syst., 16 (2017), pp. 2096--2126, https://doi.org/10.1137/17M1125236.
3.
N. Aronszajn, Theory of reproducing kernels, Trans. Amer. Math. Soc., 68 (1950), pp. 337--404.
4.
M. Belkin and P. Niyogi, Laplacian eigenmaps for dimensionality reduction and data representation, Neural Comput., 15 (2003), pp. 1373--1396, https://doi.org/10.1162/089976603321780317.
5.
A. Bensoussan, J.-L. Lions, and G. Papanicolaou, Asymptotic Analysis for Periodic Structures, Vol. 374, American Mathematical Society, Providence, RI, 2011.
6.
T. Berry, D. Giannakis, and J. Harlim, Nonparametric forecasting of low-dimensional dynamical systems, Phys. Rev. E, 91 (2015), 032915, https://doi.org/10.1103/PhysRevE.91.032915.
7.
T. Berry and J. Harlim, Variable bandwidth diffusion kernels, Appl. Comput. Harmon. Anal., 40 (2016), pp. 68--96, https://doi.org/10.1016/j.acha.2015.01.001.
8.
A. Chattopadhyay, E. Nabizadeh, and P. Hassanzadeh, Analog forecasting of extreme-causing weather patterns using deep learning, J. Adv. Model. Earth Syst., 12 (2020), e2019MS001958, https://doi.org/10.1029/2019MS001958.
9.
R. Coifman and M. Hirn, Bi-stochastic kernels via asymmetric affinity functions, Appl. Comput. Harmon. Anal., 35 (2013), pp. 177--180, https://doi.org/10.1016/j.acha.2013.01.001.
10.
R. R. Coifman and S. Lafon, Diffusion maps, Appl. Comput. Harmon. Anal., 21 (2006), pp. 5--30, https://doi.org/10.1016/j.acha.2006.04.006.
11.
R. R. Coifman and S. Lafon, Geometric harmonics: A novel tool for multiscale out-of-sample extension of empirical functions, Appl. Comput. Harmon. Anal., 21 (2006), pp. 31--52, https://doi.org/10.1016/j.acha.2005.07.005.
12.
S. Das and D. Giannakis, Delay-coordinate maps and the spectra of Koopman operators, J. Stat. Phys., 175 (2019), pp. 1107--1145, https://doi.org/10.1007/s10955-019-02272-w.
13.
S. Das, D. Giannakis, and J. Slawinska, Reproducing kernel Hilbert space compactification of unitary evolution groups, Appl. Comput. Harmon. Anal., 54 (2021), pp. 75--136, https://doi.org/10.1016/j.acha.2021.02.004.
14.
L. Delle Monache, F. A. Eckel, D. L. Rife, B. Nagarajan, and K. Searight, Probabilistic weather prediction with an analog ensemble, Monthly Weather Rev., 141 (2013), pp. 3498--3516.
15.
M. Dellnitz and O. Junge, On the approximation of complicated dynamical behavior, SIAM J. Numer. Anal., 36 (1999), pp. 491--515, https://doi.org/10.1137/S0036142996313002.
16.
I. Fatkullin and E. Vanden-Eijnden, A computational strategy for multiscale systems with applications to Lorenz 96 model, J. Comput. Phys., 200 (2004), pp. 605--638, https://doi.org/10.1016/j.jcp.2004.04.013.
17.
D. Giannakis, Data-driven spectral decomposition and forecasting of ergodic dynamical systems, Appl. Comput. Harmon. Anal., 62 (2019), pp. 338--396, https://doi.org/10.1016/j.acha.2017.09.001.
18.
D. Giannakis, A. Kolchinskaya, D. Krasnov, and J. Schumacher, Koopman analysis of the long-term evolution in a turbulent convection cell, J. Fluid Mech., 847 (2018), pp. 735--767, https://doi.org/10.1017/jfm.2018.297.
19.
D. Givon, R. Kupferman, and A. Stuart, Extracting macroscopic dynamics: Model problems and algorithms, Nonlinearity, 17 (2004), R55, https://doi.org/10.1088/0951-7715/17/6/R01.
20.
F. Hamilton, T. Berry, and T. Sauer, Ensemble Kalman filtering without a model, Phys. Rev. X, 6 (2016), 011021, https://doi.org/10.1103/PhysRevX.6.011021.
21.
J. Harlim, S. W. Jiang, S. Liang, and H. Yang, Machine learning for prediction with missing dynamics, J. Comput. Phys., 428 (2021), 109922, https://doi.org/10.1016/j.jcp.2020.109922.
22.
S. W. Jiang and J. Harlim, Modeling of Missing Dynamical Systems: Deriving Parametric Models Using a Nonparametric Framework, Res. Math. Sci., 7 (2020), pp. 1--25, https://doi.org/10.1007/s40687-020-00217-4.
23.
D. Kelly and I. Melbourne, Deterministic homogenization for fast--slow systems with chaotic noise, J. Funct. Anal., 272 (2017), pp. 4063--4102, https://doi.org/10.1016/j.jfa.2017.01.015.
24.
D. Kelly and I. Melbourne, Smooth approximation of stochastic differential equations, Ann. Probab., 44 (2016), pp. 479--520, https://doi.org/10.1214/14-AOP979.
25.
M. A. Khodkar, P. Hassanzadeh, and A. Antoulas, A Koopman-Based Framework for Forecasting the Spatiotemporal Evolution of Chaotic Dynamics with Nonlinearities Modeled as Exogenous Forcings, preprint, arXiv:1909.00076, 2019.
26.
S. Klus, F. Nüske, P. Koltai, H. Wu, I. Kevrekidis, C. Schütte, and F. Noé, Data-driven model reduction and transfer operator approximation, J. Nonlinear Sci., 28 (2018), pp. 985--1010, https://doi.org/10.1007/s00332-017-9437-7.
27.
B. O. Koopman, Hamiltonian systems and transformation in Hilbert space, Proc. Natl. Acad. Sci. USA, 17 (1931), p. 315.
28.
M. Korda, M. Putinar, and I. Mezić, Data-driven spectral analysis of the Koopman operator, Appl. Comput. Harmon. Anal., 48 (2020), pp. 599--629, https://doi.org/10.1016/j.acha.2018.08.002, in press.
29.
E. N. Lorenz, Deterministic nonperiodic flow, J. Atmos. Sci., 20 (1963), pp. 130--141, https://doi.org/10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2.
30.
E. N. Lorenz, Atmospheric predictability as revealed by naturally occurring analogues, J. Atmos. Sci., 26 (1969), pp. 636--646, https://doi.org/10.1175/1520-0469(1969)26<636:APARBN>2.0.CO;2.
31.
E. N. Lorenz, Predictability: A problem partly solved, in Proceedings of the Seminar on Predictability, Vol. 1, ECMWF, 1996.
32.
I. Melbourne and M. Nicol, Almost sure invariance principle for nonuniformly hyperbolic systems, Commun. Math. Phys., 260 (2005), pp. 131--146, https://doi.org/10.1007/s00220-005-1407-5.
33.
I. Melbourne and A. Stuart, A note on diffusion limits of chaotic skew-product flows, Nonlinearity, 24 (2011), pp. 1361--1367, https://doi.org/10.1088/0951-7715/24/4/018.
34.
I. Mezić, Spectral properties of dynamical systems, model reduction and decompositions, Nonlinear Dyn., 41 (2005), pp. 309--325, https://doi.org/10.1007/s11071-005-2824-x.
35.
I. Mezić, Analysis of fluid flows via spectral properties of the koopman operator, Annu. Rev. Fluid Mech., 45 (2013), pp. 357--378, https://doi.org/10.1146/annurev-fluid-011212-140652.
36.
B. Nadler, S. Lafon, R. R. Coifman, and I. G. Kevrekidis, Diffusion maps, spectral clustering and reaction coordinates of dynamical systems, Appl. Comput. Harmon. Anal., 21 (2006), pp. 113--127, https://doi.org/10.1016/j.acha.2005.07.004.
37.
G. C. Papanicolaou and W. Kohler, Asymptotic theory of mixing stochastic ordinary differential equations, Commun. Pure Appl. Math., 27 (1974), pp. 641--668, https://doi.org/10.1002/cpa.3160270503.
38.
G. Pavliotis and A. Stuart, Multiscale Methods: Averaging and Homogenization, Springer-Verlag, Berlin, 2008, https://doi.org/10.1007/978-0-387-73829-1.
39.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, Scikit-learn: Machine learning in Python, J. Mach. Learn. Res., 12 (2011), pp. 2825--2830.
40.
C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning), MIT Press, Cambridge, MA, 2005.
41.
C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter, and D. S. Henningson, Spectral analysis of nonlinear flows, J. Fluid Mech., 641 (2009), pp. 115--127, https://doi.org/10.1017/S0022112009992059.
42.
P. J. Schmid, Dynamic mode decomposition of numerical and experimental data, J. Fluid Mech., 656 (2010), pp. 5--28, https://doi.org/10.1017/S0022112010001217.
43.
B. Schölkopf, A. Smola, and K. Müller, Nonlinear component analysis as a kernel eigenvalue problem, Neural Comput., 10 (1998), pp. 1299--1319, https://doi.org/10.1162/089976698300017467.
44.
B. Schölkopf, A. J. Smola, and F. Bach, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, MIT Press, Cambridge, MA, 2002.
45.
A. Singer, From graph to manifold laplacian: The convergence rate, Appl. Comput. Harmon. Anal., 21 (2006), pp. 128--134, https://doi.org/10.1016/j.acha.2006.03.004.
46.
J. Slawinska and D. Giannakis, Indo-Pacific variability on seasonal to multidecadal time scales. Part I: Intrinsic SST modes in models and observations, J. Climate, 30 (2017), pp. 5265--5294, https://doi.org/10.1175/JCLI-D-16-0176.1.
47.
N. G. Trillos, M. Gerlach, M. Hein, and D. Slepčev, Error estimates for spectral convergence of the graph Laplacian on random geometric graphs towards the Laplace--Beltrami operator, Found. Comput. Math., 20 (2020), pp. 827--887, https://doi.org/10.1007/s10208-019-09436-w.
48.
W. Tucker, The Lorenz attractor exists, C. R. Acad. Sci. Ser. I Math., 328 (1999), pp. 1197--1202, https://doi.org/10.1016/S0764-4442(99)80439-X.
49.
H. Van den Dool, A new look at weather forecasting through analogues, Monthly Weather Rev., 117 (1989), pp. 2230--2247, https://doi.org/10.1175/1520-0493(1989)117<2230:ANLAWF>2.0.CO;2.
50.
U. Von Luxburg, M. Belkin, and O. Bousquet, Consistency of spectral clustering, Ann. Statist., (2008), pp. 555--586, http://doi.org/10.1214/009053607000000640.
51.
G. Wahba, Spline Models for Observational Data, Vol. 59, SIAM, Philadelphia, 1990.
52.
X. Wang, J. Slawinska, and D. Giannakis, Extended-range statistical ENSO prediction through operator-theoretic techniques for nonlinear dynamics, Sci. Rep., 10 (2020), 2636, https://doi.org/10.1038/s41598-020-59128-7.
53.
E. Weinan, Principles of Multiscale Modeling, Cambridge University Press, Cambridge, 2011.
54.
D. S. Wilks, Effects of stochastic parametrizations in the Lorenz \textup'96 system, Quart. J. Roy. Meteorol. Soc., 131 (2005), pp. 389--407, https://doi.org/10.1256/qj.04.03.
55.
Z. Zhao and D. Giannakis, Analog forecasting with dynamics-adapted kernels, Nonlinearity, 29 (2016), 2888, https://doi.org/10.1088/0951-7715/29/9/2888.

Information & Authors

Information

Published In

cover image Multiscale Modeling & Simulation
Multiscale Modeling & Simulation
Pages: 1011 - 1040
ISSN (online): 1540-3467

History

Submitted: 18 May 2020
Accepted: 2 February 2021
Published online: 16 June 2021

Keywords

  1. data-driven prediction
  2. multiscale systems
  3. kernel methods
  4. analog forecasting
  5. averaging
  6. homogenization

MSC codes

  1. 37M10
  2. 34E13
  3. 58J65

Authors

Affiliations

Funding Information

Schmidt Futures Program
Earthrise Allicance
Mountain Philanthropies
Office of Naval Research https://doi.org/10.13039/100000006 : N00014-17-1-2079
Office of Naval Research https://doi.org/10.13039/100000006 : N00014-16-1-2649, N00014-19-1-242
National Science Foundation https://doi.org/10.13039/100000001 : AGS1835860
National Science Foundation https://doi.org/10.13039/100000001 : 1842538, DMS-1854383
National Science Foundation https://doi.org/10.13039/100000001 : 1803663
National Science Foundation https://doi.org/10.13039/100000001 : DMS-1818977
Paul G. Allen Family Foundation https://doi.org/10.13039/100000952

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media

The SIAM Publications Library now uses SIAM Single Sign-On for individuals. If you do not have existing SIAM credentials, create your SIAM account https://my.siam.org.