Methods and Algorithms for Scientific Computing

Symplectic Model Reduction of Hamiltonian Systems on Nonlinear Manifolds and Approximation with Weakly Symplectic Autoencoder

Abstract.

Classical model reduction techniques project the governing equations onto linear subspaces of the high-dimensional state-space. For problems with slowly decaying Kolmogorov- \(n\) -widths such as certain transport-dominated problems, however, classical linear-subspace reduced-order models (ROMs) of low dimension might yield inaccurate results. Thus, the concept of classical linear-subspace ROMs has to be extended to more general concepts, like model order teduction (MOR) on manifolds. Moreover, as we are dealing with Hamiltonian systems, it is crucial that the underlying symplectic structure is preserved in the reduced model, as otherwise it could become unphysical in the sense that the energy is not conserved or stability properties are lost. To the best of our knowledge, existing literature addresses either MOR on manifolds or symplectic model reduction for Hamiltonian systems, but not their combination. In this work, we bridge the two aforementioned approaches by providing a novel projection technique called symplectic manifold Galerkin (SMG), which projects the Hamiltonian system onto a nonlinear symplectic trial manifold such that the reduced model is again a Hamiltonian system. We derive analytical results such as stability, energy-preservation, and a rigorous a posteriori error bound. Moreover, we construct a weakly symplectic deep convolutional autoencoder as a computationally practical approach to approximate a nonlinear symplectic trial manifold. Finally, we numerically demonstrate the ability of the method to achieve higher accuracy than (non-)structure-preserving linear-subspace ROMs and non-structure-preserving MOR on manifold techniques.

Keywords

  1. symplectic model reduction
  2. Hamiltonian systems
  3. energy preservation
  4. stability preservation
  5. nonlinear dimension reduction
  6. autoencoders

MSC codes

  1. 65P10
  2. 34C20
  3. 37J25
  4. 37M15
  5. 37N30

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
R. Abraham and J. E. Marsden, Foundations of Mechanics. 2nd ed., CRC Press, Boca Raton, FL, 1987.
2.
C. Beattie, S. Gugercin, and V. Mehrmann, Structure-preserving interpolatory model reduction for port-Hamiltonian differential-algebraic systems, in Realization and Model Reduction of Dynamical Systems, Springer, Cham, 2022, pp. 235–254.
3.
P. Benner, M. Ohlberger, A. Cohen, and K. Willcox, Model Reduction and Approximation: Theory and Algorithms, Comput. Sci. Eng. 15, SIAM, Philadelphia, 2017, https://doi.org/10.1137/1.9781611974829.
4.
A. Bhatt and B. E. Moore, Structure-preserving exponential Runge–Kutta methods, SIAM J. Sci. Comput., 39 (2017), pp. A593–A612, https://doi.org/10.1137/16M1071171.
5.
F. Black, P. Schulze, and B. Unger, Projection-based model reduction with dynamically transformed modes, ESAIM Math. Model. Numer. Anal., 54 (2020), pp. 2011–2043.
6.
P. Buchfink, A. Bhatt, and B. Haasdonk, Symplectic model order reduction with non-orthonormal bases, Math. Comput. Appl., 24 (2019), 43
7.
P. Buchfink, B. Haasdonk, and S. Rave, PSD-greedy basis generation for structure-preserving model order reduction of Hamiltonian systems, in Proceedings of ALGORITMY, 2020, pp. 151–160.
8.
N. Cagniart, Y. Maday, and B. Stamm, Model order reduction for problems with large convection effects, in Contributions to Partial Differential Equations and Applications, Springer, Cham, 2019, pp. 131–150.
9.
A. Cannas Da Silva, Lectures on Symplectic Geometry, Springer, New York, 2008.
10.
K. Carlberg, Adaptive h-refinement for reduced-order models, Internat. J. Numer. Methods Engrg., 102 (2015), pp. 1192–1210.
11.
K. Carlberg, M. Barone, and H. Antil, Galerkin v. least-squares Petrov-Galerkin projection in nonlinear model reduction, J. Comput. Phys., 330 (2017), pp. 693–734.
12.
K. Carlberg, C. Bou-Mosleh, and C. Farhat, Efficient non-linear model reduction via a least-squares Petrov-Galerkin projection and compressive tensor approximations, Internat. J. Numer. Methods Engrg., 86 (2011), pp. 155–181.
13.
K. Carlberg, R. Tuminaro, and P. Boggs, Preserving Lagrangian structure in nonlinear model reduction with application to structural dynamics, SIAM J. Sci. Comput., 37 (2015), pp. B153–B184, https://doi.org/10.1137/140959602.
14.
S. Chaturantabut, C. Beattie, and S. Gugercin, Structure-preserving model reduction for nonlinear port-Hamiltonian systems, SIAM J. Sci. Comput., 38 (2016), pp. B837–B865, https://doi.org/10.1137/15M1055085.
15.
M. Cruz Varona, Model Reduction of Nonlinear Dynamical Systems by System-Theoretic Methods, Dissertation, Technische Universität München, München, Germany, 2020.
16.
G. Deco and W. Brauer, Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures, Neural Netw., 8 (1995), pp. 525–535.
17.
V. Ehrlacher, D. Lombardi, O. Mula, and F.-X. Vialard, Nonlinear model reduction on metric spaces. Application to one-dimensional conservative PDEs in Wasserstein spaces, ESAIM Math. Model. Numer. Anal., 54 (2020), pp. 2159–2197.
18.
S. Fresca, L. Dedé, and A. Manzoni, A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEs, J. Sci. Comput., 87 (2021), pp. 1–36.
19.
S. Glas, A. T. Patera, and K. Urban, A reduced basis method for the wave equation, Int. J. Comput. Fluid Dyn., 34 (2020), pp. 139–146.
20.
X. Glorot and Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, PMLR, 2010, pp. 249–256.
21.
I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, MIT Press, Cambridge, MA, 2016.
22.
C. Greif and K. Urban, Decay of the Kolmogorov n-width for wave problems, Appl. Math. Lett., 96 (2019), pp. 216–222.
23.
C. Gu, Model Order Reduction of Nonlinear Dynamical Systems, Dissertation, University of California Berkeley, Berkeley, CA, 2011.
24.
E. Hairer, C. Lubich, and G. Wanner, Geometric Numerical Integration: Structure-Preserving Algorithms for Ordinary Differential Equations, 2nd ed., Springer Ser. Comput. Math. 31, Springer, Berlin, 2006.
25.
K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification, in Proceedings of the IEEE International Conference on Computer Vision, IEEE, Washington, DC, 2015, pp. 1026–1034.
26.
J. S. Hesthaven and C. Pagliantini, Structure-preserving reduced basis methods for Poisson systems, Math. Comput., 90 (2021), pp. 1701–1740.
27.
J. S. Hesthaven, C. Pagliantini, and N. Ripamonti, Rank-Adaptive Structure-Preserving Reduced Basis Methods for Hamiltonian Systems, preprint, https://arxiv.org/abs/2007.13153, 2020.
28.
G. E. Hinton and R. R. Salakhutdinov, Reducing the dimensionality of data with neural networks, Science, 313 (2006), pp. 504–507.
29.
M. Hutchinson, A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines, Commun. Stat. Sim. Comput., 18 (1989), pp. 1059–1076.
30.
A. Iollo and D. Lombardi, Advection modes by optimal mass transfer, Phys. Rev. E, 89 (2014), 022923.
31.
P. Jin, Z. Zhang, A. Zhu, Y. Tang, and G. Karniadakis, Sympnets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems, Neural Netw., 132 (2020), pp. 166–179.
32.
Y. Kim, Y. Choi, D. Widemann, and T. Zohdi, A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder, J. Comput. Phys., 451 (2022), 110841.
33.
D. P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, preprint, https://arxiv.org/abs/1412.6980, 2014.
34.
T. Kohonen, Self-organized formation of topologically correct feature maps, Biol. Cybernet., 43 (1982), pp. 59–69.
35.
K. Kunisch and S. Volkwein, Galerkin proper orthogonal decomposition methods for parabolic problems, Numer. Math., 90 (2001), pp. 117–148.
36.
S. Lall, P. Krysl, and J. E. Marsden, Structure-preserving model reduction for mechanical systems, Phys. D, 184 (2003), pp. 304–318.
37.
N. D. Lawrence, Gaussian process latent variable models for visualisation of high dimensional data, in Neural Information Processing Systems, Vancouver, Canada, 2003, pp. 329–336.
38.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition, Proc. IEEE, 86 (1998), pp. 2278–2324.
39.
J. M. Lee, Introduction to Smooth Manifolds, Springer-Verlag New York, 2003.
40.
K. Lee and K. Carlberg, Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders, J. Comput. Phys., 404 (2020), 108973.
41.
K. Lee and K. T. Carlberg, Deep conservation: A latent-dynamics model for exact satisfaction of physical conservation laws, in Proceedings of the 35th AAAI Conference on Artificial Intelligence, 2021, pp. 277–285.
42.
B. Maboudi Afkham and J. S. Hesthaven, Structure preserving model reduction of parametric Hamiltonian systems, SIAM J. Sci. Comput., 39 (2017), pp. A2616–A2644, https://doi.org/10.1137/17M1111991.
43.
J. E. Marsden and T. S. Ratiu, Introduction to Mechanics and Symmetry, 2nd ed., Texts Appl. Math. 17, Springer, New York, 1999.
44.
K. Meyer and D. Offin, Introduction to Hamiltonian Dynamical Systems and the N-Body Problem. 3rd ed., Springer, Cham, 2017.
45.
P. J. Morrison, The Maxwell-Vlasov equations as a continuous Hamiltonian system, Phys. Lett. A, 80 (1980), pp. 383–386.
46.
M. Ohlberger and S. Rave, Nonlinear reduced basis approximation of parameterized evolution equations via the method of freezing, C. R. Math. Acad. Sci. Paris, 351 (2013), pp. 901–906.
47.
M. Ohlberger and S. Rave, Reduced basis methods: Success, limitations and future challenges, in Proceedings of ALGORITMY, 2016, pp. 1–12.
48.
M. Ohlberger and F. Schindler, Error control for the localized reduced basis multiscale method with adaptive on-line enrichment, SIAM J. Sci. Comput., 37 (2015), pp. A2865–A2895, https://doi.org/10.1137/151003660.
49.
C. Pagliantini, Dynamical reduced basis methods for Hamiltonian systems, Numer. Math., 148 (2021), pp. 409–448.
50.
A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, PyTorch: An imperative style, high-performance deep learning library, in Neural Information Processing Systems, 2019, pp. 8026–8037.
51.
B. Peherstorfer, Model reduction for transport-dominated problems via online adaptive bases and adaptive sampling, SIAM J. Sci. Comput., 42 (2020), pp. A2803–A2836, https://doi.org/10.1137/19M1257275.
52.
B. Peherstorfer and K. Willcox, Online adaptive model reduction for nonlinear systems via low-rank updates, SIAM J. Sci. Comput., 37 (2015), pp. A2123–A2150, https://doi.org/10.1137/140989169.
53.
L. Peng and K. Mohseni, Symplectic model reduction of Hamiltonian systems, SIAM J. Sci. Comput., 38 (2016), pp. A1–A27, https://doi.org/10.1137/140978922.
54.
R. V. Polyuga and A. Van der Schaft, Structure preserving model reduction of port-Hamiltonian systems by moment matching at infinity, Automatica J. IFAC, 46 (2010), pp. 665–672.
55.
J. Reiss, P. Schulze, J. Sesterhenn, and V. Mehrmann, The shifted proper orthogonal decomposition: A mode decomposition for multiple transport phenomena, SIAM J. Sci. Comput., 40 (2018), pp. A1322–A1344, https://doi.org/10.1137/17M1140571.
56.
D. Rim, B. Peherstorfer, and K. Mandli, Manifold Approximations via Transported Subspaces: Model Reduction for Transport-Dominated Problems, preprint, https://arxiv.org/abs/1912.13024, 2019.
57.
B. Schölkopf, A. Smola, and K.-R. Müller, Nonlinear component analysis as a kernel eigenvalue problem, Neural Comput., 10 (1998), pp. 1299–1319.
58.
H. Sharma, Z. Wang, and B. Kramer, Hamiltonian operator inference: Physics-preserving learning of reduced-order models for canonical Hamiltonian systems, Phys. D, 431 (2022), 133122.
59.
T. Taddei and L. Zhang, Space-time registration-based model reduction of parameterized one-dimensional hyperbolic PDEs, ESAIM Math. Model. Numer. Anal., 55 (2021), pp. 99–130.
60.
M. Uzunca, B. Karasözen, and S. Yildiz, Structure-preserving reduced-order modeling of Korteweg-de Vries equation, Math. Comput. Simul., 188 (2021), pp. 193–211.
61.
C. Walder and B. Schölkopf, Diffeomorphic dimensionality reduction, in Neural Information Processing Systems, Vancouver, Canada, 2008, pp. 1713–1720.
62.
G. Welper, Interpolation of functions with parameter dependent jumps by transformed snapshots, SIAM J. Sci. Comput., 39 (2017), pp. A1225–A1250, https://doi.org/10.1137/16M1059904.
63.
T. Wenzel, M. Kurz, A. Beck, G. Santin, and B. Haasdonk, Structured deep kernel networks for data-driven closure terms of turbulent flows, in Large-Scale Scientific Computing, I. Lirkov and S. Margenov, eds., Springer, Cham, 2022, pp. 410–418.
64.
H. Xu, A numerical method for computing an SVD-like decomposition, SIAM J. Matrix Anal. Appl., 26 (2005), pp. 1058–1082, https://doi.org/10.1137/S0895479802410529.

Information & Authors

Information

Published In

cover image SIAM Journal on Scientific Computing
SIAM Journal on Scientific Computing
Pages: A289 - A311
ISSN (online): 1095-7197

History

Submitted: 20 December 2021
Accepted: 15 September 2022
Published online: 20 March 2023

Keywords

  1. symplectic model reduction
  2. Hamiltonian systems
  3. energy preservation
  4. stability preservation
  5. nonlinear dimension reduction
  6. autoencoders

MSC codes

  1. 65P10
  2. 34C20
  3. 37J25
  4. 37M15
  5. 37N30

Authors

Affiliations

Patrick Buchfink
Institute of Applied Analysis and Numerical Simulation, University of Stuttgart, Pfaffenwaldring 57, 70569 Stuttgart, Germany.
Department of Applied Mathematics, University of Twente, P.O. Box 217, 7500 AE Enschede, The Netherlands.
Bernard Haasdonk
Institute of Applied Analysis and Numerical Simulation, University of Stuttgart, Pfaffenwaldring 57, 70569 Stuttgart, Germany.

Funding Information

Stuttgart Center for Simulation Science (SimTech)
Funding:The work of the second author was supported by the Simons Foundation in the Collaboration on Hidden Symmetries and Fusion Energy. The work of the first and third authors was supported by Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy - EXC 2075 - 390740016, and by the Stuttgart Center for Simulation Science (SimTech).

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

View Options

View options

PDF

View PDF

Full Text

View Full Text

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media