Learning the Dynamics for Unknown Hyperbolic Conservation Laws Using Deep Neural Networks
Abstract.
We propose a new data-driven method to learn the dynamics of an unknown hyperbolic system of conservation laws using deep neural networks. Inspired by classical methods in numerical conservation laws, we develop a new conservative form network (CFN) in which the network learns to approximate the numerical flux function of the unknown system. Our numerical examples demonstrate that the CFN yields significantly better prediction accuracy than what is obtained using a standard nonconservative form network, even when it is enhanced with constraints to promote conservation. In particular, solutions obtained using the CFN consistently capture the correct shock propagation speed without introducing nonphysical oscillations into the solution. They are furthermore robust to noisy and sparse observation environments.
Keywords
MSC codes
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems, 2015 software available online from https://www.tensorflow.org.
2.
P. J. Baddoo, B. Herrmann, B. J. McKeon, J. N. Kutz, and S. L. Brunton, Physics-Informed Dynamic Mode Decomposition piDMD, preprint, arXiv:2112.04307, 2021.
3.
A. D. Beck, J. Zeifang, A. Schwarz, and D. G. Flad, A neural network based shock detection and localization approach for discontinuous Galerkin methods, J. Comput. Phys., 423 (2020), 109824.
4.
T. Bertalan, F. Dietrich, I. Mezić, and I. G. Kevrekidis, On learning Hamiltonian systems from data, Chaos, 29 (2019), 121107.
5.
D. A. Bezgin, S. J. Schmidt, and N. A. Adams, A data-driven physics-informed finite-volume scheme for nonclassical undercompressive shocks, J. Comput. Phys., 437 (2021), 110324.
6.
C. M. Bishop, Training with noise is equivalent to Tikhonov regularization, Neural Comput., 7 (1995), pp. 108–116.
7.
S. L. Brunton, J. L. Proctor, and J. N. Kutz, Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proc. Natl. Acad. Sci. USA, 113 (2016), pp. 3932–3937.
8.
Z. Chen, V. Churchill, K. Wu, and D. Xiu, Deep neural network modeling of unknown partial differential equations in nodal space, J. Comput. Phys., 449 (2022), 110782.
9.
Z. Chen and D. Xiu, On generalized residual network for deep learning of unknown dynamical systems, J. Comput. Phys., 438 (2021), 110362.
10.
Clawpack Development Team, Clawpack Software, Version 5.7.1, 2020; available online from http://www.clawpack.org.
11.
M. Cranmer, S. Greydanus, S. Hoyer, P. Battaglia, D. Spergel, and S. Ho, Lagrangian neural networks, in ICLR Workshop on Integration of Deep Neural Models and Differential Equations, 2020.
12.
X. Fu, W. Mao, L.-B. Chang, and D. Xiu, Modeling unknown dynamical systems with hidden parameters, J. Mach. Learn. Model. Comput., 3 (2022), pp. 79–95.
13.
G. H. Golub, P. C. Hansen, and D. P. O’Leary, Tikhonov regularization and total least squares, SIAM J. Matrix Anal. Appl., 21 (1999), pp. 185–194, https://doi.org/10.1137/S0895479897326432.
14.
S. Gottlieb and C.-W. Shu, Total variation diminishing Runge-Kutta schemes, Math. Comp., 67 (1998), pp. 73–85.
15.
S. Greydanus, M. Dzamba, and J. Yosinski, Hamiltonian neural networks, in Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), 2019.
16.
P. C. Hansen, Discrete Inverse Problems: Insight and Algorithms, SIAM, Philadelphia, 2010, https://doi.org/10.1137/1.9780898718836.
17.
K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.
18.
J. S. Hesthaven, Numerical Methods for Conservation Laws: From Analysis to Algorithms, SIAM, Philadelphia, 2018, https://doi.org/10.1137/1.9781611975109.
19.
A. D. Jagtap and G. E. Karniadakis, Extended physics-informed neural networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations, in Proceedings of the AAAI Spring Symposium on Combining Artificial Intelligence and Machine Learning with Physical Sciences, Stanford, CA, 2021, and CEUR Workshop Proceedings 2964, J. Lee, E. F. Darve, P. K. Kitanidis, M. W. Mahoney, A. Karpatne, M. W. Farthing, and T. J. Hesser, eds., 2021; available online from https://ceur-ws.org.
20.
A. D. Jagtap, E. Kharazmi, and G. E. Karniadakis, Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems, Comput. Methods Appl. Mech. Engrg., 365 (2020), 113028.
21.
G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, Physics-informed machine learning, Nat. Rev. Phys., 3 (2021), pp. 422–440.
22.
D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, in Proceedings of the International Conference on Learning Representations (ICLR 2015), 2015.
23.
Y. LeCun, Y. Bengio, and G. Hinton, Deep learning, Nature, 521 (2015), pp. 436–444.
24.
R. J. LeVeque, Numerical Methods for Conservation Laws, Lectures Math. ETH Zürich, Birkhäuser Verlag, Basel, 1992.
25.
R. J. LeVeque, Finite Volume Methods for Hyperbolic Problems, Cambridge Texts Appl. Math. 31, Cambridge University Press, Cambridge, UK, 2002.
26.
Z. Li, N. B. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. M. Stuart, and A. Anandkumar, Fourier neural operator for parametric partial differential equations, in Proceedings of the 9th International Conference on Learning Representations (ICLR 2021), 2021; available online from https://openreview.net.
27.
Z. Li, M. Liu-Schiaffini, N. B. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. M. Stuart, and A. Anandkumar, Learning chaotic dynamics in dissipative systems, in Proceedings of the 36rd Conference on Neural Information Processing Systems (NeurIPS 2019), 2019.
28.
Z. Liu, V. Madhavan, and M. Tegmark, Machine learning conservation laws from differential equations, Phys. Rev. E (3), 106 (2022), 045307, https://doi.org/10.1103/PhysRevE.106.045307.
29.
Z. Long, Y. Lu, X. Ma, and B. Dong, PDE-Net: Learning PDEs from data, in Proceedings of the International Conference on Machine Learning (PMLR 2018), 2018, pp. 3208–3216.
30.
L. Lu, P. Jin, G. Pang, Z. Zhang, and G. E. Karniadakis, Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, Nat. Mach. Intell., 3 (2021), pp. 218–229.
31.
H. Mori, Transport, collective motion, and Brownian motion, Prog. Theor. Phys., 33 (1965), pp. 423–455.
32.
T. Qin, Z. Chen, J. D. Jakeman, and D. Xiu, Data-driven learning of nonautonomous systems, SIAM J. Sci. Comput., 43 (2021), pp. A1607–A1624, https://doi.org/10.1137/20M1342859.
33.
T. Qin, Z. Chen, J. D. Jakeman, and D. Xiu, Deep learning of parameterized equations with applications to uncertainty quantification, Int. J. Uncertain. Quantif., 11 (2021), pp. 63–82.
34.
T. Qin, K. Wu, and D. Xiu, Data driven governing equations approximation using deep neural networks, J. Comput. Phys., 395 (2019), pp. 620–635.
35.
M. Raissi, Deep hidden physics models: Deep learning of nonlinear partial differential equations, J. Mach. Learn. Res., 19 (2018), pp. 932–955.
36.
M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378 (2019), pp. 686–707.
37.
R. Rodriguez-Torrado, P. Ruiz, L. Cueto-Felgueroso, M. C. Green, T. Friesen, S. Matringe, and J. Togelius, Physics-informed attention-based neural network for hyperbolic partial differential equations: Application to the Buckley–Leverett problem, Sci. Rep., 12 (2022), 7557.
38.
S. H. Rudy, S. L. Brunton, J. L. Proctor, and J. N. Kutz, Data-driven discovery of partial differential equations, Sci. Adv., 3 (2017), e1602614.
39.
H. Schaeffer, Learning partial differential equations via data discovery and sparse optimization, Proc. A, 473 (2017), 20160446.
40.
H. Schaeffer, G. Tran, and R. Ward, Extracting sparse high-dimensional dynamics from limited data, SIAM J. Appl. Math., 78 (2018), pp. 3279–3295, https://doi.org/10.1137/18M116798X.
41.
C.-W. Shu and S. Osher, Efficient implementation of essentially non-oscillatory shock-capturing schemes, J. Comput. Phys., 77 (1988), pp. 439–471.
42.
B. Stevens and T. Colonius, Enhancement of shock-capturing methods via machine learning, Theor. Comput. Fluid Dyn., 34 (2020), pp. 483–496.
43.
J. H. Tu, Dynamic Mode Decomposition: Theory and Applications, Ph.D. thesis, Princeton University, Princeton, NJ, 2013.
44.
K. Wu and D. Xiu, Data-driven deep learning of partial differential equations in modal space, J. Comput. Phys., 408 (2020), 109307.
45.
S. Zhang and G. Lin, Robust data-driven discovery of governing physical laws with error bars, Proc. A, 474 (2018), 20180305.
46.
Z. Zhang, Y. Shin, and G. Em Karniadakis, GFINNs: GENERIC formalism informed neural networks for deterministic and stochastic dynamical systems, Philos. Trans. R. Soc. A, 380 (2022), 20210207.
47.
Y. D. Zhong, B. Dey, and A. Chakraborty, Symplectic ODE-Net: Learning Hamiltonian dynamics with control, in Proceedings of the 8th International Conference on Learning Representations (ICLR 2020), Addis Ababa, Ethiopia, 2020; available online from https://openreview.net.
48.
R. Zwanzig, Nonlinear generalized Langevin equations, J. Statist. Phys., 9 (1973), pp. 215–220.
Information & Authors
Information
Published In
SIAM Journal on Scientific Computing
Pages: A825 - A850
DOI: 10.1137/22M1537333
ISSN (online): 1095-7197
Copyright
© 2024 Society for Industrial and Applied Mathematics.
History
Submitted: 28 November 2022
Accepted: 10 November 2023
Published online: 6 March 2024
Keywords
MSC codes
Authors
Funding Information
Multidisciplinary University Research Initiative (MURI): ONR N00014-20-1-2595
Air Force Office of Scientific Research (AFOSR): FA9550-22-1-0411
National Science Foundation (NSF): DMS 1912685
National Science Foundation (NSF): DMS 1912999
Funding: This work was supported by DoD MURI grant ONR N00014-20-1-2595. The second author’s research was also supported by AFOSR grant FA9550-22-1-0411 and NSF grant DMS 1912685. The third author’s research was also supported by NSF grant DMS 1912999.
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited By
- GoRINNs: Godunov-Riemann informed neural networks for learning hyperbolic conservation lawsJournal of Computational Physics, Vol. 534 | 1 Aug 2025
- Principal component flow map learning of PDEs from incomplete, limited, and noisy dataJournal of Computational Physics, Vol. 524 | 1 Mar 2025
- A TVD neural network closure and application to turbulent combustionJournal of Computational Physics, Vol. 523 | 1 Feb 2025
View Options
- Access via your Institution
- Questions about how to access this content? Contact SIAM at [email protected].