Abstract.

We propose a neural network-based approach to the homogenization of multiscale problems. The proposed method uses a derivative-free formulation of a training loss, which incorporates Brownian walkers to find the macroscopic description of a multiscale PDE solution. Compared with other network-based approaches for multiscale problems, the proposed method is free from the design of hand-crafted neural network architecture and the cell problem to calculate the homogenization coefficient. The exploration neighborhood of the Brownian walkers affects the overall learning trajectory. We determine the bounds of micro- and macro-time steps that capture the local heterogeneous and global homogeneous solution behaviors, respectively, through a neural network. The bounds imply that the computational cost of the proposed method is independent of the microscale periodic structure for the standard periodic problems. We validate the efficiency and robustness of the proposed method through a suite of linear and nonlinear multiscale problems with periodic and random field coefficients.

Keywords

  1. multiscale problems
  2. homogenization
  3. neural network
  4. derivative-free

MSC codes

  1. 65N99
  2. 65C05
  3. 68T07

Get full access to this article

View all available purchase options and get full access to this article.

References

1.
A. Abdulle, W. E. B. Engquist, and E. Vanden-Eijnden, The heterogeneous multiscale method, Acta Numer., 21 (2012), pp. 1–87.
2.
A. Bensoussan, J. L. Lions, and G. Papanicolaou, Asymptotic Analysis for Periodic Structures, AMS Chelsea Publishing, 2011.
3.
P. Bhattacharyya and B. K. Chakrabarti, The mean distance to the nth neighbour in a uniform distribution of random points: An application of probability theory, European J. Phys., 29 (2008), 639.
4.
W. Cai and Z.-Q. J. Xu, Multi-scale Deep Neural Networks for Solving High Dimensional PDEs, https://arxiv.org/abs/1910.11710, 2019.
5.
S. P. Carney, A. Gangal, and L. Kim, Physics Informed Neural Networks for Elliptic Equations with Oscillatory Differential Operators, https://arxiv.org/abs/2212.13531, 2022.
6.
W. E. and B. Yu, The deep Ritz method: A deep learning-based numerical algorithm for solving variational problems, Commun. Math. Stat., 6 (2018), pp. 1–12.
7.
Y. Efendiev, J. Galvis, and T. Y. Hou, Generalized multiscale finite element methods (GMsFEM), J. Comput. Phys., 251 (2013), pp. 116–135.
8.
B. Engquist and Y.-H. Tsai, Heterogeneous multiscale methods for stiff ordinary differential equations, Math. Comp., 74 (2005), pp. 1707–1742.
9.
X. Glorot and Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 2010, pp. 249–256.
10.
J. Han, A. Jentzen, and E. Weinan, Solving high-dimensional partial differential equations using deep learning, Proc. Natl. Acad. Sci. USA, 115 (2018), pp. 8505–8510.
11.
J. Han and Y. Lee, Hierarchical Learning to Solve Partial Differential Equations Using Physics-Informed Neural Networks, https://arxiv.org/abs/2112.01254, 2021.
12.
J. Han, M. Nica, and A. R. Stinchcombe, A derivative-free method for solving elliptic partial differential equations with deep neural networks, J. Comput. Phys., 419 (2020), 109672.
13.
A. Jacot, F. Gabriel, and C. Hongler, Neural tangent kernel: Convergence and generalization in neural networks, in Proceedings of the 32nd International Conference on Neural Information Processing Systems, 2018, pp. 8580–8589.
14.
A. D. Jagtap, K. Kawaguchi, and G. E. Karniadakis, Adaptive activation functions accelerate convergence in deep and physics-informed neural networks, J. Comput. Phys., 404 (2020), 109136.
15.
S. Jin, Asymptotic preserving (AP) schemes for multiscale kinetic and hyperbolic equations: A review, in Lecture Notes for Summer School on Methods and Models of Kinetic Theory (M&MKT), Porto Ercole, Grosseto, Italy, 2010, pp. 177–216.
16.
I. J. Jordaan, D. G. Matskevitch, and I. L. Meglis, Disintegration of ice under fast compressive loading, Int. J. Fract., 97 (1999), pp. 279–300.
17.
I. Karatzas and S. Shreve, Brownian Motion and Stochastic Calculus, 2nd ed., Grad. Texts in Math. 113, Springer, 1998.
18.
D. P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, https://arxiv.org/abs/1412.6980, 2014.
19.
F. Kröpfl, R. Maier, and D. Peterseim, Neural Network Approximation of Coarse-scale Surrogates in Numerical Homogenization, https://arxiv.org/abs/2209.02624, 2022.
20.
Y. Lee and B. Engquist, Variable step size multiscale methods for stiff and highly oscillatory dynamical systems, Discrete Contin. Dyn. Syst, 34 (2014), pp. 1079–1097.
21.
Y. Lee and B. Engquist, Multiscale numerical methods for advection-diffusion in incompressible turbulent flow fields, J. Comput. Phys., 317 (2016), pp. 33–46.
22.
W. T. Leung, G. Lin, and Z. Zhang, NH-PINN: Neural homogenization-based physics-informed neural network for multiscale problems, J. Comput. Phys., 470 (2022), 111539.
23.
J.-E. Lundstern and M. D. Zoback, Multiscale variations of the crustal stress field throughout North America, Nat. Commun., 11 (2020), 1951.
24.
K. M. S. Park and A. R. Stinchcombe, Deep reinforcement learning of viscous incompressible flow, J. Comput. Phys., 467 (2022), 111455.
25.
J. Pedlosky, Geophysical Fluid Dynamics, 2nd ed., Springer, 1987.
26.
H. Pitsch, Large-eddy simulation of turbulent combustion, Annu. Rev. Fluid Mech., 38 (2006), pp. 453–482.
27.
N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio, and A. Courville, On the spectral bias of neural networks, in International Conference on Machine Learning, PMLR, 2019, pp. 5301–5310.
28.
M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378 (2019), pp. 686–707.
29.
J. A. Sanders and F. Verhulst, Averaging Methods in Nonlinear Dynamical Systems, Appl. Math. Sci. 59, Springer-Verlag, New York, Berlin, Heidelberg, Tokyo, 1985.
30.
J. Sirignano and K. Spiliopoulos, DGM: A deep learning algorithm for solving partial differential equations, J. Comput. Phys., 375 (2018), pp. 1339–1364.
31.
S. Wang, Y. Teng, and P. Perdikaris, Understanding and mitigating gradient flow pathologies in physics-informed neural networks, SIAM J. Sci. Comput., 43 (2021), pp. A3055–A3081, https://doi.org/10.1137/20M1318043.
32.
S. Wang, H. Wang, and P. Perdikaris, On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks, Comput. Methods Appl. Mech. Engrg., 384 (2021), 113938.
33.
S. Wang, X. Yu, and P. Perdikaris, When and why PINNs fail to train: A neural tangent kernel perspective, J. Comput. Phys., 449 (2022), 110768.
34.
Z.-Q. J. Xu, Y. Zhang, T. Luo, Y. Xiao, and Z. Ma, Frequency principle: Fourier analysis sheds light on deep neural networks, Commun. Comput. Phys., 28 (2020), pp. 1746–1767.
35.
Z.-Q. J. Xu, Y. Zhang, and Y. Xiao, Training behavior of deep neural network in frequency domain, in International Conference on Neural Information Processing, Springer, 2019, pp. 264–274.

Information & Authors

Information

Published In

cover image Multiscale Modeling & Simulation
Multiscale Modeling & Simulation
Pages: 716 - 734
ISSN (online): 1540-3467

History

Submitted: 6 June 2022
Accepted: 6 February 2023
Published online: 1 June 2023

Keywords

  1. multiscale problems
  2. homogenization
  3. neural network
  4. derivative-free

MSC codes

  1. 65N99
  2. 65C05
  3. 68T07

Authors

Affiliations

Department of Mathematics, Dartmouth College, Hanover, NH 03755 USA.
Yoonsang Lee
Department of Mathematics, Dartmouth College, Hanover, NH 03755 USA.

Funding Information

Funding: This work was supported by NSF DMS-1912999 and ONR MURI N00014-20-1-2595.

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

View Options

View options

PDF

View PDF

Full Text

View Full Text

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media