Inscrição na biblioteca: Guest
Portal Digital Begell Biblioteca digital da Begell eBooks Diários Referências e Anais Coleções de pesquisa
Journal of Machine Learning for Modeling and Computing

ISSN Imprimir: 2689-3967
ISSN On-line: 2689-3975

Journal of Machine Learning for Modeling and Computing

DOI: 10.1615/.2020034232
pages 97-118

LEARNING REDUCED SYSTEMS VIA DEEP NEURAL NETWORKS WITH MEMORY

Xiaohan Fu
Department of Statistics, Ohio State University, Columbus, OH 43210, USA
Lo-Bin Chang
Department of Statistics, Ohio State University, Columbus, OH 43210, USA
Dongbin Xiu
Ohio Eminent Scholar Department of Mathematics The Ohio State University Columbus, Ohio 43210, USA

RESUMO

We present a general numerical approach for constructing governing equations for unknown dynamical systems when data on only a subset of the state variables are available. The unknown equations for these observed variables are thus a reduced system of the complete set of state variables. Reduced systems possess memory integrals, based on the well-known Mori-Zwanzig (MZ) formulation. Our numerical strategy to recover the reduced system starts by formulating a discrete approximation of the memory integral in the MZ formulation. The resulting unknown approximate MZ equations are of finite dimensional, in the sense that a finite number of past history data are involved. We then present a deep neural network structure that directly incorporates the history terms to produce memory in the network. The approach is suitable for any practical systems with finite memory length. We then use a set of numerical examples to demonstrate the effectiveness of our method.

Referências

  1. Bernstein, D., Optimal Prediction of Burgerss Equation, Multiscale Model. Simul, vol. 6, pp. 27-52, 2007.

  2. Brennan, C. and Venturi, D., Data-Driven Closures for Stochastic Dynamical Systems, J. Comput. Phys, vol. 372, pp. 281-298,2018.

  3. Brunton, S.L., Proctor, J.L., and Kutz, J.N., Discovering Governing Equations from Data by Sparse Identification of Nonlinear Dynamical Systems, Proc. Natl. Acad. Sci., vol. 113, no. 15, pp. 3932-3937, 2016.

  4. Chen, Z. and Xiu, D., On Generalized Residue Network for Deep Learning of Unknown Dynamical Systems, J. Comput. Phys, submitted, 2020.

  5. Chertock, A., Gottlieb, D., and Solomonoff, A., Modified Optimal Prediction and Its Application to a Particle-Method Problem, J. Sci. Comput, vol. 37,no.2,pp. 189-201,2008.

  6. Chorin, A.J., Hald, O.H., and Kupferman, R., Optimal Prediction with Memory, Physica D: Nonlinear Phenomena, vol. 166, nos. 3-4, pp. 239-257,2002.

  7. Hald, O. and Stinis, P., Optimal Prediction and the Rate of Decay for Solutions of the Euler Equations in Two and Three Dimensions, Proc. Natl. Acad. Sci., vol. 104, pp. 6527-6532,2007.

  8. He, K., Zhang, X., Ren, S., and Sun, J., Deep Residual Learning for Image Recognition, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778,2016.

  9. Kang, S.H., Liao, W., and Liu, Y., IDENT: Identifying Differential Equations with Numerical Time Evolution, 2019. arXiv: 1904.03538.

  10. Lei, H., Baker, N., and Li, X., Data-Driven Parameterization of the Generalized Langevin Equation, Proc. Natl. Acad. Sci., vol. 113, no. 50, pp. 14183-14188,2016.

  11. Long, Z., Lu, Y., and Dong, B., PDE-Net 2.0: Learning PDEs from Data with Numeric-Symbolic Hybrid Deep Network, 2018a. arXiv: 1812.04426.

  12. Long, Z., Lu, Y., Ma, X., and Dong, B., PDE-Net: Learning PDEs from Data, Proceedings of the 35th International Conference on Machine Learning, Proceedings of Machine Learning Research, J. Dy and A. Krause, Eds., vol. 80, Stockholm, Sweden: PMLR, pp. 3208-3216,2018b.

  13. Mori, H., Transport, Collective Motion, and Brownian Motion, Progress Theor. Phys, vol. 33, no. 3, pp. 423-455,1965.

  14. Pavliotis, G. and Stuart, A., Multiscale Methods: Averaging and Homogenization, Berlin: Springer, 2008.

  15. Qin, T., Chen, Z., Jakeman, J., and Xiu, D., A Neural Network Approach for Uncertainty Quantification for Time-Dependent Problems with Random Parameters, Int. J. Uncertainty Quantif., submitted, 2020.

  16. Qin, T., Wu, K., and Xiu, D., Data Driven Governing Equations Approximation Using Deep Neural Networks, J. Comput. Phys, vol. 395, pp. 620-635,2019a.

  17. Qin, T., Wu, K., and Xiu, D., Structure-Preserving Method for Reconstructing Unknown Hamiltonian Systems from Trajectory Data, SIAMJ. Sci. Comput, submitted, 2019b.

  18. Raissi, M., Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations, J. Mach. Learn. Res, vol. 19, no. 25, pp. 1-24,2018.

  19. Raissi, M., Perdikaris, P., and Karniadakis, G.E., Physics Informed Deep Learning (Part I): Data-Driven Solutions of Nonlinear Partial Differential Equations, 2017a. arXiv: 1711.10561.

  20. Raissi, M., Perdikaris, P., and Karniadakis, G.E., Physics Informed Deep Learning (Part II): Data-Driven Discovery of Nonlinear Partial Differential Equations, 2017b. arXiv: 1711.10566.

  21. Raissi, M., Perdikaris, P., and Karniadakis, G.E., Multistep Neural Networks for Data-Driven Discovery of Nonlinear Dynamical Systems, 2018. arXiv: 1801.01236.

  22. Rudy, S.H., Brunton, S.L., Proctor, J.L., and Kutz, J.N., Data-Driven Discovery of Partial Differential Equations, Sci. Adv., vol. 3, no. 4, p. e1602614,2017.

  23. Rudy, S.H., Kutz, J.N., and Brunton, S.L., Deep Learning of Dynamics and Signal-Noise Decomposition with Time-Stepping Constraints, J. Comput. Phys., vol. 396, pp. 483-506,2019.

  24. Schaeffer, H., Learning Partial Differential Equations via Data Discovery and Sparse Optimization, Proc. Royal Soc. London A: Math. Phys. Eng. Sci, vol. 473, no. 2197,2017.

  25. Schaeffer, H. and McCalla, S.G., Sparse Model Selection via Integral Terms, Phys. Rev. E, vol. 96, no. 2, p. 023302,2017.

  26. Schaeffer, H., Tran, G., and Ward, R., Extracting Sparse High-Dimensional Dynamics from Limited Data, SIAMJ. Appl. Math., vol. 78, no. 6, pp. 3279-3295,2018.

  27. Stinis, P., Higher Order MoriZwanzig Models for the Euler Equations, Multiscale Model. Simul., vol. 6, pp. 741-760,2007.

  28. Sun, Y., Zhang, L., and Schaeffer, H., NEUPDE: Neural Network based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data, 2019. arXiv: 1908.03190.

  29. Tibshirani, R., Regression Shrinkage and Selection via the Lasso, J. R. Stat. Soc. B (Methodological), pp. 267-288,1996.

  30. Tran, G. and Ward, R., Exact Recovery of Chaotic Systems from Highly Corrupted Data, Multiscale Model. Simul., vol. 15, no. 3, pp. 1108-1129,2017.

  31. Venturi, D. and Karniadakis, G., ConvolutionlessNakajima-Zwanzig Equations for Stochastic Analysis in Nonlinear Dynamical Systems, Proc. R. Soc. A, vol. 470, no. 2166, p. 20130754,2014.

  32. Wang, Q., Ripamonti, N., and Hesthaven, J., Recurrent Neural Network Closure of Parametric POD-Galerkin Reduced-Order Models based on the Mori-Zwanzig Formalism, J. Comput. Phys, vol. 410, 2020.

  33. Wu, K., Qin, T., and Xiu, D., Structure-Preserving Method for Reconstructing Unknown Hamiltonian Systems from Trajectory Data, 2019. arXiv: 1905.10396.

  34. Wu, K. and Xiu, D., Numerical Aspects for Approximating Governing Equations Using Data, J. Comput. Phys., vol. 384, pp. 200-221,2019.

  35. Wu, K. and Xiu, D., Data-Driven Deep Learning of Partial Differential Equations in Modal Space, J. Comput. Phys, vol. 408, p. 109307,2020.

  36. Zhu, Y. and Venturi, D., Faber Approximation of the Mori-Zwanzig Equation, J. Comput. Phys, vol. 372, pp. 694-718,2018.

  37. Zwanzig, R., Nonlinear Generalized Langevin Equations, J. Stat. Phys., vol. 9, no. 3, pp. 215-220, 1973.


Articles with similar content:

AN INTEGRAL FORMULATION FOR A RADIATING CYLINDER WITH A CONTINUOUS HEAT SOURCE
International Heat Transfer Conference 10, Vol.17, 1994, issue
P. A. Krutitskii, R. Villasenor
A PRIORI ERROR ANALYSIS OF STOCHASTIC GALERKIN PROJECTION SCHEMES FOR RANDOMLY PARAMETRIZED ORDINARY DIFFERENTIAL EQUATIONS
International Journal for Uncertainty Quantification, Vol.6, 2016, issue 4
Christophe Audouze , Prasanth B. Nair
A Non Split Projection Strategy for Low Mach Number Flows
International Journal for Multiscale Computational Engineering, Vol.2, 2004, issue 3
P.P. Pebay, Habib N. Najm, J. G. Pousin
A STOPPING CRITERION FOR ITERATIVE SOLUTION OF STOCHASTIC GALERKIN MATRIX EQUATIONS
International Journal for Uncertainty Quantification, Vol.6, 2016, issue 3
Christophe Audouze , Pär Håkansson, Prasanth B. Nair
CONCURRENT COUPLING OF BOND-BASED PERIDYNAMICS AND THE NAVIER EQUATION OF CLASSICAL ELASTICITY BY BLENDING
International Journal for Multiscale Computational Engineering, Vol.13, 2015, issue 2
Youn Doh Ha, Samir Beneddine, Pablo Seleson