图书馆订阅: Guest
Journal of Machine Learning for Modeling and Computing

每年出版 4 

ISSN 打印: 2689-3967

ISSN 在线: 2689-3975

Indexed in

DEEP LEARNING OF CHAOTIC SYSTEMS FROM PARTIALLY-OBSERVED DATA

卷 3, 册 3, 2022, pp. 97-119
DOI: 10.1615/JMachLearnModelComput.2022045602
Get accessDownload

摘要

Recently, a general data-driven numerical framework was developed for learning and modeling of unknown dynamical systems using fully- or partially-observed data. The method utilizes deep neural networks (DNNs) to construct a model for the flow map of the unknown system. Once an accurate DNN approximation of the flow map is constructed, it can be recursively executed to serve as an effective predictive model of the unknown system. In this paper, we apply this framework to chaotic systems, in particular the well-known Lorenz 63 and 96 systems, and critically examine the predictive performance of the approach. A distinct feature of chaotic systems is that even the smallest perturbations will lead to large (albeit bounded) deviations in the solution trajectories. This makes long-term predictions of the method, or any data-driven methods, questionable, as the local model accuracy will eventually degrade and lead to large pointwise errors. Here we employ several other qualitative and quantitative measures to determine whether the chaotic dynamics has been learned. These include phase plots, histograms, autocorrelation, correlation dimension, approximate entropy, and Lyapunov exponent. Using these measures, we demonstrate that the flow map based DNN learning method is capable of accurately modeling chaotic systems, even when only a subset of the state variables is available to the DNNs. For example, for the Lorenz 96 system with 40 state variables, when the data of only three variables are available, the method is able to learn an effective DNN model for the three variables and produce accurately the chaotic behavior of the system.

参考文献
  1. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mane, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viegas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., and Zheng, X., TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems, accessed from tensorflow.org, 2015.

  2. Bakarji, J., Champion, K., Kutz, J.N., and Brunton, S.L., Discovering Governing Equations from Partial Measurements with Deep Delay Autoencoders, arXiv: 2201.05136,2022.

  3. Bakker, R., Schouten, J.C., Giles, C.L., Takens, F., and Van den Bleek, C.M., Learning Chaotic Attractors by Neural Networks, Neural Comput, vol. 12, no. 10, pp. 2355-2383,2000.

  4. Bhat, U. and Munch, S.B., Recurrent Neural Networks for Partially Observed Dynamical Systems, Phys. Rev. E, vol. 105, no. 4, p. 044205,2022.

  5. Box, G.E., Jenkins, G.M., Reinsel, G.C., and Ljung, G.M., Time Series Analysis: Forecasting and Control, Hoboken, NJ: John Wiley & Sons, 2015.

  6. Brunton, S.L., Brunton, B.W., Proctor, J.L., Kaiser, E., and Kutz, J.N., Chaos as an Intermittently Forced Linear System, Nat. Commun., vol. 8, p. 19, 2017.

  7. Brunton, S.L., Proctor, J.L., and Kutz, J.N., Discovering Governing Equations from Data by Sparse Identification of Nonlinear Dynamical Systems, Proc. Natl. Acad. Sci. USA, vol. 113, no. 15, pp. 3932-3937, 2016.

  8. Champion, K., Lusch, B., Kutz, J.N., and Brunton, S.L., Data-Driven Discovery of Coordinates and Governing Equations, Proc. Natl. Acad. Sci. USA, vol. 116, no. 45, pp. 22445-22451,2019.

  9. Chattopadhyay, A., Hassanzadeh, P., and Subramanian, D., Data-Driven Predictions of a Multiscale Lorenz 96 Chaotic System Using Machine-Learning Methods: Reservoir Computing, Artificial Neural Network, and Long Short-Term Memory Network, Nonlinear Process. Geophys., vol. 27, no. 3, pp. 373-389, 2020.

  10. Chen, Z., Churchill, V., Wu, K., and Xiu, D., Deep Neural Network Modeling of Unknown Partial Differential Equations in Nodal Space, J. Comput. Phys., vol. 449, p. 110782,2022.

  11. Churchill, V., Manns, S., Chen, Z., and Xiu, D., Robust Modeling of Unknown Dynamical Systems via Ensemble Averaged Learning, arXiv: 2203.03458,2022.

  12. Dubois, P., Gomez, T., Planckaert, L., and Perret, L., Data-Driven Predictions of the Lorenz System, Phys. D: Nonlinear Phenom, vol. 408, p. 132495,2020.

  13. Fu, X., Chang, L.-B., and Xiu, D., Learning Reduced Systems via Deep Neural Networks with Memory, J. Mach. Learn. Model. Comput:, vol. 1, no. 2, pp. 97-118,2020.

  14. Han, M., Shi, Z., and Wang, W., Modeling Dynamic System by Recurrent Neural Network with State Variables, Int. Symposium Neural Networks, Dalian, China, pp. 200-205,2004a.

  15. Han, M., Xi, J., Xu, S., and Yin, F.-L., Prediction of Chaotic Time Series Based on the Recurrent Predictor Neural Network, IEEE Trans. Signal Process., vol. 52, no. 12, pp. 3409-3416,2004b.

  16. He, K., Zhang, X., Ren, S., and Sun, J., Deep Residual Learning for Image Recognition, in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Las Vegas, NV, pp. 770-778,2016.

  17. Kang, S.H., Liao, W., and Liu, Y., IDENT: Identifying Differential Equations with Numerical Time Evolution, arXiv: 1904.03538,2019.

  18. Kim, H., Eykholt, R., and Salas, J., Nonlinear Dynamics, Delay Times, and Embedding Windows, Phys. D: Nonlinear Phenom., vol. 127, nos. 1-2, pp. 48-60,1999.

  19. Kingma, D.P. and Ba, J., Adam: A Method for Stochastic Optimization, arXiv: 1412.6980,2014.

  20. Long, Z., Lu, Y., and Dong, B., PDE-Net 2.0: Learning PDEs from Data with a Numeric-Symbolic Hybrid Deep Network, arXiv: 1812.044267,2018a.

  21. Long, Z., Lu, Y., Ma, X., and Dong, B., PDE-Net: Learning PDEs from Data, in Proc. of the 35th Int. Conf. on Machine Learning, Stockholm, Sweden, pp. 3208-3216,2018b.

  22. Lorenz, E.N., Deterministic Nonperiodic Flow, J. Atmos. Sci., vol. 20, no. 2, pp. 130-141,1963.

  23. Lorenz, E.N., Predictability: A Problem Partly Solved, in Proc. of Seminar on Predictability, Shinfield Park, Reading, UK, 1996.

  24. Lorenz, E.N. and Emanuel, K.A., Optimal Sites for Supplementary Weather Observations: Simulation with a Small Model, J. Atmos. Sci, vol. 55, no. 3, pp. 399-414,1998.

  25. Lu, L., Jin, P., Pang, G., Zhang, Z., and Karniadakis, G.E., Learning Nonlinear Operators via DeepONet Based on the Universal Approximation Theorem of Operators, Nat. Mach. Intell., vol. 3, no. 3, pp. 218-229,2021a.

  26. Lu, L., Meng, X., Mao, Z., and Karniadakis, G.E., DeepXDE: A Deep Learning Library for Solving Differential Equations, SIAMRev., vol. 63, no. 1, pp. 208-228,2021b.

  27. Lusch, B., Kutz, J.N., and Brunton, S.L., Deep Learning for Universal Linear Embeddings of Nonlinear Dynamics, Nat. Commun., vol. 9, no. 1, pp. 1-10,2018.

  28. MATLAB, R2022a, The MathWorks Inc., Natick, MA, 2022.

  29. Miyoshi, T., Ichihashi, H., Okamoto, S., and Hayakawa, T., Learning Chaotic Dynamics in Recurrent RBF Network, in Proc. of ICNN'95-Int. Conf. on Neural Networks, Perth, Australia, pp. 588-593,1995.

  30. Mori, H., Transport, Collective Motion, and Brownian Motion, Prog. Theor. Phys, vol. 33, no. 3, pp. 423-455,1965.

  31. Pan, S. and Duraisamy, K., Data-Driven Discovery of Closure Models, SIAMJ. Appl. Dyn. Syst., vol. 17, no. 4, pp. 2381-2413,2018.

  32. Pawar, S., San, O., Rasheed, A., and Navon, I.M., A Nonintrusive Hybrid Neural-Physics Modeling of Incomplete Dynamical Systems: Lorenz Equations, GEM - Int. J. Geomath, vol. 12, no. 1, pp. 1-31, 2021.

  33. Pincus, S.M., Approximate Entropy as a Measure of System Complexity. Proc. Natl. Acad. Sci., vol. 88, no. 6, pp. 2297-2301,1991.

  34. Qin, T., Wu, K., and Xiu, D., Data Driven Governing Equations Approximation Using Deep Neural Net-works, J. Comput. Phys, vol. 395, pp. 620-635,2019.

  35. Qin, T., Chen, Z., Jakeman, J., and Xiu, D., Data-Driven Learning of Non-Autonomous Systems, SIAM J. Sci. Comput, vol. 43, no. 3, pp. A1607-A1624,2021a.

  36. Qin, T., Chen, Z., Jakeman, J., and Xiu, D., Deep Learning of Parameterized Equations with Applications to Uncertainty Quantification, Int. J. Uncertainty Quantif., vol. 11, no. 2, pp. 63-82,2021b.

  37. Raissi, M., Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations, J. Mach. Learn. Res, vol. 19, no. 25, pp. 1-24,2018.

  38. Raissi, M., Perdikaris, P., and Karniadakis, G.E., Physics Informed Deep Learning (Part I): Data-Driven Solutions of Nonlinear Partial Differential Equations, arXiv: 1711.10561,2017a.

  39. Raissi, M., Perdikaris, P., and Karniadakis, G.E., Physics Informed Deep Learning (Part II): Data-Driven Discovery of Nonlinear Partial Differential Equations, arXiv: 1711.10566,2017b.

  40. Raissi, M., Perdikaris, P., and Karniadakis, G.E., Multistep Neural Networks for Data-Driven Discovery of Nonlinear Dynamical Systems, arXiv: 1801.01236,2018.

  41. Rosenstein, M.T., Collins, J.J., and De Luca, C.J., A Practical Method for Calculating Largest Lyapunov Exponents from Small Data Sets, Phys. D: Nonlinear Phenom., vol. 65, nos. 1-2, pp. 117-134,1993.

  42. Rudy, S.H., Brunton, S.L., Proctor, J.L., and Kutz, J.N., Data-Driven Discovery of Partial Differential Equations, Sci. Adv., vol. 3, no. 4, p. e1602614,2017.

  43. Rudy, S.H., Kutz, J.N., and Brunton, S.L., Deep Learning of Dynamics and Signal-Noise Decomposition with Time-Stepping Constraints, J. Comput. Phys, vol. 396, pp. 483-506,2019.

  44. Schaeffer, H., Learning Partial Differential Equations via Data Discovery and Sparse Optimization, Proc. R. Soc. London A: Math. Phys. Eng. Sci, vol. 473, p. 2197,2017.

  45. Schaeffer, H. and McCalla, S.G., Sparse Model Selection via Integral Terms, Phys. Rev. E, vol. 96, no. 2, p. 023302,2017.

  46. Schaeffer, H., Tran, G., and Ward, R., Extracting Sparse High-Dimensional Dynamics from Limited Data, SIAMJ. Appl. Math., vol. 78, no. 6, pp. 3279-3295,2018.

  47. Scher, S. and Messori, G., Generalization Properties of Feed-Forward Neural Networks Trained on Lorenz Systems, Nonlinear Process. Geophys., vol. 26, no. 4, pp. 381-399,2019.

  48. Sun, Y., Zhang, L., and Schaeffer, H., NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data, arXiv: 1908.03190,2019.

  49. Takens, F., Detecting Strange Attractors in Turbulence, in Dynamical Systems and Turbulence, Warwick 1980, pp. 366-381, Berlin: Springer, 1981.

  50. The MathWorks, Econometrics Toolbox, Natick, MA, 2022a.

  51. The MathWorks, Predictive Maintenance Toolbox, Natick, MA, 2022b.

  52. Theiler, J., Efficient Algorithm for Estimating the Correlation Dimension from a Set of Discrete Points, Phys. Rev. A, vol. 36, no. 9, p. 4456,1987.

  53. Tran, G. and Ward, R., Exact Recovery of Chaotic Systems from Highly Corrupted Data, Multiscale Model.

  54. Simul, vol. 15, no. 3, pp. 1108-1129,2017. Trischler, A.P. and D'Eleuterio, G.M., Synthesis of Recurrent Neural Networks for Dynamical System Simulation, Neural Networks, vol. 80, pp. 67-78,2016.

  55. Vlachas, P. R . , Byeon, W. , Wan, Z . Y. , Sapsis, T. P. , and Koumoutsakos, P. , Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks, Proc. R. Soc. A: Math. Phys. Eng. Sci., vol. 474, no. 2213, p. 20170844,2018. Wang, Q., Ripamonti, N., and Hesthaven, J., Recurrent Neural Network Closure of Parametric POD- Galerkin Reduced-Order Models Based on the Mori-Zwanzig Formalism, J. Comput. Phys, vol. 410, p. 109402,2020.

  56. Wu, K. and Xiu, D., Data-Driven Deep Learning of Partial Differential Equations in Modal Space, J. Comput. Phys, vol. 408, p. 109307,2020.

  57. Wulkow, N., Koltai, P., Sunkara, V., and Schutte, C., Data-Driven Modelling of Nonlinear Dynamics by.

  58. Barycentric Coordinates and Memory, arXiv: 2112.06742,2021. Zimmermann, H.G. and Neuneier, R., Modeling Dynamical Systems by Recurrent Neural Networks, WIT Trans. Inf. Commun. Technol., vol. 25, 2000.

  59. Zwanzig, R., Nonlinear Generalized Langevin Equations, J. Stat. Phys., vol. 9, no. 3, pp. 215-220,1973.

对本文的引用
  1. Churchill Victor, Xiu Dongbin, LEARNING FINE SCALE DYNAMICS FROM COARSE OBSERVATIONS VIA INNER RECURRENCE, Journal of Machine Learning for Modeling and Computing, 3, 3, 2022. Crossref

Begell Digital Portal Begell 数字图书馆 电子图书 期刊 参考文献及会议录 研究收集 订购及政策 Begell House 联系我们 Language English 中文 Русский Português German French Spain