Begell House Inc.
Journal of Machine Learning for Modeling and Computing
JMLMC
2689-3967
1
1
2020
TENSOR BASIS GAUSSIAN PROCESS MODELS OF HYPERELASTIC MATERIALS
1-17
10.1615/JMachLearnModelComput.2020033325
Ari L.
Frankel
Sandia National Laboratories, Livermore, California 94551, USA
Reese E.
Jones
Sandia National Laboratories, Livermore, California, 94551, USA
Laura P.
Swiler
Optimization and Uncertainty Quantification Department, Center for Computing Research, Sandia National Laboratories, P.O. Box 5800, Albuquerque, New Mexico
87123-1320, USA
Gaussian process regression
hyperelastic materials
physics-informed machine learning
In this work, we develop Gaussian process regression (GPR) models of isotropic hyperelastic material behavior. First, we consider the direct approach of modeling the components of the Cauchy stress tensor as a function of the components of the Finger stretch tensor in a Gaussian process. We then consider an improvement on this approach that embeds rotational invariance of the stress-stretch constitutive relation in the GPR representation. This approach requires fewer training examples and achieves higher accuracy while maintaining invariance to rotations exactly. Finally, we consider an approach that recovers the strain-energy density function and derives the stress tensor from this potential. Although the error of this model for predicting the stress tensor is higher, the strain-energy density is recovered with high accuracy from limited training data. The approaches presented here are examples of physics-informed machine learning. They go beyond purely data-driven approaches by embedding the physical system constraints directly into the Gaussian process representation of materials models.
LIMITATIONS OF PHYSICS INFORMED MACHINE LEARNING FOR NONLINEAR TWO-PHASE TRANSPORT IN POROUS MEDIA
19-37
10.1615/JMachLearnModelComput.2020033905
Olga
Fuks
Department of Energy Resources Engineering, Stanford University, 367 Panama
Street, Stanford, California 94305, USA
Hamdi A.
Tchelepi
Department of Energy Resources Engineering, Stanford University, 367 Panama
Street, Stanford, California 94305, USA
two-phase transport
physics informed machine learning
partial differential equations
Deep learning techniques have recently been applied to a wide range of computational physics problems. In this paper, we focus on developing a physics-based approach that enables the neural network to learn the solution of a dynamic fluid-flow problem governed by a nonlinear partial differential equation (PDE). The main idea of physics informed machine learning (PIML) approaches is to encode the underlying physical law (i.e., the PDE) into the neural network as prior information. We investigate the applicability of the PIML approach to the forward problem of immiscible two-phase fluid transport in porous media, which is governed by a nonlinear first-order hyperbolic PDE subject to initial and boundary data. We employ the PIML strategy to solve this forward problem without any additional labeled data in the interior of the domain. Particularly, we are interested in non-convex flux functions in the PDE, where the solution involves shocks and mixed waves (shocks and rarefactions). We have found that such a PIML approach fails to provide reasonable approximations to the solution in the presence of shocks in the saturation field. We investigated several architectures and experimented with a large number of neural-network parameters, and the overall finding is that PIML strategies that employ the nonlinear hyperbolic conservation equation in the loss function are inadequate. However, we have found that employing a parabolic form of the conservation equation, whereby a small amount of diffusion is added, the neural network is consistently able to learn accurate approximation of the solutions containing shocks and mixed waves.
TRAINABILITY OF ReLU NETWORKS AND DATA-DEPENDENT INITIALIZATION
39-74
10.1615/JMachLearnModelComput.2020034126
Yeonjong
Shin
Division of Applied Mathematics, Brown University, Providence,
Rhode Island 02912, USA
George Em
Karniadakis
School of Engineering, Brown University, Providence, Rhode Island, 02912,
USA; Division of Applied Mathematics, Brown University, Providence, Rhode Island,
02912, USA
ReLU networks
trainability
dying ReLU
overparameterization
over-specification
data-dependent initialization
In this paper we study the trainability of rectified linear unit (ReLU) networks at initialization. A ReLU neuron is said to be dead if it only outputs a constant for any input. Two death states of neurons are introduced−tentative and permanent death. A network is then said to be trainable if the number of permanently dead neurons is sufficiently small for a learning task. We refer to the probability of a randomly initialized network being trainable as trainability. We show that a network being trainable is a necessary condition for successful training, and the trainability serves as an upper bound of training success rates. In order to quantify the trainability, we study the probability distribution of the number of active neurons at initialization. In many applications, overspecified or overparameterized neural networks are successfully employed and shown to be trained effectively. With the notion of trainability, we show that overparameterization is both a necessary and a sufficient condition for achieving a zero training loss. Furthermore, we propose a data-dependent initialization method in an overparameterized setting. Numerical examples are provided to demonstrate the effectiveness of the method and our theoretical findings.
MACHINE LEARNING FOR TRAJECTORIES OF PARAMETRIC NONLINEAR DYNAMICAL SYSTEMS
75-95
10.1615/JMachLearnModelComput.2020034093
Roland
Pulch
Institute for Mathematics and Computer Science, University of Greifswald,
Walther-Rathenau-Str. 47, D-17489 Greifswald, Germany
Maha
Youssef
Institute for Mathematics and Computer Science, University of Greifswald,
Walther-Rathenau-Str. 47, D-17489 Greifswald, Germany
nonlinear dynamical system
differential-algebraic equation
initial value problem
parametric model order reduction
proper orthogonal decomposition
machine learning
neural network
polynomial regression
sensitivity analysis
We investigate parameter-dependent nonlinear dynamical systems consisting of ordinary differential equations or differential-algebraic equations. A single quantity of interest is observed, which depends on the solution of a system. Our aim is to determine efficient approximations of the trajectories belonging to the quantity of interest in the time domain. We arrange a set of samples including trajectories of this quantity. A proper orthogonal decomposition of this data yields a reduced basis. Consequently, the mapping from the parameter domain to the basis coefficients is approximated. We apply machine learning with artificial neural networks for this approximation, where the degrees of freedom are fitted to the data of the sample trajectories in a nonlinear optimization. Alternatively, we consider a polynomial approximation, which is identified by regression, for comparison. Furthermore, concepts of sensitivity analysis are examined to characterize the impact of an input parameter on the output of the exact mapping or the approximations from the neural networks. We present results of numerical computations for examples of nonlinear dynamical systems.