arXiv subjects

George Em Karniadakis

Publications and source records attributed to George Em Karniadakis.

At least 235 records · Page 13Linked to original sources

arXiv2019

A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems

We propose a new composite neural network (NN) that can be trained based on multi-fidelity data. It is comprised of three NNs, with the first NN trained using the low-fidelity data and coupled to two high-fidelity NNs, one with activation functions and another one without, in order to discover and exploit nonlinear and linear correlations, respectively, between the low-fidelity and the high-fidelity data. We first demonstrate the accuracy of the new multi-fidelity NN for approximating some standard benchmark functions but also a 20-dimensional function. Subsequently, we extend the recently developed physics-informed neural networks (PINNs) to be trained with multi-fidelity data sets (MPINNs). MPINNs contain four fully-connected neural networks, where the first one approximates the low-fidelity data, while the second and third construct the correlation between the low- and high-fidelity data and produce the multi-fidelity approximation, which is then used in the last NN that encodes the partial differential equations (PDEs). Specifically, in the two high-fidelity NNs a relaxation parameter is introduced, which can be optimized to combine the linear and nonlinear sub-networks. By optimizing this parameter, the present model is capable of learning both the linear and complex nonlinear correlations between the low- and high-fidelity data adaptively. By training the MPINNs, we can:(1) obtain the correlation between the low- and high-fidelity data, (2) infer the quantities of interest based on a few scattered data, and (3) identify the unknown parameters in the PDEs. In particular, we employ the MPINNs to learn the hydraulic conductivity field for unsaturated flows as well as the reactive models for reactive transport. The results demonstrate that MPINNs can achieve relatively high accuracy based on a very small set of high-fidelity data.

physics.comp-ph↗

arXiv2018

Collapse of Deep and Narrow Neural Nets

Recent theoretical work has demonstrated that deep neural networks have superior performance over shallow networks, but their training is more difficult, e.g., they suffer from the vanishing gradient problem. This problem can be typically resolved by the rectified linear unit (ReLU) activation. However, here we show that even for such activation, deep and narrow neural networks (NNs) will converge to erroneous mean or median states of the target function depending on the loss with high probability. Deep and narrow NNs are encountered in solving partial differential equations with high-order derivatives. We demonstrate this collapse of such NNs both numerically and theoretically, and provide estimates of the probability of collapse. We also construct a diagram of a safe region for designing NNs that avoid the collapse to erroneous states. Finally, we examine different ways of initialization and normalization that may avoid the collapse problem. Asymmetric initializations may reduce the probability of collapse but do not totally eliminate it.

stat.ML↗

arXiv2018

Efficient multistep methods for tempered fractional calculus: Algorithms and Simulations

In this work, we extend the fractional linear multistep methods in [C. Lubich, SIAM J. Math. Anal., 17 (1986), pp.704--719] to the tempered fractional integral and derivative operators in the sense that the tempered fractional derivative operator is interpreted in terms of the Hadamard finite-part integral. We develop two fast methods, Fast Method I and Fast Method II, with linear complexity to calculate the discrete convolution for the approximation of the (tempered) fractional operator. Fast Method I is based on a local approximation for the contour integral that represents the convolution weight. Fast Method II is based on a globally uniform approximation of the trapezoidal rule for the integral on the real line. Both methods are efficient, but numerical experimentation reveals that Fast Method II outperforms Fast Method I in terms of accuracy, efficiency, and coding simplicity. The memory requirement and computational cost of Fast Method II are $O(Q)$ and $O(Qn_T)$, respectively, where $n_T$ is the number of the final time steps and $Q$ is the number of quadrature points used in the trapezoidal rule. The effectiveness of the fast methods is verified through a series of numerical examples for long-time integration, including a numerical study of a fractional reaction-diffusion model.

math.NA↗

arXiv2018

fPINNs: Fractional Physics-Informed Neural Networks

Physics-informed neural networks (PINNs) are effective in solving integer-order partial differential equations (PDEs) based on scattered and noisy data. PINNs employ standard feedforward neural networks (NNs) with the PDEs explicitly encoded into the NN using automatic differentiation, while the sum of the mean-squared PDE-residuals and the mean-squared error in initial/boundary conditions is minimized with respect to the NN parameters. We extend PINNs to fractional PINNs (fPINNs) to solve space-time fractional advection-diffusion equations (fractional ADEs), and we demonstrate their accuracy and effectiveness in solving multi-dimensional forward and inverse problems with forcing terms whose values are only known at randomly scattered spatio-temporal coordinates (black-box forcing terms). A novel element of the fPINNs is the hybrid approach that we introduce for constructing the residual in the loss function using both automatic differentiation for the integer-order operators and numerical discretization for the fractional operators. We consider 1D time-dependent fractional ADEs and compare white-box (WB) and black-box (BB) forcing. We observe that for the BB forcing fPINNs outperform FDM. Subsequently, we consider multi-dimensional time-, space-, and space-time-fractional ADEs using the directional fractional Laplacian and we observe relative errors of $10^{-4}$. Finally, we solve several inverse problems in 1D, 2D, and 3D to identify the fractional orders, diffusion coefficients, and transport velocities and obtain accurate results even in the presence of significant noise.

physics.comp-ph↗

arXiv2018

Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations

We developed a new class of physics-informed generative adversarial networks (PI-GANs) to solve in a unified manner forward, inverse and mixed stochastic problems based on a limited number of scattered measurements. Unlike standard GANs relying only on data for training, here we encoded into the architecture of GANs the governing physical laws in the form of stochastic differential equations (SDEs) using automatic differentiation. In particular, we applied Wasserstein GANs with gradient penalty (WGAN-GP) for its enhanced stability compared to vanilla GANs. We first tested WGAN-GP in approximating Gaussian processes of different correlation lengths based on data realizations collected from simultaneous reads at sparsely placed sensors. We obtained good approximation of the generated stochastic processes to the target ones even for a mismatch between the input noise dimensionality and the effective dimensionality of the target stochastic processes. We also studied the overfitting issue for both the discriminator and generator, and we found that overfitting occurs also in the generator in addition to the discriminator as previously reported. Subsequently, we considered the solution of elliptic SDEs requiring approximations of three stochastic processes, namely the solution, the forcing, and the diffusion coefficient. We used three generators for the PI-GANs, two of them were feed forward deep neural networks (DNNs) while the other one was the neural network induced by the SDE. Depending on the data, we employed one or multiple feed forward DNNs as the discriminators in PI-GANs. Here, we have demonstrated the accuracy and effectiveness of PI-GANs in solving SDEs for up to 30 dimensions, but in principle, PI-GANs could tackle very high dimensional problems given more sensor data with low-polynomial growth in computational cost.

stat.ML↗

arXiv2018

Self-cleaning of hydrophobic rough surfaces by coalescence-induced wetting transition

The superhydrophobic leaves of a lotus plant and other natural surfaces with self-cleaning function have been studied intensively for the development of artificial biomimetic surfaces. Surface roughness generated by hierarchical structures is a crucial property required for superhydrophobicity and self-cleaning. Here, we demonstrate a novel self-cleaning mechanism of textured surfaces attributed to a spontaneous coalescence-induced wetting transition. We focus on the wetting transition as it represents a new mechanism, which can explain why droplets on rough surfaces are able to change from the highly adhesive Wenzel state to the low-adhesion Cassie-Baxter state and achieve self-cleaning. In particular, we perform many-body dissipative particle dynamics simulations of liquid droplets sitting on mechanically textured substrates. We quantitatively investigate the wetting behavior of an isolated droplet as well as coalescence of droplets for both Cassie-Baxter and Wenzel states. Our simulation results reveal that droplets in the Cassie-Baxter state have much lower contact angle hysteresis and smaller hydrodynamic resistance than droplets in the Wenzel state. When small neighboring droplets coalesce into bigger ones on textured hydrophobic substrates, we observe a spontaneous wetting transition from a Wenzel state to a Cassie-Baxter state, which is powered by the surface energy released upon coalescence of the droplets. For superhydrophobic surfaces, the released surface energy may be sufficient to cause a jumping motion of droplets off the surface, in which case adding one more droplet to coalescence may increase the jumping velocity by one order of magnitude. When multiple droplets are involved, we find that the spatial distribution of liquid components in the coalesced droplet can be controlled by properly designing the overall arrangement of droplets and the distance between them.

physics.comp-ph↗

arXiv2018

Nonlocal flocking dynamics: Learning the fractional order of PDEs from particle simulations

Flocking refers to collective behavior of a large number of interacting entities, where the interactions between discrete individuals produce collective motion on the large scale. We employ an agent-based model to describe the microscopic dynamics of each individual in a flock, and use a fractional PDE to model the evolution of macroscopic quantities of interest. The macroscopic models with phenomenological interaction functions are derived by applying the continuum hypothesis to the microscopic model. Instead of specifying the fPDEs with an ad hoc fractional order for nonlocal flocking dynamics, we learn the effective nonlocal influence function in fPDEs directly from particle trajectories generated by the agent-based simulations. We demonstrate how the learning framework is used to connect the discrete agent-based model to the continuum fPDEs in 1D and 2D nonlocal flocking dynamics. In particular, a Cucker-Smale particle model is employed to describe the microscale dynamics of each individual, while Euler equations with nonlocal interaction terms are used to compute the evolution of macroscale quantities. The trajectories generated by the particle simulations mimic the field data of tracking logs that can be obtained experimentally. They can be used to learn the fractional order of the influence function using a Gaussian process regression model implemented with the Bayesian optimization. We show that the numerical solution of the learned Euler equations solved by the finite volume scheme can yield correct density distributions consistent with the collective behavior of the agent-based system. The proposed method offers new insights on how to scale the discrete agent-based models to the continuum-based PDE models, and could serve as a paradigm on extracting effective governing equations for nonlocal flocking dynamics directly from particle trajectories.

math.NA↗

arXiv2018

Concurrent coupling of atomistic simulation and mesoscopic hydrodynamics for flows over soft multi-functional surfaces

We develop an efficient parallel multiscale method that bridges the atomistic and mesoscale regimes, from nanometer to micron and beyond, via concurrent coupling of atomistic simulation and mesoscopic dynamics. In particular, we combine an all-atom molecular dynamics (MD) description for specific atomistic details in the vicinity of the functional surface, with a dissipative particle dynamics (DPD) approach that captures mesoscopic hydrodynamics in the domain away from the functional surface. In order to achieve a seamless transition in dynamic properties we endow the MD simulation with a DPD thermostat, which is validated against experimental results by modeling water at different temperatures. We then validate the MD-DPD coupling method for transient Couette and Poiseuille flows, demonstrating that the concurrent MD-DPD coupling can resolve accurately the continuum-based analytical solutions. Subsequently, we simulate shear flows over polydimethylsiloxane (PDMS)-grafted surfaces (polymer brushes) for various grafting densities, and investigate the slip flow as a function of the shear stress. We verify that a "universal" power law exists for the sliplength, in agreement with published results. Having validated the MD-DPD coupling method, we simulate time-dependent flows past an endothelial glycocalyx layer (EGL) in a microchannel. Coupled simulation results elucidate the dynamics of EGL changing from an equilibrium state to a compressed state under shear by aligning the molecular structures along the shear direction. MD-DPD simulation results agree well with results of a single MD simulation, but with the former more than two orders of magnitude faster than the latter for system sizes above one micron.

physics.comp-ph↗

arXiv2018

Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems

Physics-informed neural networks (PINNs) have recently emerged as an alternative way of solving partial differential equations (PDEs) without the need of building elaborate grids, instead, using a straightforward implementation. In particular, in addition to the deep neural network (DNN) for the solution, a second DNN is considered that represents the residual of the PDE. The residual is then combined with the mismatch in the given data of the solution in order to formulate the loss function. This framework is effective but is lacking uncertainty quantification of the solution due to the inherent randomness in the data or due to the approximation limitations of the DNN architecture. Here, we propose a new method with the objective of endowing the DNN with uncertainty quantification for both sources of uncertainty, i.e., the parametric uncertainty and the approximation uncertainty. We first account for the parametric uncertainty when the parameter in the differential equation is represented as a stochastic process. Multiple DNNs are designed to learn the modal functions of the arbitrary polynomial chaos (aPC) expansion of its solution by using stochastic data from sparse sensors. We can then make predictions from new sensor measurements very efficiently with the trained DNNs. Moreover, we employ dropout to correct the over-fitting and also to quantify the uncertainty of DNNs in approximating the modal functions. We then design an active learning strategy based on the dropout uncertainty to place new sensors in the domain to improve the predictions of DNNs. Several numerical tests are conducted for both the forward and the inverse problems to quantify the effectiveness of PINNs combined with uncertainty quantification. This NN-aPC new paradigm of physics-informed deep learning with uncertainty quantification can be readily applied to other types of stochastic PDEs in multi-dimensions.

math.AP↗

arXiv2018

Multi-domain Spectral Collocation Method for Variable-Order Nonlinear Fractional Differential Equations

Spectral and spectral element methods using Galerkin type formulations are efficient for solving linear fractional PDEs (FPDEs) of constant order but are not efficient in solving nonlinear FPDEs and cannot handle FPDEs with variable-order. In this paper, we present a multi-domain spectral collocation method that addresses these limitations. We consider FPDEs in the Riemann-Liouville sense, and employ Jacobi Lagrangian interpolants to represent the solution in each element. We provide variable-order differentiation formulas, which can be computed efficiently for the multi-domain discretization taking into account the nonlocal interactions. We enforce the interface continuity conditions by matching the solution values at the element boundaries via the Lagrangian interpolants, and in addition we minimize the jump in (integer) fluxes using a penalty method. We analyze numerically the effect of the penalty parameter on the condition number of the global differentiation matrix and on the stability and convergence of the penalty collocation scheme. We demonstrate the effectiveness of the new method for the fractional Helmholtz equation of constant and variable-order using $h-p$ refinement for different values of the penalty parameter. We also solve the fractional Burgers equation with constant and variable-order and compare with solutions obtained with a single domain spectral collocation method.

math.NA↗

arXiv2018

A Universal Fractional Model of Wall-Turbulence

Modeling of wall-bounded turbulent flows is still an open problem in classical physics, with only modest progress made in the last few decades beyond the so-called `log law', which describes only the intermediate region in wall-bounded turbulence, i.e., $30-50 y^+ \text{ to } 0.1-0.2 R^+$ (in wall units) in a pipe of radius $R$. Here we propose a fundamentally new approach based on fractional calculus to model the {\em entire} mean velocity profile from the wall to the centerline of the pipe. Specifically, we represent the Reynolds stresses with a non-local fractional derivative of {\em variable order} that decays with the distance from the wall. Surprisingly, we find that this variable fractional order has a universal form for all Reynolds numbers and for three different flow types, i.e., channel flow, Couette flow, and pipe flow. We first use existing data bases from direct numerical simulations (DNS) to learn the variable fractional order function, and subsequently we test it against other DNS data and experimental measurements, including the Princeton superpipe experiments. Taken together, our findings reveal the continuous and decaying change of rate of turbulent diffusion from the wall as well as the strong non-locality of turbulent interactions that intensify away from the wall.

physics.flu-dyn↗

arXiv2018

Deep Learning of Vortex Induced Vibrations

Vortex induced vibrations of bluff bodies occur when the vortex shedding frequency is close to the natural frequency of the structure. Of interest is the prediction of the lift and drag forces on the structure given some limited and scattered information on the velocity field. This is an inverse problem that is not straightforward to solve using standard computational fluid dynamics (CFD) methods, especially since no information is provided for the pressure. An even greater challenge is to infer the lift and drag forces given some dye or smoke visualizations of the flow field. Here we employ deep neural networks that are extended to encode the incompressible Navier-Stokes equations coupled with the structure's dynamic motion equation. In the first case, given scattered data in space-time on the velocity field and the structure's motion, we use four coupled deep neural networks to infer very accurately the structural parameters, the entire time-dependent pressure field (with no prior training data), and reconstruct the velocity vector field and the structure's dynamic motion. In the second case, given scattered data in space-time on a concentration field only, we use five coupled deep neural networks to infer very accurately the vector velocity field and all other quantities of interest as before. This new paradigm of inference in fluid mechanics for coupled multi-physics problems enables velocity and pressure quantification from flow snapshots in small subdomains and can be exploited for flow control applications and also for system identification.

physics.flu-dyn↗

arXiv2018

Hidden Fluid Mechanics: A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data

We present hidden fluid mechanics (HFM), a physics informed deep learning framework capable of encoding an important class of physical laws governing fluid motions, namely the Navier-Stokes equations. In particular, we seek to leverage the underlying conservation laws (i.e., for mass, momentum, and energy) to infer hidden quantities of interest such as velocity and pressure fields merely from spatio-temporal visualizations of a passive scaler (e.g., dye or smoke), transported in arbitrarily complex domains (e.g., in human arteries or brain aneurysms). Our approach towards solving the aforementioned data assimilation problem is unique as we design an algorithm that is agnostic to the geometry or the initial and boundary conditions. This makes HFM highly flexible in choosing the spatio-temporal domain of interest for data acquisition as well as subsequent training and predictions. Consequently, the predictions made by HFM are among those cases where a pure machine learning strategy or a mere scientific computing approach simply cannot reproduce. The proposed algorithm achieves accurate predictions of the pressure and velocity fields in both two and three dimensional flows for several benchmark problems motivated by real-world applications. Our results demonstrate that this relatively simple methodology can be used in physical and biomedical problems to extract valuable quantitative information (e.g., lift and drag forces or wall shear stresses in arteries) for which direct measurements may not be possible.

cs.CE↗

arXiv2018

A new class of semi-implicit methods with linear complexity for nonlinear fractional differential equations

We propose a new class of semi-implicit methods for solving nonlinear fractional differential equations and study their stability. Several versions of our new schemes are proved to be unconditionally stable by choosing suitable parameters. Subsequently, we develop an efficient strategy to calculate the discrete convolution for the approximation of the fractional operator in the semi-implicit method and we derive an error bound of the fast convolution. The memory requirement and computational cost of the present semi-implicit methods with a fast convolution are about $O(N\log n_T)$ and $O(Nn_T\log n_T)$, respectively, where $N$ is a suitable positive integer and $n_T$ is the final number of time steps. Numerical simulations, including the solution of a system of two nonlinear fractional diffusion equations with different fractional orders in two-dimensions, are presented to verify the effectiveness of the semi-implicit methods.

math.NA↗

arXiv2018

A fast solver for spectral element approximation applied to fractional differential equations using hierarchical matrix approximation

We develop a fast solver for the spectral element method (SEM) applied to the two-sided fractional diffusion equation on uniform, geometric and graded meshes. By approximating the singular kernel with a degenerate kernel, we construct a hierarchical matrix (H-matrix) to represent the stiffness matrix of the SEM and provide error estimates verified numerically. We can solve efficiently the H-matrix approximation problem using a hierarchical LU decomposition method, which reduces the computational cost to $O(R^2 N_d \log^2N) +O(R^3 N_d \log N)$, where $R$ it is the rank of submatrices of the H-matrix approximation, $N_d$ is the total number of degrees of freedom and $N$ is the number of elements. However, we lose the high accuracy of the SEM. Thus, we solve the corresponding preconditioned system by using the H-matrix approximation problem as a preconditioner, recovering the high order accuracy of the SEM. The condition number of the preconditioned system is independent of the polynomial degree $P$ and grows with the number of elements, but at modest values of the rank $R$ is below order 10 in our experiments, which represents a reduction of more than 11 orders of magnitude from the unpreconditioned system; this reduction is higher in the two-sided fractional derivative compared to one-sided fractional derivative. The corresponding cost is $O(R^2 N_d \log^2 N)+O(R^3 N_d \log N)+O(N_d^2)$. Moreover, by using a structured mesh (uniform or geometric mesh), we can further reduce the computational cost to $O(R^2 N_d\log^2 N) +O(R^3 N_d \log N)+ O(P^2 N\log N)$ for the preconditioned system. We present several numerical tests to illustrate the proposed algorithm using $h$ and $p$ refinements.

math.NA↗

arXiv2018

Nature of Intrinsic Uncertainties in Equilibrium Molecular Dynamics Estimation of Shear Viscosity for Simple and Complex Fluids

We study two types of intrinsic uncertainties, statistical errors and system size effects, in estimating shear viscosity via equilibrium molecular dynamics simulations and compare them with the corresponding uncertainties in evaluating the self-diffusion coefficient. Uncertainty quantification formulas for the statistical errors in the shear-stress autocorrelation function and shear viscosity are obtained under the assumption that shear stress follows a Gaussian process. Analyses of simulation results for simple and complex fluids reveal that the Gaussianity is more pronounced in the shear-stress process (related to shear viscosity estimation) compared with the velocity process of an individual molecule (related to self-diffusion coefficient). At relatively high densities corresponding to a liquid state, we observe that the shear viscosity exhibits complex size-dependent behavior unless the system is larger than a certain length scale, beyond which reliable shear viscosity values are obtained without any noticeable scaling behavior with respect to the system size. We verify that this size-dependent behavior is configurational and relate the characteristic length scale to the shear-stress correlation length.

physics.chem-ph↗

arXiv2018

A spectral penalty method for two-sided fractional differential equations with general boundary conditions

We consider spectral approximations to the conservative form of the two-sided Riemann-Liouville (R-L) and Caputo fractional differential equations (FDEs) with nonhomogeneous Dirichlet (fractional and classical, respectively) and Neumann (fractional) boundary conditions. In particular, we develop a spectral penalty method (SPM) by using the Jacobi poly-fractonomial approximation for the conservative R-L FDEs while using the polynomial approximation for the conservative Caputo FDEs. We establish the well-posedness of the corresponding weak problems and analyze sufficient conditions for the coercivity of the SPM for different types of fractional boundary value problems. This analysis allows us to estimate the proper values of the penalty parameters at boundary points. We present several numerical examples to verify the theory and demonstrate the high accuracy of SPM, both for stationary and time dependent FDEs. Moreover, we compare the results against a Petrov-Galerkin spectral tau method (PGS-$τ$, an extension of [Z. Mao, G.E. Karniadakis, SIAM J. Numer. Anal., 2018]) and demonstrate the superior accuracy of SPM for all cases considered.

math.NA↗

arXiv2018

Neural-net-induced Gaussian process regression for function approximation and PDE solution

Neural-net-induced Gaussian process (NNGP) regression inherits both the high expressivity of deep neural networks (deep NNs) as well as the uncertainty quantification property of Gaussian processes (GPs). We generalize the current NNGP to first include a larger number of hyperparameters and subsequently train the model by maximum likelihood estimation. Unlike previous works on NNGP that targeted classification, here we apply the generalized NNGP to function approximation and to solving partial differential equations (PDEs). Specifically, we develop an analytical iteration formula to compute the covariance function of GP induced by deep NN with an error-function nonlinearity. We compare the performance of the generalized NNGP for function approximations and PDE solutions with those of GPs and fully-connected NNs. We observe that for smooth functions the generalized NNGP can yield the same order of accuracy with GP, while both NNGP and GP outperform deep NN. For non-smooth functions, the generalized NNGP is superior to GP and comparable or superior to deep NN.

cs.LG↗