arXiv subjects

Daniel M. Tartakovsky

Publications and source records attributed to Daniel M. Tartakovsky.

At least 37 records · Page 2Linked to original sources

arXiv2022

Deep Learning for Simultaneous Inference of Hydraulic and Transport Properties

Identifying the heterogeneous conductivity field and reconstructing the contaminant release history are key aspects of subsurface remediation. Achieving these two goals with limited and noisy hydraulic head and concentration measurements is challenging. The obstacles include solving an inverse problem for high-dimensional parameters, and the high-computational cost needed for the repeated forward modeling. We use a convolutional adversarial autoencoder (CAAE) for the parameterization of the heterogeneous non-Gaussian conductivity field with a low-dimensional latent representation. Additionally, we trained a three-dimensional dense convolutional encoder-decoder (DenseED) network to serve as the forward surrogate for the flow and transport processes. Combining the CAAE and DenseED forward surrogate models, the ensemble smoother with multiple data assimilation (ESMDA) algorithm is used to sample from the Bayesian posterior distribution of the unknown parameters, forming a CAAE-DenseED-ESMDA inversion framework. We applied this CAAE-DenseED-ESMDA inversion framework in a three-dimensional contaminant source and conductivity field identification problem. A comparison of the inversion results from CAAE-ESMDA with physical flow and transport simulator and CAAE-DenseED-ESMDA is provided, showing that accurate reconstruction results were achieved with a much higher computational efficiency.

cs.LG↗

arXiv2022

Model Reduction via Dynamic Mode Decomposition

This work proposes a new framework of model reduction for parametric complex systems. The framework employs a popular model reduction technique dynamic mode decomposition (DMD), which is capable of combining data-driven learning and physics ingredients based on the Koopman operator theory. In the offline step of the proposed framework, DMD constructs a low-rank linear surrogate model for the high dimensional quantities of interest (QoIs) derived from the (nonlinear) complex high fidelity models (HFMs) of unknown forms. Then in the online step, the resulting local reduced order bases (ROBs) and parametric reduced order models (PROMs) at the training parameter sample points are interpolated to construct a new PROM with the corresponding ROB for a new set of target/test parameter values. The interpolations need to be done on the appropriate manifolds within consistent sets of generalized coordinates. The proposed framework is illustrated by numerical examples for both linear and nonlinear problems. In particular, its advantages in computational costs and accuracy are demonstrated by the comparisons with projection-based proper orthogonal decomposition (POD)-PROM and Kriging.

math.NA↗

arXiv2022

Machine Learning in Heterogeneous Porous Materials

The "Workshop on Machine learning in heterogeneous porous materials" brought together international scientific communities of applied mathematics, porous media, and material sciences with experts in the areas of heterogeneous materials, machine learning (ML) and applied mathematics to identify how ML can advance materials research. Within the scope of ML and materials research, the goal of the workshop was to discuss the state-of-the-art in each community, promote crosstalk and accelerate multi-disciplinary collaborative research, and identify challenges and opportunities. As the end result, four topic areas were identified: ML in predicting materials properties, and discovery and design of novel materials, ML in porous and fractured media and time-dependent phenomena, Multi-scale modeling in heterogeneous porous materials via ML, and Discovery of materials constitutive laws and new governing equations. This workshop was part of the AmeriMech Symposium series sponsored by the National Academies of Sciences, Engineering and Medicine and the U.S. National Committee on Theoretical and Applied Mechanics.

cs.LG↗

arXiv2021

Impact of Airways Geometry on Transport of Gases to Blood

Topological structure of bronchial trees affects transport of gases and aerosols in the respiratory system. We start by providing a quantitative assessment of the ability of the alternative tree representations to predict observable geometric and mechanistic characteristics, such as network resistance, dead space volume, and path length. Then we present a model of dynamic transport of oxygen and carbon dioxide along the airways, in the alveoli, across the alveolar membrane, and along the pulmonary blood capillaries. The model also accounts for the exchange of these two gases with blood in the capillaries, as well as for age, gender and other in-species characteristics. Our model's predictions are compared with corresponding observations, providing an additional venue to assess the validity of the existing representations of the lung's bronchial tree.

physics.med-ph↗

arXiv2021

Autonomous learning of nonlocal stochastic neuron dynamics

Neuronal dynamics is driven by externally imposed or internally generated random excitations/noise, and is often described by systems of random or stochastic ordinary differential equations. Such systems admit a distribution of solutions, which is (partially) characterized by the single-time joint probability density function (PDF) of system states. It can be used to calculate such information-theoretic quantities as the mutual information between the stochastic stimulus and various internal states of the neuron (e.g., membrane potential), as well as various spiking statistics. When random excitations are modeled as Gaussian white noise, the joint PDF of neuron states satisfies exactly a Fokker-Planck equation. However, most biologically plausible noise sources are correlated (colored). In this case, the resulting PDF equations require a closure approximation. We propose two methods for closing such equations: a modified nonlocal large-eddy-diffusivity closure and a data-driven closure relying on sparse regression to learn relevant features. The closures are tested for the stochastic non-spiking leaky integrate-and-fire and FitzHugh-Nagumo (FHN) neurons driven by sine-Wiener noise. Mutual information and total correlation between the random stimulus and the internal states of the neuron are calculated for the FHN neuron.

q-bio.NC↗

arXiv2021

Thermal experiments for fractured rock characterization: theoretical analysis and inverse modeling

Field-scale properties of fractured rocks play crucial role in many subsurface applications, yet methodologies for identification of the statistical parameters of a discrete fracture network (DFN) are scarce. We present an inversion technique to infer two such parameters, fracture density and fractal dimension, from cross-borehole thermal experiments data. It is based on a particle-based heat-transfer model, whose evaluation is accelerated with a deep neural network (DNN) surrogate that is integrated into a grid search. The DNN is trained on a small number of heat-transfer model runs, and predicts the cumulative density function of the thermal field. The latter is used to compute fine posterior distributions of the (to-be-estimated) parameters. Our synthetic experiments reveal that fracture density is well constrained by data, while fractal dimension is harder to determine. Adding non-uniform prior information related to the DFN connectivity improves the inference of this parameter.

physics.geo-ph↗

arXiv2021

Mutual Information for Explainable Deep Learning of Multiscale Systems

Timely completion of design cycles for complex systems ranging from consumer electronics to hypersonic vehicles relies on rapid simulation-based prototyping. The latter typically involves high-dimensional spaces of possibly correlated control variables (CVs) and quantities of interest (QoIs) with non-Gaussian and possibly multimodal distributions. We develop a model-agnostic, moment-independent global sensitivity analysis (GSA) that relies on differential mutual information to rank the effects of CVs on QoIs. The data requirements of this information-theoretic approach to GSA are met by replacing computationally intensive components of the physics-based model with a deep neural network surrogate. Subsequently, the GSA is used to explain the network predictions, and the surrogate is deployed to close design loops. Viewed as an uncertainty quantification method for interrogating the surrogate, this framework is compatible with a wide variety of black-box models. We demonstrate that the surrogate-driven mutual information GSA provides useful and distinguishable rankings on two applications of interest in energy storage. Consequently, our information-theoretic GSA provides an "outer loop" for accelerated product design by identifying the most and least sensitive input directions and performing subsequent optimization over appropriately reduced parameter subspaces.

cs.LG↗

arXiv2021

Transfer Learning on Multi-Fidelity Data

Neural networks (NNs) are often used as surrogates or emulators of partial differential equations (PDEs) that describe the dynamics of complex systems. A virtually negligible computational cost of such surrogates renders them an attractive tool for ensemble-based computation, which requires a large number of repeated PDE solves. Since the latter are also needed to generate sufficient data for NN training, the usefulness of NN-based surrogates hinges on the balance between the training cost and the computational gain stemming from their deployment. We rely on multi-fidelity simulations to reduce the cost of data generation for subsequent training of a deep convolutional NN (CNN) using transfer learning. High- and low-fidelity images are generated by solving PDEs on fine and coarse meshes, respectively. We use theoretical results for multilevel Monte Carlo to guide our choice of the numbers of images of each kind. We demonstrate the performance of this multi-fidelity training strategy on the problem of estimation of the distribution of a quantity of interest, whose dynamics is governed by a system of nonlinear PDEs (parabolic PDEs of multi-phase flow in heterogeneous porous media) with uncertain/random parameters. Our numerical experiments demonstrate that a mixture of a comparatively large number of low-fidelity data and smaller numbers of high- and low-fidelity data provides an optimal balance of computational speed-up and prediction accuracy. The former is reported relative to both CNN training on high-fidelity images only and Monte Carlo solution of the PDEs. The latter is expressed in terms of both the Wasserstein distance and the Kullback-Leibler divergence.

math.NA↗

arXiv2021

Information-geometry of physics-informed statistical manifolds and its use in data assimilation

The data-aware method of distributions (DA-MD) is a low-dimension data assimilation procedure to forecast the behavior of dynamical systems described by differential equations. It combines sequential Bayesian update with the MD, such that the former utilizes available observations while the latter propagates the (joint) probability distribution of the uncertain system state(s). The core of DA-MD is the minimization of a distance between an observation and a prediction in distributional terms, with prior and posterior distributions constrained on a statistical manifold defined by the MD. We leverage the information-geometric properties of the statistical manifold to reduce predictive uncertainty via data assimilation. Specifically, we exploit the information geometric structures induced by two discrepancy metrics, the Kullback-Leibler divergence and the Wasserstein distance, which explicitly yield natural gradient descent. To further accelerate optimization, we build a deep neural network as a surrogate model for the MD that enables automatic differentiation. The manifold's geometry is quantified without sampling, yielding an accurate approximation of the gradient descent direction. Our numerical experiments demonstrate that accounting for the information-geometry of the manifold significantly reduces the computational cost of data assimilation by facilitating the calculation of gradients and by reducing the number of required iterations. Both storage needs and computational cost depend on the dimensionality of a statistical manifold, which is typically small by MD construction. When convergence is achieved, the Kullback-Leibler and $L_2$ Wasserstein metrics have similar performances, with the former being more sensitive to poor choices of the prior.

math.ST↗

arXiv2020

Data-informed Emulators for Multi-Physics Simulations

Machine learning techniques are powerful tools for construction of emulators for complex systems. We explore different machine learning methods and conceptual methodologies, ranging from functional approximations to dynamical approximations, to build such emulators for coupled thermal, hydrological, mechanical and chemical processes that occur near an engineered barrier system in the nuclear waste repository. Two nonlinear approximators, random forests and neural networks, are deployed to capture the complexity of the physics-based model and to identify its most significant hydrological and geochemical parameters. Our emulators capture the temporal evolution of the Uranium distribution coefficient of the clay buffer, and identify its functional dependence on these key parameters. The emulators' accuracy is further enhanced by assimilating relevant simulated predictors and clustering strategy. The relative performance of random forests and neural networks shows the advantage of ensemble learning in random forests algorithm, especially for highly nonlinear problems with limited data.

math.DS↗

arXiv2020

Markov Chain Monte Carlo with Neural Network Surrogates: Application to Contaminant Source Identification

Subsurface remediation often involves reconstruction of contaminant release history from sparse observations of solute concentration. Markov Chain Monte Carlo (MCMC), the most accurate and general method for this task, is rarely used in practice because of its high computational cost associated with multiple solves of contaminant transport equations. We propose an adaptive MCMC method, in which a transport model is replaced with a fast and accurate surrogate model in the form of a deep convolutional neural network (CNN). The CNN-based surrogate is trained on a small number of the transport model runs based on the prior knowledge of the unknown release history. Thus reduced computational cost allows one to reduce the sampling error associated with construction of the approximate likelihood function. As all MCMC strategies for source identification, our method has an added advantage of quantifying predictive uncertainty and accounting for measurement errors. Our numerical experiments demonstrate the accuracy comparable to that of MCMC with the forward transport model, which is obtained at a fraction of the computational cost of the latter.

cs.CE↗

arXiv2020

Data-Driven Discovery of Coarse-Grained Equations

Statistical (machine learning) tools for equation discovery require large amounts of data that are typically computer generated rather than experimentally observed. Multiscale modeling and stochastic simulations are two areas where learning on simulated data can lead to such discovery. In both, the data are generated with a reliable but impractical model, e.g., molecular dynamics simulations, while a model on the scale of interest is uncertain, requiring phenomenological constitutive relations and ad-hoc approximations. We replace the human discovery of such models, which typically involves spatial/stochastic averaging or coarse-graining, with a machine-learning strategy based on sparse regression that can be executed in two modes. The first, direct equation-learning, discovers a differential operator from the whole dictionary. The second, constrained equation-learning, discovers only those terms in the differential operator that need to be discovered, i.e., learns closure approximations. We illustrate our approach by learning a deterministic equation that governs the spatiotemporal evolution of the probability density function of a system state whose dynamics are described by a nonlinear partial differential equation with random inputs. A series of examples demonstrates the accuracy, robustness, and limitations of our approach to equation discovery.

stat.ML↗

arXiv2020

GINNs: Graph-Informed Neural Networks for Multiscale Physics

We introduce the concept of a Graph-Informed Neural Network (GINN), a hybrid approach combining deep learning with probabilistic graphical models (PGMs) that acts as a surrogate for physics-based representations of multiscale and multiphysics systems. GINNs address the twin challenges of removing intrinsic computational bottlenecks in physics-based models and generating large data sets for estimating probability distributions of quantities of interest (QoIs) with a high degree of confidence. Both the selection of the complex physics learned by the NN and its supervised learning/prediction are informed by the PGM, which includes the formulation of structured priors for tunable control variables (CVs) to account for their mutual correlations and ensure physically sound CV and QoI distributions. GINNs accelerate the prediction of QoIs essential for simulation-based decision-making where generating sufficient sample data using physics-based models alone is often prohibitively expensive. Using a real-world application grounded in supercapacitor-based energy storage, we describe the construction of GINNs from a Bayesian network-embedded homogenized model for supercapacitor dynamics, and demonstrate their ability to produce kernel density estimates of relevant non-Gaussian, skewed QoIs with tight confidence intervals.

physics.comp-ph↗

arXiv2020

Learning on dynamic statistical manifolds

Hyperbolic balance laws with uncertain (random) parameters and inputs are ubiquitous in science and engineering. Quantification of uncertainty in predictions derived from such laws, and reduction of predictive uncertainty via data assimilation, remain an open challenge. That is due to nonlinearity of governing equations, whose solutions are highly non-Gaussian and often discontinuous. To ameliorate these issues in a computationally efficient way, we use the method of distributions, which here takes the form of a deterministic equation for spatiotemporal evolution of the cumulative distribution function (CDF) of the random system state, as a means of forward uncertainty propagation. Uncertainty reduction is achieved by recasting the standard loss function, i.e., discrepancy between observations and model predictions, in distributional terms. This step exploits the equivalence between minimization of the square error discrepancy and the Kullback-Leibler divergence. The loss function is regularized by adding a Lagrangian constraint enforcing fulfillment of the CDF equation. Minimization is performed sequentially, progressively updating the parameters of the CDF equation as more measurements are assimilated.

math.ST↗

arXiv2020

Extended Dynamic Mode Decomposition for Inhomogeneous Problems

Dynamic mode decomposition (DMD) is a powerful data-driven technique for construction of reduced-order models of complex dynamical systems. Multiple numerical tests have demonstrated the accuracy and efficiency of DMD, but mostly for systems described by partial differential equations (PDEs) with homogeneous boundary conditions. We propose an extended dynamic mode decomposition (xDMD) approach to cope with the potential unknown sources/sinks in PDEs. Motivated by similar ideas in deep neural networks, we equipped our xDMD with two new features. First, it has a bias term, which accounts for inhomogeneity of PDEs and/or boundary conditions. Second, instead of learning a flow map, xDMD learns the residual increment by subtracting the identity operator. Our theoretical error analysis demonstrates the improved accuracy of xDMD relative to standard DMD. Several numerical examples are presented to illustrate this result.

math.NA↗

arXiv2020

Dynamic Mode Decomposition for Construction of Reduced-Order Models of Hyperbolic Problems with Shocks

Construction of reduced-order models (ROMs) for hyperbolic conservation laws is notoriously challenging mainly due to the translational property and nonlinearity of the governing equations. While the Lagrangian framework for ROM construction resolves the translational issue, it is valid only before a shock forms. Once that occurs, characteristic lines cross each other and projection from a high-fidelity model space onto a ROM space distorts a moving grid, resulting in numerical instabilities. We address this grid distortion issue by developing a physics-aware dynamic mode decomposition (DMD) method based on hodograph transformation. The latter provides a map between the original nonlinear system and its linear counterpart, which coincides with the Koopman operator. This strategy is consistent with the spirit of physics-aware DMDs in that it retains information about shock dynamics. Several numerical examples are presented to validate the proposed physics-aware DMD approach to constructing accurate ROMs.

math.NA↗

arXiv2020

Dynamics of Data-driven Ambiguity Sets for Hyperbolic Conservation Laws with Uncertain Inputs

Ambiguity sets of probability distributions are used to hedge against uncertainty about the true probabilities of random quantities of interest (QoIs). When available, these ambiguity sets are constructed from both data (collected at the initial time and along the boundaries of the physical domain) and concentration-of-measure results on the Wasserstein metric. To propagate the ambiguity sets into the future, we use a physics-dependent equation governing the evolution of cumulative distribution functions (CDF) obtained through the method of distributions. This study focuses on the latter step by investigating the spatio-temporal evolution of data-driven ambiguity sets and their associated guarantees when the random QoIs they describe obey hyperbolic partial-differential equations with random inputs. For general nonlinear hyperbolic equations with smooth solutions, the CDF equation is used to propagate the upper and lower envelopes of pointwise ambiguity bands. For linear dynamics, the CDF equation allows us to construct an evolution equation for tighter ambiguity balls. We demonstrate that, in both cases, the ambiguity sets are guaranteed to contain the true (unknown) distributions within a prescribed confidence.

math.OC↗

arXiv2020

Stochastic Pore Collapse Models in Granular Materials

Stochastic models for pore collapse in granular materials are developed. First, a general fluctuating stress-strain relation for a plastic flow rule is derived. The fluctuations account for non-associativity in plastic deformations typically observed in heterogeneous materials. Second, an axisymmetric spherical shell compaction model is extended to account for fluctuations in the material microstructure due to granular interactions at the pore scale. This changes the stress-strain constitutive equation determining the dynamics of pore collapse. Results show that stochastic differential equations can account for multiscale interactions in a statistical sense.

cond-mat.soft↗