arXiv subjects

Guang Lin

Publications and source records attributed to Guang Lin.

At least 145 records · Page 8Linked to original sources

arXiv2022

A consistent and conservative Phase-Field method for multiphase incompressible flows

A consistent and conservative Phase-Field method, including both the model and scheme, is developed for multiphase flows with an arbitrary number of immiscible and incompressible fluid phases. The consistency of mass conservation and the consistency of mass and momentum transport are implemented to address the issue of physically coupling the Phase-Field equation, which locates different phases, to the hydrodynamics. These two consistency conditions provide the ``optimal'' coupling because (i) the new momentum equation resulting from them is Galilean invariant and implies the kinetic energy conservation, regardless of the details of the Phase-Field equation, and (ii) failures of satisfying the second law of thermodynamics or the consistency of reduction of the multiphase flow model only result from the same failures of the Phase-Field equation but are not due to the new momentum equation. Physical interpretation of the consistency conditions and their formulations are first provided, and general formulations that are obtained from the consistency conditions and independent of the interpretation of the velocity are summarized. Several novel techniques are developed to inherit the physical properties of the multiphase flows after discretization, including the gradient-based phase selection procedure, the momentum conservative method for the surface force, and the general theorems to preserve the consistency conditions on the discrete level. Equipped with those novel techniques, a consistent and conservative scheme for the present multiphase flow model is developed and analyzed. Numerical applications demonstrate that the present model and scheme are robust and effective in studying complicated multiphase dynamics, especially for those with large-density ratios.

physics.comp-ph↗

arXiv2022

glassoformer: a query-sparse transformer for post-fault power grid voltage prediction

We propose GLassoformer, a novel and efficient transformer architecture leveraging group Lasso regularization to reduce the number of queries of the standard self-attention mechanism. Due to the sparsified queries, GLassoformer is more computationally efficient than the standard transformers. On the power grid post-fault voltage prediction task, GLassoformer shows remarkably better prediction than many existing benchmark algorithms in terms of accuracy and stability.

cs.LG↗

arXiv2021

Deformation Robust Roto-Scale-Translation Equivariant CNNs

Incorporating group symmetry directly into the learning process has proved to be an effective guideline for model design. By producing features that are guaranteed to transform covariantly to the group actions on the inputs, group-equivariant convolutional neural networks (G-CNNs) achieve significantly improved generalization performance in learning tasks with intrinsic symmetry. General theory and practical implementation of G-CNNs have been studied for planar images under either rotation or scaling transformation, but only individually. We present, in this paper, a roto-scale-translation equivariant CNN (RST-CNN), that is guaranteed to achieve equivariance jointly over these three groups via coupled group convolutions. Moreover, as symmetry transformations in reality are rarely perfect and typically subject to input deformation, we provide a stability analysis of the equivariance of representation to input distortion, which motivates the truncated expansion of the convolutional filters under (pre-fixed) low-frequency spatial modes. The resulting model provably achieves deformation-robust RST equivariance, i.e., the RST symmetry is still "approximately" preserved when the transformation is "contaminated" by a nuisance data deformation, a property that is especially important for out-of-distribution generalization. Numerical experiments on MNIST, Fashion-MNIST, and STL-10 demonstrate that the proposed model yields remarkable gains over prior arts, especially in the small data regime where both rotation and scaling variations are present within the data.

cs.CV↗

arXiv2021

A consistent and conservative Phase-Field model for thermo-gas-liquid-solid flows including liquid-solid phase change

In the present study, a consistent and conservative Phase-Field model is developed to study thermo-gas-liquid-solid flows with liquid-solid phase change. The proposed model is derived with the help of the consistency conditions and exactly reduces to the consistent and conservative Phase-Field method for incompressible two-phase flows, the fictitious domain Brinkman penalization (FD/BP) method for fluid-structure interactions, and the Phase-Field model of solidification of pure material. It honors the mass conservation, defines the volume fractions of individual phases unambiguously, and therefore captures the volume change due to phase change. The momentum is conserved when the solid phase is absent, but it changes when the solid phase appears due to the no-slip condition at the solid boundary. The proposed model also conserves the energy, preserves the temperature equilibrium, and is Galilean invariant. A novel continuous surface tension force to confine its contribution at the gas-liquid interface and a drag force modified from the Carman-Kozeny equation to reduce solid velocity to zero are proposed. The issue of initiating phase change in the original Phase-Field model of solidification is addressed by physically modifying the interpolation function. The corresponding consistent scheme is developed to solve the model, and the numerical results agree well with the analytical solutions and the existing experimental and numerical data. Two challenging problems having a wide range of material properties and complex dynamics are conducted to demonstrate the capability of the proposed model.

physics.comp-ph↗

arXiv2021

Accelerated replica exchange stochastic gradient Langevin diffusion enhanced Bayesian DeepONet for solving noisy parametric PDEs

The Deep Operator Networks~(DeepONet) is a fundamentally different class of neural networks that we train to approximate nonlinear operators, including the solution operator of parametric partial differential equations (PDE). DeepONets have shown remarkable approximation and generalization capabilities even when trained with relatively small datasets. However, the performance of DeepONets deteriorates when the training data is polluted with noise, a scenario that occurs very often in practice. To enable DeepONets training with noisy data, we propose using the Bayesian framework of replica-exchange Langevin diffusion. Such a framework uses two particles, one for exploring and another for exploiting the loss function landscape of DeepONets. We show that the proposed framework's exploration and exploitation capabilities enable (1) improved training convergence for DeepONets in noisy scenarios and (2) attaching an uncertainty estimate for the predicted solutions of parametric PDEs. In addition, we show that replica-exchange Langeving Diffusion (remarkably) also improves the DeepONet's mean prediction accuracy in noisy scenarios compared with vanilla DeepONets trained with state-of-the-art gradient-based optimization algorithms (e.g. Adam). To reduce the potentially high computational cost of replica, in this work, we propose an accelerated training framework for replica-exchange Langevin diffusion that exploits the neural network architecture of DeepONets to reduce its computational cost up to 25% without compromising the proposed framework's performance. Finally, we illustrate the effectiveness of the proposed Bayesian framework using a series of experiments on four parametric PDE problems.

math.NA↗

arXiv2021

Theoretical and numerical studies of inverse source problem for the linear parabolic equation with sparse boundary measurements

We consider the inverse source problem in the parabolic equation, where the unknown source possesses the semi-discrete formulation. Theoretically, we prove that the flux data from any nonempty open subset of the boundary can uniquely determine the semi-discrete source. This means the observed area can be extremely small, and that is why we call the data as sparse boundary data. For the numerical reconstruction, we formulate the problem from the Bayesian sequential prediction perspective and conduct the numerical examples which estimate the space-time-dependent source state by state. To better demonstrate the performance of the method, we solve two common multiscale problems from two models with a long sequence of the source. The numerical results illustrate that the inversion is accurate and efficient.

math.NA↗

arXiv2021

DAE-PINN: A Physics-Informed Neural Network Model for Simulating Differential-Algebraic Equations with Application to Power Networks

Deep learning-based surrogate modeling is becoming a promising approach for learning and simulating dynamical systems. Deep-learning methods, however, find very challenging learning stiff dynamics. In this paper, we develop DAE-PINN, the first effective deep-learning framework for learning and simulating the solution trajectories of nonlinear differential-algebraic equations (DAE), which present a form of infinite stiffness and describe, for example, the dynamics of power networks. Our DAE-PINN bases its effectiveness on the synergy between implicit Runge-Kutta time-stepping schemes (designed specifically for solving DAEs) and physics-informed neural networks (PINN) (deep neural networks that we train to satisfy the dynamics of the underlying problem). Furthermore, our framework (i) enforces the neural network to satisfy the DAEs as (approximate) hard constraints using a penalty-based method and (ii) enables simulating DAEs for long-time horizons. We showcase the effectiveness and accuracy of DAE-PINN by learning and simulating the solution trajectories of a three-bus power network.

cs.LG↗

arXiv2021

HEI: hybrid explicit-implicit learning for multiscale problems

Splitting is a method to handle application problems by splitting physics, scales, domain, and so on. Many splitting algorithms have been designed for efficient temporal discretization. In this paper, our goal is to use temporal splitting concepts in designing machine learning algorithms and, at the same time, help splitting algorithms by incorporating data and speeding them up. Since the spitting solution usually has an explicit and implicit part, we will call our method hybrid explicit-implict (HEI) learning. We will consider a recently introduced multiscale splitting algorithms. To approximate the dynamics, only a few degrees of freedom are solved implicitly, while others explicitly. In this paper, we use this splitting concept in machine learning and propose several strategies. First, the implicit part of the solution can be learned as it is more difficult to solve, while the explicit part can be computed. This provides a speed-up and data incorporation for splitting approaches. Secondly, one can design a hybrid neural network architecture because handling explicit parts requires much fewer communications among neurons and can be done efficiently. Thirdly, one can solve the coarse grid component via PDEs or other approximation methods and construct simpler neural networks for the explicit part of the solutions. We discuss these options and implement one of them by interpreting it as a machine translation task. This interpretation successfully enables us using the Transformer since it can perform model reduction for multiple time series and learn the connection. We also find that the splitting scheme is a great platform to predict the coarse solution with insufficient information of the target model: the target problem is partially given and we need to solve it through a known problem. We conduct four numerical examples and the results show that our method is stable and accurate.

math.NA↗

arXiv2021

NH-PINN: Neural homogenization based physics-informed neural network for multiscale problems

Physics-informed neural network (PINN) is a data-driven approach to solve equations. It is successful in many applications; however, the accuracy of the PINN is not satisfactory when it is used to solve multiscale equations. Homogenization is a way of approximating a multiscale equation by a homogenized equation without multiscale property; it includes solving cell problems and the homogenized equation. The cell problems are periodic; and we propose an oversampling strategy which greatly improves the PINN accuracy on periodic problems. The homogenized equation has constant or slow dependency coefficient and can also be solved by PINN accurately. We hence proposed a 3-step method to improve the PINN accuracy for solving multiscale problems with the help of the homogenization. We apply our method to solve three equations which represent three different homogenization. The results show that the proposed method greatly improves the PINN accuracy. Besides, we also find that the PINN aided homogenization may achieve better accuracy than the numerical methods driven homogenization; PINN hence is a potential alternative to implementing the homogenization.

math.NA↗

arXiv2021

Multi-variance replica exchange stochastic gradient MCMC for inverse and forward Bayesian physics-informed neural network

Physics-informed neural network (PINN) has been successfully applied in solving a variety of nonlinear non-convex forward and inverse problems. However, the training is challenging because of the non-convex loss functions and the multiple optima in the Bayesian inverse problem. In this work, we propose a multi-variance replica exchange stochastic gradient Langevin diffusion method to tackle the challenge of the multiple local optima in the optimization and the challenge of the multiple modal posterior distribution in the inverse problem. Replica exchange methods are capable of escaping from the local traps and accelerating the convergence. However, it may not be efficient to solve mathematical inversion problems by using the vanilla replica method directly since the method doubles the computational cost in evaluating the forward solvers (likelihood functions) in the two chains. To address this issue, we propose to make different assumptions on the energy function estimation and this facilities one to use solvers of different fidelities in the likelihood function evaluation. We give an unbiased estimate of the swapping rate and give an estimation of the discretization error of the scheme. To verify our idea, we design and solve four inverse problems which have multiple modes. The proposed method is also employed to train the Bayesian PINN to solve the forward and inverse problems; faster and more accurate convergence has been observed when compared to the stochastic gradient Langevin diffusion (SGLD) method and vanila replica exchange methods.

math.NA↗

arXiv2021

A consistent and conservative volume distribution algorithm and its applications to multiphase flows using Phase-Field models

In the present study, the multiphase volume distribution problem, where there can be an arbitrary number of phases, is addressed using a consistent and conservative volume distribution algorithm. The proposed algorithm satisfies the summation constraint, the conservation constraint, and the \textit{consistency of reduction}. The first application of the volume distribution algorithm is to determine the Lagrange multipliers in multiphase Phase-Field models that enforce the mass conservation, and a multiphase conservative Allen-Cahn model that satisfies the \textit{consistency of reduction} is developed. A corresponding consistent and conservative numerical scheme is developed for the model. The multiphase conservative Allen-Cahn model has a better ability than the multiphase Cahn-Hilliard model to preserve under-resolved structures. The second application is to develop a numerical procedure, called the boundedness mapping, to map the order parameters, obtained numerically from a multiphase model, into their physical interval, and at the same time to preserve the physical properties of the order parameters. Along with the consistent and conservative schemes for the multiphase Phase-Field models, the numerical solutions of the order parameters are reduction consistent, conservative, and bounded, which are theoretically analyzed and numerically validated. Then, the multiphase Phase-Field models are coupled with the momentum equation by satisfying the \textit{consistency of mass conservation} and the \textit{consistency of mass and momentum transport}, thanks to the consistent formulation. It is demonstrated that the proposed model and scheme converge to the sharp-interface solution and are capable of capturing the complicated multiphase dynamics even when there is a large density and/or viscosity ratio.

physics.comp-ph↗

arXiv2021

Augmented Gaussian Random Field: Theory and Computation

We propose the novel augmented Gaussian random field (AGRF), which is a universal framework incorporating the data of observable and derivatives of any order. Rigorous theory is established. We prove that under certain conditions, the observable and its derivatives of any order are governed by a single Gaussian random field, which is the aforementioned AGRF. As a corollary, the statement ``the derivative of a Gaussian process remains a Gaussian process'' is validated, since the derivative is represented by a part of the AGRF. Moreover, a computational method corresponding to the universal AGRF framework is constructed. Both noiseless and noisy scenarios are considered. Formulas of the posterior distributions are deduced in a nice closed form. A significant advantage of our computational method is that the universal AGRF framework provides a natural way to incorporate arbitrary order derivatives and deal with missing data. We use four numerical examples to demonstrate the effectiveness of the computational method. The numerical examples are composite function, damped harmonic oscillator, Korteweg-De Vries equation, and Burgers' equation.

math.ST↗

arXiv2021

A consistent and conservative model and its scheme for $N$-phase-$M$-component incompressible flows

In the present work, we propose a consistent and conservative model for multiphase and multicomponent incompressible flows, where there can be arbitrary numbers of phases and components. Each phase has a background fluid called the pure phase, each pair of phases is immiscible, and components are dissolvable in some specific phases. The model is developed based on the multiphase Phase-Field model including the contact angle boundary condition, the diffuse domain approach, and the analyses on the proposed consistency conditions for multiphase and multicomponent flows. The model conserves the mass of individual pure phases, the amount of each component in its dissolvable region, and thus the mass of the fluid mixture, and the momentum of the flow. It ensures that no fictitious phases or components can be generated and that the summation of the volume fractions from the Phase-Field model is unity everywhere so that there is no local void or overfilling. It satisfies a physical energy law and it is Galilean invariant. A corresponding numerical scheme is developed for the proposed model, whose formal accuracy is 2nd-order in both time and space. It is shown to be consistent and conservative and its solution is demonstrated to preserve the Galilean invariance and energy law. Numerical tests indicate that the proposed model and scheme are effective and robust to study various challenging multiphase and multicomponent flows.

physics.comp-ph↗

arXiv2021

Non-convex Learning via Replica Exchange Stochastic Gradient MCMC

Replica exchange Monte Carlo (reMC), also known as parallel tempering, is an important technique for accelerating the convergence of the conventional Markov Chain Monte Carlo (MCMC) algorithms. However, such a method requires the evaluation of the energy function based on the full dataset and is not scalable to big data. The naïve implementation of reMC in mini-batch settings introduces large biases, which cannot be directly extended to the stochastic gradient MCMC (SGMCMC), the standard sampling method for simulating from deep neural networks (DNNs). In this paper, we propose an adaptive replica exchange SGMCMC (reSGMCMC) to automatically correct the bias and study the corresponding properties. The analysis implies an acceleration-accuracy trade-off in the numerical discretization of a Markov jump process in a stochastic environment. Empirically, we test the algorithm through extensive experiments on various setups and obtain the state-of-the-art results on CIFAR10, CIFAR100, and SVHN in both supervised learning and semi-supervised learning tasks.

stat.ML↗

arXiv2021

Accelerating Convergence of Replica Exchange Stochastic Gradient MCMC via Variance Reduction

Replica exchange stochastic gradient Langevin dynamics (reSGLD) has shown promise in accelerating the convergence in non-convex learning; however, an excessively large correction for avoiding biases from noisy energy estimators has limited the potential of the acceleration. To address this issue, we study the variance reduction for noisy energy estimators, which promotes much more effective swaps. Theoretically, we provide a non-asymptotic analysis on the exponential acceleration for the underlying continuous-time Markov jump process; moreover, we consider a generalized Girsanov theorem which includes the change of Poisson measure to overcome the crude discretization based on the Gröwall's inequality and yields a much tighter error in the 2-Wasserstein ($\mathcal{W}_2$) distance. Numerically, we conduct extensive experiments and obtain the state-of-the-art results in optimization and uncertainty estimates for synthetic experiments and image data.

stat.ML↗

arXiv2021

DeepLight: Deep Lightweight Feature Interactions for Accelerating CTR Predictions in Ad Serving

Click-through rate (CTR) prediction is a crucial task in online display advertising. The embedding-based neural networks have been proposed to learn both explicit feature interactions through a shallow component and deep feature interactions using a deep neural network (DNN) component. These sophisticated models, however, slow down the prediction inference by at least hundreds of times. To address the issue of significantly increased serving delay and high memory usage for ad serving in production, this paper presents \emph{DeepLight}: a framework to accelerate the CTR predictions in three aspects: 1) accelerate the model inference via explicitly searching informative feature interactions in the shallow component; 2) prune redundant layers and parameters at intra-layer and inter-layer level in the DNN component; 3) promote the sparsity of the embedding layer to preserve the most discriminant signals. By combining the above efforts, the proposed approach accelerates the model inference by 46X on Criteo dataset and 27X on Avazu dataset without any loss on the prediction accuracy. This paves the way for successfully deploying complicated embedding-based neural networks in production for ad serving.

cs.LG↗

arXiv2020

Vapor-liquid equilibrium predictions of n-alkane/nitrogen mixtures using neural networks

Understanding fluid phase behavior in high pressure and high temperature conditions is crucial for developing high-fidelity simulations of chemically reacting flows in liquid-fueled combustion systems. The study of vapor-liquid equilibrium (VLE) curves also forms an integral part of the design and modeling of the control processes in chemical and oil-gas industries. The main objective of this study was to develop data-driven models to predict VLE of Type III binary mixtures involving long-chained n-alkanes and nitrogen. Two data-driven models have been proposed in this study, each of which was competent in estimating VLE for the binary systems of C10/N2 and C12/N2, at pressures ranging up to 50-60 MPa. Both the models showed better performance (less average absolute percentage error) in predicting equilibrium pressure of the binary mixtures as compared to the VLE modeled using Peng-Robinson equation of state (PR-EOS). The data-driven models were also able to correctly trace the change in curvature of the vapor phase composition, close to the mixture critical point, at high pressures and temperatures-a feature of the n-alkane/nitrogen system VLE which the PR-EOS model fails to capture.

physics.flu-dyn↗

arXiv2020

SubTSBR to tackle high noise and outliers for data-driven discovery of differential equations

Data-driven discovery of differential equations has been an emerging research topic. We propose a novel algorithm subsampling-based threshold sparse Bayesian regression (SubTSBR) to tackle high noise and outliers. The subsampling technique is used for improving the accuracy of the Bayesian learning algorithm. It has two parameters: subsampling size and the number of subsamples. When the subsampling size increases with fixed total sample size, the accuracy of our algorithm goes up and then down. When the number of subsamples increases, the accuracy of our algorithm keeps going up. We demonstrate how to use our algorithm step by step and compare our algorithm with threshold sparse Bayesian regression (TSBR) for the discovery of differential equations. We show that our algorithm produces better results. We also discuss the merits of discovering differential equations from data and demonstrate how to discover models with random initial and boundary condition as well as models with bifurcations. The numerical examples are: (1) predator-prey model with noise, (2) shallow water equations with outliers, (3) heat diffusion with random initial and boundary condition, and (4) fish-harvesting problem with bifurcations.

stat.ML↗