arXiv subjects

Jiayu Zhai

Publications and source records attributed to Jiayu Zhai.

11 recordsLinked to original sources

arXiv2026

Predictive Moving Sample Method for Physics-Informed Neural Solvers of Time-Dependent PDEs

Time-dependent partial differential equations (PDEs) often develop sharp fronts, localized peaks, and other moving structures that occupy only a small portion of the space--time domain but dominate the approximation error. This makes fixed or uniformly sampled collocation strategies inefficient for physics-informed neural networks (PINNs), especially in high dimensions and over long-time prediction intervals. We propose the predictive moving sample method (PMSM), which builds on the moving sample method (MSM) in \cite{xu2026moving} by replacing its full time domain iterative training with a progressive time-stepping strategy and simplifying the velocity-field loss to further reduce the per-step cost. To improve practicality for long-time prediction, we further introduce the windowed-reset predictive moving sample method (WR-PMSM), which restricts extension training to an active time window and periodically resets the reference state, thereby reducing the growth of optimization cost while preserving global consistency through a final refinement stage. Across four representative benchmarks, PMSM consistently outperforms both standard PINNs and the original MSM under matched collocation budgets. These results suggest that transporting samples according to residual dynamics provides an effective and practical route to neural network solvers for time-dependent PDEs.

math.NA↗

arXiv2026

Moving sample method for solving time-dependent partial differential equations

Solving time-dependent partial differential equations (PDEs) that exhibit sharp gradients or local singularities is computationally demanding, as traditional physics-informed neural networks (PINNs) often suffer from inefficient point allocation that wastes resources on regions already well-resolved. This paper presents an adaptive sampling framework for PINNs aimed at efficiently solving time-dependent partial differential equations with pronounced local singularities. The method employs a residual-driven strategy, where the spatial-temporal distribution of training points is iteratively updated according to the error field from the previous iteration. This targeted allocation enables the network to concentrate computational effort on regions with significant residuals, achieving higher accuracy with fewer sampling points compared to uniform sampling. Numerical experiments on representative PDE benchmarks demonstrate that the proposed approach improves solution quality.

math.NA↗

arXiv2025

A Discrete Neural Operator with Adaptive Sampling for Surrogate Modeling of Parametric Transient Darcy Flows in Porous Media

This study proposes a new discrete neural operator for surrogate modeling of transient Darcy flow fields in heterogeneous porous media with random parameters. The new method integrates temporal encoding, operator learning and UNet to approximate the mapping between vector spaces of random parameter and spatiotemporal flow fields. The new discrete neural operator can achieve higher prediction accuracy than the SOTA attention-residual-UNet structure. Derived from the finite volume method, the transmissibility matrices rather than permeability is adopted as the inputs of surrogates to enhance the prediction accuracy further. To increase sampling efficiency, a generative latent space adaptive sampling method is developed employing the Gaussian mixture model for density estimation of generalization error. Validation is conducted on test cases of 2D/3D single- and two-phase Darcy flow field prediction. Results reveal consistent enhancement in prediction accuracy given limited training set.

math.NA↗

arXiv2025

Nonlinear Expectation Inference for Efficient Uncertainty Quantification and History Matching of Transient Darcy Flows in Porous Media with Random Parameters Under Distribution Uncertainty

The uncertainty quantification of Darcy flows using history matching is important for the evaluation and prediction of subsurface reservoir performance. Conventional methods aim to obtain the maximum a posterior or maximum likelihood estimate (MLE) using gradient-based, heuristic or ensemble-based methods. These methods can be computationally expensive for high-dimensional problems since forward simulation needs to be run iteratively as physical parameters are updated. In the current study, we propose a nonlinear expectation inference (NEI) method for efficient history matching and uncertainty quantification accounting for distribution or Knightian uncertainty. Forward simulation runs are conducted on prior realisations once, and then a range of expectations are computed in the data space based on subsets of prior realisations with no repetitive forward runs required. In NEI, no prior probability distribution for data is assumed. Instead, the probability distribution is assumed to be uncertain with prior and posterior uncertainty quantified by nonlinear expectations. The inferred result of NEI is the posterior subsets on which the expected flow rates are consistent with observation. The accuracy and efficiency of the new method are validated using single- and two-phase Darcy flows in 2D and 3D heterogeneous reservoirs.

math.NA↗

arXiv2024

Adversarial Adaptive Sampling: Unify PINN and Optimal Transport for the Approximation of PDEs

Solving partial differential equations (PDEs) is a central task in scientific computing. Recently, neural network approximation of PDEs has received increasing attention due to its flexible meshless discretization and its potential for high-dimensional problems. One fundamental numerical difficulty is that random samples in the training set introduce statistical errors into the discretization of loss functional which may become the dominant error in the final approximation, and therefore overshadow the modeling capability of the neural network. In this work, we propose a new minmax formulation to optimize simultaneously the approximate solution, given by a neural network model, and the random samples in the training set, provided by a deep generative model. The key idea is to use a deep generative model to adjust random samples in the training set such that the residual induced by the approximate PDE solution can maintain a smooth profile when it is being minimized. Such an idea is achieved by implicitly embedding the Wasserstein distance between the residual-induced distribution and the uniform distribution into the loss, which is then minimized together with the residual. A nearly uniform residual profile means that its variance is small for any normalized weight function such that the Monte Carlo approximation error of the loss functional is reduced significantly for a certain sample size. The adversarial adaptive sampling (AAS) approach proposed in this work is the first attempt to formulate two essential components, minimizing the residual and seeking the optimal training set, into one minmax objective functional for the neural network approximation of PDEs.

stat.ML↗

arXiv2024

Deep adaptive sampling for surrogate modeling without labeled data

Surrogate modeling is of great practical significance for parametric differential equation systems. In contrast to classical numerical methods, using physics-informed deep learning methods to construct simulators for such systems is a promising direction due to its potential to handle high dimensionality, which requires minimizing a loss over a training set of random samples. However, the random samples introduce statistical errors, which may become the dominant errors for the approximation of low-regularity and high-dimensional problems. In this work, we present a deep adaptive sampling method for surrogate modeling ($\text{DAS}^2$), where we generalize the deep adaptive sampling (DAS) method [62] [Tang, Wan and Yang, 2023] to build surrogate models for low-regularity parametric differential equations. In the parametric setting, the residual loss function can be regarded as an unnormalized probability density function (PDF) of the spatial and parametric variables. This PDF is approximated by a deep generative model, from which new samples are generated and added to the training set. Since the new samples match the residual-induced distribution, the refined training set can further reduce the statistical error in the current approximate solution. We demonstrate the effectiveness of $\text{DAS}^2$ with a series of numerical experiments, including the parametric lid-driven 2D cavity flow problem with a continuous range of Reynolds numbers from 100 to 1000.

math.NA↗

arXiv2023

Bregman Graph Neural Network

Numerous recent research on graph neural networks (GNNs) has focused on formulating GNN architectures as an optimization problem with the smoothness assumption. However, in node classification tasks, the smoothing effect induced by GNNs tends to assimilate representations and over-homogenize labels of connected nodes, leading to adverse effects such as over-smoothing and misclassification. In this paper, we propose a novel bilevel optimization framework for GNNs inspired by the notion of Bregman distance. We demonstrate that the GNN layer proposed accordingly can effectively mitigate the over-smoothing issue by introducing a mechanism reminiscent of the "skip connection". We validate our theoretical results through comprehensive empirical studies in which Bregman-enhanced GNNs outperform their original counterparts in both homophilic and heterophilic graphs. Furthermore, our experiments also show that Bregman GNNs can produce more robust learning accuracy even when the number of layers is high, suggesting the effectiveness of the proposed method in alleviating the over-smoothing issue.

cs.LG↗

arXiv2020

A deep learning method for solving Fokker-Planck equations

The time evolution of the probability distribution of a stochastic differential equation follows the Fokker-Planck equation, which usually has an unbounded, high-dimensional domain. Inspired by our early study in \cite{li2018data}, we propose a mesh-free Fokker-Planck solver, in which the solution to the Fokker-Planck equation is now represented by a neural network. The presence of the differential operator in the loss function improves the accuracy of the neural network representation and reduces the the demand of data in the training process. Several high dimensional numerical examples are demonstrated.

math.NA↗

arXiv2019

Using coupling methods to estimate sample quality for stochastic differential equations

A probabilistic approach for estimating sample qualities for stochastic differential equations is introduced in this paper. The aim is to provide a quantitative upper bound of the distance between the invariant probability measure of a stochastic differential equation and that of its numerical approximation. In order to extend estimates of finite time truncation error to infinite time, it is crucial to know the rate of contraction of the transition kernel of the SDE. We find that suitable numerical coupling methods can effectively estimate such rate of contraction, which gives the distance between two invariant probability measures. Our algorithms are tested with several low and high dimensional numerical examples.

math.NA↗

arXiv2019

An efficient data-driven solver for Fokker-Planck equations: algorithm and analysis

Computing the invariant probability measure of a randomly perturbed dynamical system usually means solving the stationary Fokker-Planck equation. This paper studies several key properties of a novel data-driven solver for low-dimensional Fokker-Planck equations proposed in [15]. Based on these results, we propose a new `block solver' for the stationary Fokker-Planck equation, which significantly improves the performance of the original algorithm. Some possible ways of reducing numerical artifacts caused by the block solver are discussed and tested with examples.

math.NA↗

arXiv2018

Convergence analysis of a finite element approximation of minimum action methods

In this work, we address the convergence of a finite element approximation of the minimizer of the Freidlin-Wentzell (F-W) action functional for non-gradient dynamical systems perturbed by small noise. The F-W theory of large deviations is a rigorous mathematical tool to study small-noise-induced transitions in a dynamical system. The central task in the application of F-W theory of large deviations is to seek the minimizer and minimum of the F-W action functional. We discretize the F-W action functional using linear finite elements, and establish the convergence of {the approximation} through $Γ$-convergence.

math.NA↗