arXiv subjects

Jingrun Chen

Publications and source records attributed to Jingrun Chen.

At least 37 records · Page 2Linked to original sources

arXiv2025

Weak Random Feature Method for Solving Partial Differential Equations

The random feature method (RFM) has demonstrated great potential in bridging traditional numerical methods and machine learning techniques for solving partial differential equations (PDEs). It retains the advantages of mesh-free approaches while achieving spectral accuracy for smooth solutions, without the need for iterative procedures. However, the implementation of RFM in the identification of weak solutions remains a subject of limited comprehension, despite crucial role of weak solutions in addressing numerous applied problems. While the direct application of RFM to problems without strong solutions is fraught with potential challenges, we propose an enhancement to the original random feature method that is specifically suited for finding weak solutions and is termed as Weak RFM. Essentially, Weak RFM reformulates the original RFM by adopting the weak form of the governing equations and constructing a new linear system through the use of carefully designed test functions, ensuring that the resulting solution satisfies the weak form by default. To rigorously evaluate the performance of the proposed method, we conduct extensive experiments on a variety of benchmark problems, including challenging three-dimensional cases, and compare its performance with state of the art machine learning-based approaches. The results demonstrate that Weak RFM achieves comparable or superior accuracy while significantly reducing computational time and memory consumption, highlighting its potential as a highly efficient and robust tool for finding weak solutions to various PDE problems.

math.NA↗

arXiv2025

An Unsupervised Network Architecture Search Method for Solving Partial Differential Equations

Solving partial differential equations (PDEs) has been indispensable in scientific and engineering applications. Recently, deep learning methods have been widely used to solve high-dimensional problems, one of which is the physics-informed neural network (PINN). Typically, a deep learning method has three main components: a neural network, a loss function, and an optimizer. While the construction of the loss function is rooted in the definition of solution space, how to choose a optimal neural network is somewhat ad hoc, leaving much room for improvement. In the framework of PINN, we propose an unsupervised network architecture search method for solving PDEs, termed PINN-DARTS, which applies the differentiable architecture search (DARTS) to find the optimal network architecture structure in a given set of neural networks. In this set, the number of layers and the number of neurons in each layer can change. In the searching phase, both network and architecture parameters are updated simultaneously, so the running time is close to that of PINN with a pre-determined network structure. Unlike available works, our approach is unsupervised and purely based on the PDE residual without any prior usage of solutions. PINN-DARTS outputs the optimal network structure as well as the associated numerical solution. The performance of PINN-DARTS is verified on several benchmark PDEs, including elliptic, parabolic, wave, and Burgers' equations. Compared to traditional architecture search methods, PINN-DARTS achieves significantly higher architectural accuracy. Another interesting observation is that both the solution complexity and the PDE type have a prominent impact on the optimal network architecture. Our study suggests that architectures with uneven widths from layer to layer may have superior performance across different solution complexities and different PDE types.

math.NA↗

arXiv2025

STSA: Spatial-Temporal Semantic Alignment for Visual Dubbing

Existing audio-driven visual dubbing methods have achieved great success. Despite this, we observe that the semantic ambiguity between spatial and temporal domains significantly degrades the synthesis stability for the dynamic faces. We argue that aligning the semantic features from spatial and temporal domains is a promising approach to stabilizing facial motion. To achieve this, we propose a Spatial-Temporal Semantic Alignment (STSA) method, which introduces a dual-path alignment mechanism and a differentiable semantic representation. The former leverages a Consistent Information Learning (CIL) module to maximize the mutual information at multiple scales, thereby reducing the manifold differences between spatial and temporal domains. The latter utilizes probabilistic heatmap as ambiguity-tolerant guidance to avoid the abnormal dynamics of the synthesized faces caused by slight semantic jittering. Extensive experimental results demonstrate the superiority of the proposed STSA, especially in terms of image quality and synthesis stability. Pre-trained weights and inference code are available at https://github.com/SCAILab-USTC/STSA.

cs.CV↗

arXiv2025

A PINN-enriched finite element method for linear elliptic problems

In this paper, we propose a hybrid method that combines finite element method (FEM) and physics-informed neural network (PINN) for solving linear elliptic problems. This method contains three steps: (1) train a PINN and obtain an approximate solution $u_θ$; (2) enrich the finite element space with $u_θ$; (3) obtain the final solution by FEM in the enriched space. In the second step, the enriched space is constructed by addition $v + u_θ$ or multiplication $v \cdot u_θ$, where $v$ belongs to the standard finite element space. We conduct the convergence analysis for the proposed method. Compared to the standard FEM, the same convergence order is obtained and higher accuracy can be achieved when solution derivatives are well approximated in PINN. Numerical examples from one dimension to three dimensions verify these theoretical results. For some examples, the accuracy of the proposed method can be reduced by a couple of orders of magnitude compared to the standard FEM.

math.NA↗

arXiv2025

Consensus Based Stochastic Control

We propose a gradient-free deep reinforcement learning algorithm to solve high-dimensional, finite-horizon stochastic control problems. Although the recently developed deep reinforcement learning framework has achieved great success in solving these problems, direct estimation of policy gradients from Monte Carlo sampling often suffers from high variance. To address this, we introduce the Momentum Consensus-Based Optimization (M-CBO) and Adaptive Momentum Consensus-Based Optimization (Adam-CBO) frameworks. These methods optimize policies using Monte Carlo estimates of the value function, rather than its gradients. Adjustable Gaussian noise supports efficient exploration, helping the algorithm converge to optimal policies in complex, nonconvex environments. Numerical results confirm the accuracy and scalability of our approach across various problem dimensions and show the potential for extension to mean-field control problems. Theoretically, we prove that M-CBO can converge to the optimal policy under some assumptions.

math.OC↗

arXiv2024

Random Batch Method with Momentum Correction

The Random Batch Method (RBM) is an effective technique to reduce the computational complexity when solving certain stochastic differential problems (SDEs) involving interacting particles. It can transform the computational complexity from O(N^2) to O(N), where N represents the number of particles. However, the traditional RBM can only be effectively applied to interacting particle systems with relatively smooth kernel functions to achieve satisfactory results. To address the issue of non-convergence of the RBM in particle interaction systems with significant singularities, we propose some enhanced methods to make the modified algorithm more applicable. The idea for improvement primarily revolves around a momentum-like correction, and we refer to the enhanced algorithm as the Random Batch Method with Momentum Correction ( RBM-M). We provide a theoretical proof to control the error of the algorithm, which ensures that under ideal conditions it has a smaller error than the original algorithm. Finally, numerical experiments have demonstrated the effectiveness of the RBM-M algorithm.

math.NA↗

arXiv2024

A Multigrid Graph U-Net Framework for Simulating Multiphase Flow in Heterogeneous Porous Media

Numerical simulation of multi-phase fluid dynamics in porous media is critical to a variety of geoscience applications. Data-driven surrogate models using Convolutional Neural Networks (CNNs) have shown promise but are constrained to regular Cartesian grids and struggle with unstructured meshes necessary for accurately modeling complex geological features in subsurface simulations. To tackle this difficulty, we build surrogate models based on Graph Neural Networks (GNNs) to approximate space-time solutions of multi-phase flow and transport processes. Particularly, a novel Graph U-Net framework, referred to as AMG-GU, is developed to enable hierarchical graph learning for the parabolic pressure component of the coupled partial differential equation (PDE) system. Drawing inspiration from aggregation-type Algebraic Multigrid (AMG), we propose a graph coarsening strategy adapted to heterogeneous PDE coefficients, achieving an effective graph pooling operation. Results of three-dimensional heterogeneous test cases demonstrate that the multi-level surrogates predict pressure and saturation dynamics with high accuracy, significantly outperforming the single-level baseline. Our Graph U-Net model exhibits great generalization capability to unseen model configurations.

physics.comp-ph↗

arXiv2024

Error Analysis of the Deep Mixed Residual Method for High-order Elliptic Equations

This paper presents an a priori error analysis of the Deep Mixed Residual method (MIM) for solving high-order elliptic equations with non-homogeneous boundary conditions, including Dirichlet, Neumann, and Robin conditions. We examine MIM with two types of loss functions, referred to as first-order and second-order least squares systems. By providing boundedness and coercivity analysis, we leverage Céa's Lemma to decompose the total error into the approximation, generalization, and optimization errors. Utilizing the Barron space theory and Rademacher complexity, an a priori error is derived regarding the training samples and network size that are exempt from the curse of dimensionality. Our results reveal that MIM significantly reduces the regularity requirements for activation functions compared to the deep Ritz method, implying the effectiveness of MIM in solving high-order equations.

math.NA↗

arXiv2024

High-precision randomized iterative methods for the random feature method

This paper focuses on solving large-scale, ill-conditioned, and overdetermined sparse least squares problems that arise from numerical partial differential equations (PDEs), mainly from the random feature method. To address these difficulties, we introduce (1) a count sketch technique to sketch the original matrix to a smaller matrix; (2) a QR factorization or a singular value decomposition for the smaller matrix to obtain the preconditioner, which is multiplied to the original matrix from the right-hand side; (3) least squares iterative solvers to solve the preconditioned least squares system. Therefore, the methods we develop are termed CSQRP-LSQR and CSSVDP-LSQR. Under mild assumptions, we prove that the preconditioned problem holds a condition number whose upper bound is independent of the condition number of the original matrix, and provide error estimates for both methods. Ample numerical experiments, including least squares problems arising from two-dimensional and three-dimensional PDEs and the Florida Sparse Matrix Collection, are conducted. Both methods are comparable to or even better than direct methods in accuracy and are computationally more efficient for large-scale problems. This opens up the applicability of the random feature method for PDEs over complicated geometries with high-complexity solutions.

math.NA↗

arXiv2024

R-adaptive DeepONet: Learning Solution Operators for PDEs with Discontinuous Solutions Using an R-adaptive Strategy

DeepONet has recently been proposed as a representative framework for learning nonlinear mappings between function spaces. However, when it comes to approximating solution operators of partial differential equations (PDEs) with discontinuous solutions, DeepONet poses a foundational approximation lower bound due to its linear reconstruction property. Inspired by the moving mesh (R-adaptive) method, we propose an R-adaptive DeepONet method, which contains the following components: (1) the output data representation is transformed from the physical domain to the computational domain using the equidistribution principle; (2) the maps from input parameters to the solution and the coordinate transformation function over the computational domain are learned using DeepONets separately; (3) the solution over the physical domain is obtained via post-processing methods such as the (linear) interpolation method. Additionally, we introduce a solution-dependent weighting strategy in the training process to reduce the final error. We establish an upper bound for the reconstruction error based on piecewise linear interpolation and show that the introduced R-adaptive DeepONet can reduce this bound. Moreover, for two prototypical PDEs with sharp gradients or discontinuities, we prove that the approximation error decays at a superlinear rate with respect to the trunk basis size, unlike the linear decay observed in vanilla DeepONets. Therefore, the R-adaptive DeepONet overcomes the limitations of DeepONet, and can reduce the approximation error for problems with discontinuous solutions. Numerical experiments on PDEs with discontinuous solutions, including the linear advection equation, the Burgers' equation with low viscosity, and the compressible Euler equations of gas dynamics, are conducted to verify the advantages of the R-adaptive DeepONet over available variants of DeepONet.

math.NA↗

arXiv2024

XI-DeepONet: An operator learning method for elliptic interface problems

Scientific computing has been an indispensable tool in applied sciences and engineering, where traditional numerical methods are often employed due to their superior accuracy guarantees. However, these methods often encounter challenges when dealing with problems involving complex geometries. Machine learning-based methods, on the other hand, are mesh-free, thus providing a promising alternative. In particular, operator learning methods have been proposed to learn the mapping from the input space to the solution space, enabling rapid inference of solutions to partial differential equations (PDEs) once trained. In this work, we address the parametric elliptic interface problem. Building upon the deep operator network (DeepONet), we propose an extended interface deep operator network (XI-DeepONet). XI-DeepONet exhibits three unique features: (1) The interface geometry is incorporated into the neural network as an additional input, enabling the network to infer solutions for new interface geometries once trained; (2) The level set function associated with the interface geometry is treated as the input, on which the solution mapping is continuous and can be effectively approximated by the deep operator network; (3) The network can be trained without any input-output data pairs, thus completely avoiding the need for meshes of any kind, directly or indirectly. We conduct a comprehensive series of numerical experiments to demonstrate the accuracy and robustness of the proposed method.

math.NA↗

arXiv2024

Continuum Limit of Spin Dynamics on Hexagonal Lattice

This study investigates the atomistic spin system in $\rm CrCl_{3}$, which exhibits topologically nontrivial meron structures within its layered hexagonal lattice framework. We analyze the complete model of discrete spin dynamics on a two-dimensional hexagonal lattice and demonstrate its convergence to the continuum Landau-Lifshitz-Gilbert equation in the weak sense. The primary challenge lies in defining appropriate difference quotient and interpolation operators for the hexagonal lattice since the loss of symmetry. To address these, we utilized a one-step difference quotient for the 2nd nearest neighbors and introduced novel multi-step difference quotients for the 1st and 3rd nearest neighbors, enabling the integration by parts formula. Additionally, we generalized Ladysenskaya's interpolation operator for hexagonal lattices and provided an alternative strategy for the convergence procedure by applying an isometric mapping property. This work provides necessary tools for analyzing weak convergence in other atomistic nonlinear problems on hexagonal lattices towards the continuum limit.

math-ph↗

arXiv2023

IMEX-RK methods for Landau-Lifshitz equation with arbitrary damping

Magnetization dynamics in ferromagnetic materials is modeled by the Landau-Lifshitz (LL) equation, a nonlinear system of partial differential equations. Among the numerical approaches, semi-implicit schemes are widely used in the micromagnetics simulation, due to a nice compromise between accuracy and efficiency. At each time step, only a linear system needs to be solved and a projection is then applied to preserve the length of magnetization. However, this linear system contains variable coefficients and a non-symmetric structure, and thus an efficient linear solver is highly desired. If the damping parameter becomes large, it has been realized that efficient solvers are only available to a linear system with constant, symmetric, and positive definite (SPD) structure. In this work, based on the implicit-explicit Runge-Kutta (IMEX-RK) time discretization, we introduce an artificial damping term, which is treated implicitly. The remaining terms are treated explicitly. This strategy leads to a semi-implicit scheme with the following properties: (1) only a few linear system with constant and SPD structure needs to be solved at each time step; (2) it works for the LL equation with arbitrary damping parameter; (3) high-order accuracy can be obtained with high-order IMEX-RK time discretization. Numerically, second-order and third-order IMEX-RK methods are designed in both the 1-D and 3-D domains. A comparison with the backward differentiation formula scheme is undertaken, in terms of accuracy and efficiency. The robustness of both numerical methods is tested on the first benchmark problem from National Institute of Standards and Technology. The linearized stability estimate and optimal rate convergence analysis are provided for an alternate IMEX-RK2 numerical scheme as well.

math.NA↗

arXiv2023

Exploring the Optimal Choice for Generative Processes in Diffusion Models: Ordinary vs Stochastic Differential Equations

The diffusion model has shown remarkable success in computer vision, but it remains unclear whether the ODE-based probability flow or the SDE-based diffusion model is more superior and under what circumstances. Comparing the two is challenging due to dependencies on data distributions, score training, and other numerical issues. In this paper, we study the problem mathematically for two limiting scenarios: the zero diffusion (ODE) case and the large diffusion case. We first introduce a pulse-shape error to perturb the score function and analyze error accumulation of sampling quality, followed by a thorough analysis for generalization to arbitrary error. Our findings indicate that when the perturbation occurs at the end of the generative process, the ODE model outperforms the SDE model with a large diffusion coefficient. However, when the perturbation occurs earlier, the SDE model outperforms the ODE model, and we demonstrate that the error of sample generation due to such a pulse-shape perturbation is exponentially suppressed as the diffusion term's magnitude increases to infinity. Numerical validation of this phenomenon is provided using Gaussian, Gaussian mixture, and Swiss roll distribution, as well as realistic datasets like MNIST and CIFAR-10.

cs.LG↗

arXiv2023

Solving multiscale elliptic problems by sparse radial basis function neural networks

Machine learning has been successfully applied to various fields of scientific computing in recent years. In this work, we propose a sparse radial basis function neural network method to solve elliptic partial differential equations (PDEs) with multiscale coefficients. Inspired by the deep mixed residual method, we rewrite the second-order problem into a first-order system and employ multiple radial basis function neural networks (RBFNNs) to approximate unknown functions in the system. To aviod the overfitting due to the simplicity of RBFNN, an additional regularization is introduced in the loss function. Thus the loss function contains two parts: the $L_2$ loss for the residual of the first-order system and boundary conditions, and the $\ell_1$ regularization term for the weights of radial basis functions (RBFs). An algorithm for optimizing the specific loss function is introduced to accelerate the training process. The accuracy and effectiveness of the proposed method are demonstrated through a collection of multiscale problems with scale separation, discontinuity and multiple scales from one to three dimensions. Notably, the $\ell_1$ regularization can achieve the goal of representing the solution by fewer RBFs. As a consequence, the total number of RBFs scales like $\mathcal{O}(\varepsilon^{-nτ})$, where $\varepsilon$ is the smallest scale, $n$ is the dimensionality, and $τ$ is typically smaller than $1$. It is worth mentioning that the proposed method not only has the numerical convergence and thus provides a reliable numerical solution in three dimensions when a classical method is typically not affordable, but also outperforms most other available machine learning methods in terms of accuracy and robustness.

math.NA↗

arXiv2023

Roughness Index for Loss Landscapes of Neural Network Models of Partial Differential Equations

Loss landscape is a useful tool to characterize and compare neural network models. The main challenge for analysis of loss landscape for the deep neural networks is that they are generally highly non-convex in very high dimensional space. In this paper, we develop "the roughness"concept for understanding such landscapes in high dimensions and apply this technique to study two neural network models arising from solving differential equations. Our main innovation is the proposal of a well-defined and easy-to-compute roughness index (RI) which is based on the mean and variance of the (normalized) total variation for one-dimensional functions projected on randomly sampled directions. A large RI at the local minimizer hints an oscillatory landscape profile and indicates a severe challenge for the first-order optimization method. Particularly, we observe the increasing-then-decreasing pattern for RI along the gradient descent path in most models. We apply our method to two types of loss functions used to solve partial differential equations (PDEs) when the solution of PDE is parametrized by neural networks. Our empirical results on these PDE problems reveal important and consistent observations that the landscapes from the deep Galerkin method around its local minimizers are less rough than the deep Ritz method.

math.NA↗

arXiv2023

The Random Feature Method for Solving Interface Problems

Interface problems have long been a major focus of scientific computing, leading to the development of various numerical methods. Traditional mesh-based methods often employ time-consuming body-fitted meshes with standard discretization schemes or unfitted meshes with tailored schemes to achieve controllable accuracy and convergence rate. Along another line, mesh-free methods bypass mesh generation but lack robustness in terms of convergence and accuracy due to the low regularity of solutions. In this study, we propose a novel method for solving interface problems within the framework of the random feature method. This approach utilizes random feature functions in conjunction with a partition of unity as approximation functions. It evaluates partial differential equations, boundary conditions, and interface conditions on collocation points in equal footing, and solves a linear least-squares system to obtain the approximate solution. To address the issue of low regularity, two sets of random feature functions are used to approximate the solution on each side of the interface, which are then coupled together via interface conditions. We validate our method through a series of increasingly complex numerical examples. Our findings show that despite the solution often being only continuous or even discontinuous, our method not only eliminates the need for mesh generation but also maintains high accuracy, akin to the spectral collocation method for smooth solutions. Remarkably, for the same accuracy requirement, our method requires two to three orders of magnitude fewer degrees of freedom than traditional methods, demonstrating its significant potential for solving interface problems with complex geometries.

math.NA↗

arXiv2023

The Random Feature Method for Time-dependent Problems

We present a framework for solving time-dependent partial differential equations (PDEs) in the spirit of the random feature method. The numerical solution is constructed using a space-time partition of unity and random feature functions. Two different ways of constructing the random feature functions are investigated: feature functions that treat the spatial and temporal variables (STC) on the same footing, or functions that are the product of two random feature functions depending on spatial and temporal variables separately (SoV). Boundary and initial conditions are enforced by penalty terms. We also study two ways of solving the resulting least-squares problem: the problem is solved as a whole or solved using the block time-marching strategy. The former is termed ``the space-time random feature method'' (ST-RFM). Numerical results for a series of problems show that the proposed method, i.e. ST-RFM with STC and ST-RFM with SoV, have spectral accuracy in both space and time. In addition, ST-RFM only requires collocation points, not a mesh. This is important for solving problems with complex geometry. We demonstrate this by using ST-RFM to solve a two-dimensional wave equation over a complex domain. The two strategies differ significantly in terms of the behavior in time. In the case when block time-marching is used, we prove a lower error bound that shows an exponentially growing factor with respect to the number of blocks in time. For ST-RFM, we prove an upper bound with a sublinearly growing factor with respect to the number of subdomains in time. These estimates are also confirmed by numerical results.

math.NA↗