arXiv subjects

Zhiqiang Cai

Publications and source records attributed to Zhiqiang Cai.

At least 19 recordsLinked to original sources

arXiv2026

Generating Samples of Stationary Distributions of Weakly Interacting Diffusion Models Without Finite Particle Truncation: A Weak Generative Approach

Computing the stationary probability density and generating corresponding samples for the mean-field model of an infinite number of weakly interacting diffusion particles pose significant numerical challenges, particularly in the phase transition regime where the interchangeability of infinite-time and infinite-particle limits breaks down. Traditional approaches, such as direct simulation of finite-particle systems, often fail to accurately pinpoint multiple stationary distributions in the mean-field meta-stable setting. On the other hand, solving the high-dimensional McKean-Vlasov partial differential equation using neural networks typically yields only the density function, limiting its utility for estimating statistical quantities from generating samples. In this work, we propose a novel generative framework based on the weak PDE formulation of the mean-field model to address these challenges. Our approach simultaneously computes the stationary distributions of McKean-Vlasov processes and generates independent and identically distributed samples that satisfy these distributions. This integrated approach not only reveals the true stationary distributions without the random perturbation of finite particle truncation, but also offers deeper insight into the system's behavior in the mean-field limit. Extensive numerical experiments demonstrate the effectiveness of the proposed method, showcasing its ability to accurately approximate stationary distributions, capture intricate phase transitions, and handle high-dimensional complex systems.

physics.comp-ph↗

arXiv2026

DyneTrion: A Spatio-temporally Coherent Generative Emulator for Protein Dynamics Across Timescales

Proteins function through coordinated motion across multiple spatial and temporal scales, underpinning processes such as ligand binding, allostery, and catalysis. However, accessing long-timescale conformational change through molecular dynamics (MD) simulations remains prohibitively expensive for systematic exploration across diverse systems. Here, we present DyneTrion, a generative protein dynamics emulator that jointly enforces geometric symmetry, structural consistency and temporal coherence within a single framework. DyneTrion uses a tri-attention architecture that integrates invariant point attention (IPA) for SE(3)-robust geometric updates, spatial attention anchored to a reference conformation to preserve structural integrity, and temporal attention to model correlated evolution across time frames. Across 100-ns MD trajectory simulation benchmarks, DyneTrion reproduces MD-derived flexibility, ensemble distributions and interaction observables while maintaining stereochemical validity during extrapolation. To evaluate long time-scale generalization, we introduce dynamicPDB, a dataset of over 10,000 proteins with up to 1-$μ$s all-atom trajectories at 10-ps resolution and accompanying physical annotations. On microsecond trajectories, DyneTrion preserves free-energy landscapes and metastable-state populations, and it supports large conformational propagation in apo-to-holo transitions and fast folders. Together, DyneTrion provides a scalable path from static structure prediction toward time-resolved, ensemble-faithful protein modeling. The code is publicly available at https://github.com/fudan-generative-vision/DyneTrion

q-bio.QM↗

arXiv2026

Structure-Guided Gauss-Newton Method: Linear Advection-Reaction Equation

The least-squares neural network (LSNN) method introduced in [5] for linear advection-reaction equations is capable of accurately approximating discontinuous solutions without a priori knowledge of the interface location. However, the resulting discretization is a non-convex optimization problem that is computationally intensive and complex. In this paper, we propose a structure-guided Gauss-Newton (SgGN) method that alternates between the linear (output) and the nonlinear (hidden layer) parameters. At each outer iteration, the linear parameters are computed by a linear solver, and the nonlinear parameters are updated by a modified Gauss-Newton (GN) method that explicitly removes the singularities of the GN matrix. Numerical experiments for all test problems presented in [5] show that the SgGN method is superior to the Adam optimizer [13], the commonly used first-order optimization algorithm, not only in computational cost but, more importantly, in accuracy

math.NA↗

arXiv2026

Learning Implicit Bias in Generative Spaces for Accelerating Protein Dynamics Emulation

Generative emulators of protein dynamics produce plausible trajectories at a fraction of the cost of molecular dynamics, but they inherit their training distribution and tend to revisit known states rather than reach rare ones under long-horizon extrapolation. Inspired by classical enhanced sampling, we introduce an implicit, history-dependent bias in the generative space of a pretrained emulator. Specifically, a history-aware score estimator augments the frozen emulator with a distance-weighted bias that steers reverse-time sampling away from previously generated structures, regularized by an environment-support term. To preserve structural validity at long horizons, a score-based refinement step re-projects drifted samples onto the data manifold using the frozen emulator. Our experiments demonstrate that the method (i) raises diversity by $35\%$ on DynamicPDB-80; (ii) on $12$ zero-shot Fast-Folding proteins, the learned bias alone reaches the unbiased emulator's coverage up to ${\sim}15\times$ faster, and pairing it with refinement reaches the coverage up to ${\sim}37\times$ faster while covering ${\sim}3\times$ as many low-energy states. Code will be released soon.

cs.LG↗

arXiv2026

Convergence Analysis of Block Newton Methods for 1D Shallow Neural Network Approximation

This paper analyzes local convergence of the block Newton (BN) method introduced in [5, 6] for one-dimensional shallow neural network approximation to functions and diffusion-reaction problems. The BN method consists of the 2x2 block nonlinear Gauss-Seidel, linear Gauss-Seidel, or Jacobi method for outer iteration and the Newton method for inner iteration. The blocks are corresponding to the linear and the nonlinear parameters. Under some reasonable assumptions, we establish local convergence of the BN methods as well as the reduced BN (rBN) method for one-dimensional diffusion-reaction problems and least-squares function approximation. Unlike common optimization methods, the rBN allows for the reduction of the number of parameters during the optimization process when some neurons contribute little to the approximation or are at nearly optimal locations.

math.NA↗

arXiv2026

Least-Squares Neural Network (LSNN) Method for Scalar Hyperbolic Partial Differential Equations

This chapter offers a comprehensive introduction to the least-squares neural network (LSNN) method introduced in [14,16], for solving scalar first-order hyperbolic partial differential equations, specifically linear advection-reaction equations and nonlinear hyperbolic conservation laws. The LSNN method is built on an equivalent least-squares formulation of the underlying problem on an admissible solution set that accommodates discontinuous solutions. It employs ReLU neural networks (in place of finite elements) as the approximating functions, uses a carefully designed physics-preserved numerical differentiation, and avoids penalization techniques such as artificial viscosity, entropy condition, and/or total variation. This approach captures shock features in the solution without oscillations or overshooting. Efficiently and reliably solving the resulting non-convex optimization problem posed by the LSNN method remains an open challenge. This chapter concludes with a brief discussion on application of the structure-guided Gauss-Newton (SgGN) method developed recently in [21] for solving shallow NN approximation.

math.NA↗

arXiv2025

Efficient Shallow Ritz Method For 1D Diffusion Problems

This paper studies the shallow Ritz method for solving the one-dimensional diffusion problem. It is shown that the shallow Ritz method improves the order of approximation dramatically for non-smooth problems. To realize this optimal or nearly optimal order of the shallow Ritz approximation, we develop a damped block Newton (dBN) method that alternates between updates of the linear and non-linear parameters. Per each iteration, the linear and the non-linear parameters are updated by exact inversion and one step of a modified, damped Newton method applied to a reduced non-linear system, respectively. The computational cost of each dBN iteration is $O(n)$. Starting with the non-linear parameters as a uniform partition of the interval, numerical experiments show that the dBN is capable of efficiently moving mesh points to nearly optimal locations. To improve efficiency of the dBN further, we propose an adaptive damped block Newton (AdBN) method by combining the dBN with the adaptive neuron enhancement (ANE) method [26].

math.NA↗

arXiv2025

Efficient Shallow Ritz Method For 1D Diffusion-Reaction Problems

This paper studies the shallow Ritz method for solving one-dimensional diffusion-reaction problems. The method is capable of improving the order of approximation for non-smooth problems. By following a similar approach to the one presented in [9], we present a damped block Newton (dBN) method to achieve nearly optimal order of approximation. The dBN method optimizes the Ritz functional by alternating between the linear and non-linear parameters of the shallow ReLU neural network (NN). For diffusion-reaction problems, new difficulties arise: (1) for the linear parameters, the mass matrix is dense and even more ill-conditioned than the stiffness matrix, and (2) for the non-linear parameters, the Hessian matrix is dense and may be singular. This paper addresses these challenges, resulting in a dBN method with computational cost of ${\cal O}(n)$. The ideas presented for diffusion-reaction problems can also be applied to least-squares approximation problems. For both applications, starting with the non-linear parameters as a uniform partition, numerical experiments show that the dBN method moves the mesh points to nearly optimal locations.

math.NA↗

arXiv2025

A Structure-Guided Gauss-Newton Method for Shallow ReLU Neural Network

In this paper, we propose a structure-guided Gauss-Newton (SgGN) method for solving least squares problems using a shallow ReLU neural network. The method effectively takes advantage of both the least squares structure and the neural network structure of the objective function. By categorizing the weights and biases of the hidden and output layers of the network as nonlinear and linear parameters, respectively, the method iterates back and forth between the nonlinear and linear parameters. The nonlinear parameters are updated by a damped Gauss-Newton method and the linear ones are updated by a linear solver. Moreover, at the Gauss-Newton step, a special form of the Gauss-Newton matrix is derived for the shallow ReLU neural network and is used for efficient iterations. It is shown that the corresponding mass and Gauss-Newton matrices in the respective linear and nonlinear steps are symmetric and positive definite under reasonable assumptions. Thus, the SgGN method naturally produces an effective search direction without the need of additional techniques like shifting in the Levenberg-Marquardt method to achieve invertibility of the Gauss-Newton matrix. The convergence and accuracy of the method are demonstrated numerically for several challenging function approximation problems, especially those with discontinuities or sharp transition layers that pose significant challenges for commonly used training algorithms in machine learning.

cs.LG↗

arXiv2025

Weak Generative Sampler to Efficiently Sample Invariant Distribution of Stochastic Differential Equation

Sampling invariant distributions from an Itô diffusion process presents a significant challenge in stochastic simulation. Traditional numerical solvers for stochastic differential equations require both a fine step size and a lengthy simulation period, resulting in biased and correlated samples. The current deep learning-based method solves the stationary Fokker--Planck equation to determine the invariant probability density function in the form of deep neural networks, but they generally do not directly address the problem of sampling from the computed density function. In this work, we introduce a framework that employs a weak generative sampler (WGS) to directly generate independent and identically distributed (iid) samples induced by a transformation map derived from the stationary Fokker--Planck equation. Our proposed loss function is based on the weak form of the Fokker--Planck equation, integrating normalizing flows to characterize the invariant distribution and facilitate sample generation from a base distribution. Our randomized test function circumvents the need for min-max optimization in the traditional weak formulation. Our method necessitates neither the computationally intensive calculation of the Jacobian determinant nor the invertibility of the transformation map. A crucial component of our framework is the adaptively chosen family of test functions in the form of Gaussian kernel functions with centers related to the generated data samples. Experimental results on several benchmark examples demonstrate the effectiveness and scalability of our method, which offers both low computational costs and excellent capability in exploring multiple metastable states.

cs.LG↗

arXiv2024

Fractional-order dependent Radial basis functions meshless methods for the integral fractional Laplacian

We study the numerical evaluation of the integral fractional Laplacian and its application in solving fractional diffusion equations. We derive a pseudo-spectral formula for the integral fractional Laplacian operator based on fractional order-dependent, generalized multi-quadratic radial basis functions (RBFs) to address efficient computation of the hyper-singular integral. We apply the proposed formula to solving fractional diffusion equations and design a simple, easy-to-implement and nearly integration-free meshless method. We discuss the convergence of the novel meshless method through equivalent Galerkin formulations. We carry out numerical experiments to demonstrate the accuracy and efficiency of the proposed approach compared to the existing method using Gaussian RBFs.

math.NA↗

arXiv2024

ReLU neural network approximation to piecewise constant functions

This paper studies the approximation property of ReLU neural networks (NNs) to piecewise constant functions with unknown interfaces in bounded regions in $\mathbb{R}^d$. Under the assumption that the discontinuity interface $Γ$ may be approximated by a connected series of hyperplanes with a prescribed accuracy $\varepsilon >0$, we show that a three-layer ReLU NN is sufficient to accurately approximate any piecewise constant function and establish its error bound. Moreover, if the discontinuity interface is convex, an analytical formula of the ReLU NN approximation with exact weights and biases is provided.

math.FA↗

arXiv2024

Dynamic PDB: A New Dataset and a SE(3) Model Extension by Integrating Dynamic Behaviors and Physical Properties in Protein Structures

Despite significant progress in static protein structure collection and prediction, the dynamic behavior of proteins, one of their most vital characteristics, has been largely overlooked in prior research. This oversight can be attributed to the limited availability, diversity, and heterogeneity of dynamic protein datasets. To address this gap, we propose to enhance existing prestigious static 3D protein structural databases, such as the Protein Data Bank (PDB), by integrating dynamic data and additional physical properties. Specifically, we introduce a large-scale dataset, Dynamic PDB, encompassing approximately 12.6K proteins, each subjected to all-atom molecular dynamics (MD) simulations lasting 1 microsecond to capture conformational changes. Furthermore, we provide a comprehensive suite of physical properties, including atomic velocities and forces, potential and kinetic energies of proteins, and the temperature of the simulation environment, recorded at 1 picosecond intervals throughout the simulations. For benchmarking purposes, we evaluate state-of-the-art methods on the proposed dataset for the task of trajectory prediction. To demonstrate the value of integrating richer physical properties in the study of protein dynamics and related model design, we base our approach on the SE(3) diffusion model and incorporate these physical properties into the trajectory prediction process. Preliminary results indicate that this straightforward extension of the SE(3) model yields improved accuracy, as measured by MAE and RMSD, when the proposed physical properties are taken into consideration. https://fudan-generative-vision.github.io/dynamicPDB/ .

q-bio.BM↗

arXiv2024

Least-Squares Neural Network (LSNN) Method For Linear Advection-Reaction Equation: Non-constant Jumps

The least-squares ReLU neural network (LSNN) method was introduced and studied for solving linear advection-reaction equation with discontinuous solution in \cite{Cai2021linear,cai2023least}. The method is based on an equivalent least-squares formulation and \cite{cai2023least} employs ReLU neural network (NN) functions with $\lceil \log_2(d+1)\rceil+1$-layer representations for approximating solutions. In this paper, we show theoretically that the method is also capable of accurately approximating non-constant jumps along discontinuous interfaces that are not necessarily straight lines. Theoretical results are confirmed through multiple numerical examples with $d=2,3$ and various non-constant jumps and interface shapes, showing that the LSNN method with $\lceil \log_2(d+1)\rceil+1$ layers approximates solutions accurately with degrees of freedom less than that of mesh-based methods and without the common Gibbs phenomena along discontinuous interfaces having non-constant jumps.

math.NA↗

arXiv2024

Qubit-Wise Architecture Search Method for Variational Quantum Circuits

Considering the noise level limit, one crucial aspect for quantum machine learning is to design a high-performing variational quantum circuit architecture with small number of quantum gates. As the classical neural architecture search (NAS), quantum architecture search methods (QAS) employ methods like reinforcement learning, evolutionary algorithms and supernet optimiza-tion to improve the search efficiency. In this paper, we propose a novel qubit-wise architec-ture search (QWAS) method, which progres-sively search one-qubit configuration per stage, and combine with Monte Carlo Tree Search al-gorithm to find good quantum architectures by partitioning the search space into several good and bad subregions. The numerical experimental results indicate that our proposed method can balance the exploration and exploitation of cir-cuit performance and size in some real-world tasks, such as MNIST, Fashion and MOSI. As far as we know, QWAS achieves the state-of-art re-sults of all tasks in the terms of accuracy and circuit size.

quant-ph↗

arXiv2024

Least-Squares Neural Network (LSNN) Method For Linear Advection-Reaction Equation: Discontinuity Interface

We studied the least-squares ReLU neural network (LSNN) method for solving linear advection-reaction equation with discontinuous solution in [Cai, Zhiqiang, Jingshuang Chen, and Min Liu. ``Least-squares ReLU neural network (LSNN) method for linear advection-reaction equation.'' Journal of Computational Physics 443 (2021), 110514]. The method is based on a least-squares formulation and uses a new class of approximating functions: ReLU neural network (NN) functions. A critical and additional component of the LSNN method, differing from other NN-based methods, is the introduction of a properly designed and physics preserved discrete differential operator. In this paper, we study the LSNN method for problems with discontinuity interfaces. First, we show that ReLU NN functions with depth $\lceil \log_2(d+1)\rceil+1$ can approximate any $d$-dimensional step function on a discontinuity interface generated by a vector field as streamlines with any prescribed accuracy. By decomposing the solution into continuous and discontinuous parts, we prove theoretically that discretization error of the LSNN method using ReLU NN functions with depth $\lceil \log_2(d+1)\rceil+1$ is mainly determined by the continuous part of the solution provided that the solution jump is constant. Numerical results for both two- and three-dimensional test problems with various discontinuity interfaces show that the LSNN method with enough layers is accurate and does not exhibit the common Gibbs phenomena along discontinuity interfaces.

math.NA↗

arXiv2023

Evolving Neural Network (ENN) Method for One-Dimensional Scalar Hyperbolic Conservation Laws: I Linear and Quadratic Fluxes

We propose and study the evolving neural network (ENN) method for solving one-dimensional scalar hyperbolic conservation laws with linear and quadratic spatial fluxes. The ENN method first represents the initial data and the inflow boundary data by neural networks. Then, it evolves the neural network representation of the initial data along the temporal direction. The evolution is computed using a combination of characteristic and finite volume methods. For the linear spatial flux, the method is not subject to any time step size, and it is shown theoretically that the error at any time step is bounded by the representation errors of the initial and boundary condition. For the quadratic flux, an error estimate is studied in a companion paper. Finally, numerical results for the linear advection equation and the inviscid Burgers equation are presented to show that the ENN method is more accurate and cost efficient than traditional mesh-based methods.

math.NA↗

arXiv2023

Deep Ritz Method with Adaptive Quadrature for Linear Elasticity

In this paper, we study the deep Ritz method for solving the linear elasticity equation from a numerical analysis perspective. A modified Ritz formulation using the $H^{1/2}(Γ_D)$ norm is introduced and analyzed for linear elasticity equation in order to deal with the (essential) Dirichlet boundary condition. We show that the resulting deep Ritz method provides the best approximation among the set of deep neural network (DNN) functions with respect to the ``energy'' norm. Furthermore, we demonstrate that the total error of the deep Ritz simulation is bounded by the sum of the network approximation error and the numerical integration error, disregarding the algebraic error. To effectively control the numerical integration error, we propose an adaptive quadrature-based numerical integration technique with a residual-based local error indicator. This approach enables efficient approximation of the modified energy functional. Through numerical experiments involving smooth and singular problems, as well as problems with stress concentration, we validate the effectiveness and efficiency of the proposed deep Ritz method with adaptive quadrature.

math.NA↗