arXiv subjects

Zhiqiang Cai

Publications and source records attributed to Zhiqiang Cai.

At least 37 records · Page 2Linked to original sources

arXiv2023

Deep Ritz Method with Adaptive Quadrature for Linear Elasticity

In this paper, we study the deep Ritz method for solving the linear elasticity equation from a numerical analysis perspective. A modified Ritz formulation using the $H^{1/2}(Γ_D)$ norm is introduced and analyzed for linear elasticity equation in order to deal with the (essential) Dirichlet boundary condition. We show that the resulting deep Ritz method provides the best approximation among the set of deep neural network (DNN) functions with respect to the ``energy'' norm. Furthermore, we demonstrate that the total error of the deep Ritz simulation is bounded by the sum of the network approximation error and the numerical integration error, disregarding the algebraic error. To effectively control the numerical integration error, we propose an adaptive quadrature-based numerical integration technique with a residual-based local error indicator. This approach enables efficient approximation of the modified energy functional. Through numerical experiments involving smooth and singular problems, as well as problems with stress concentration, we validate the effectiveness and efficiency of the proposed deep Ritz method with adaptive quadrature.

math.NA↗

arXiv2023

Least-Squares Neural Network (LSNN) Method For Scalar Nonlinear Hyperbolic Conservation Laws: Discrete Divergence Operator

A least-squares neural network (LSNN) method was introduced for solving scalar linear and nonlinear hyperbolic conservation laws (HCLs) in [7, 6]. This method is based on an equivalent least-squares (LS) formulation and uses ReLU neural network as approximating functions, making it ideal for approximating discontinuous functions with unknown interface location. In the design of the LSNN method for HCLs, the numerical approximation of differential operators is a critical factor, and standard numerical or automatic differentiation along coordinate directions can often lead to a failed NN-based method. To overcome this challenge, this paper rewrites HCLs in their divergence form of space and time and introduces a new discrete divergence operator. As a result, the proposed LSNN method is free of penalization of artificial viscosity. Theoretically, the accuracy of the discrete divergence operator is estimated even for discontinuous solutions. Numerically, the LSNN method with the new discrete divergence operator was tested for several benchmark problems with both convex and non-convex fluxes, and was able to compute the correct physical solution for problems with rarefaction, shock or compound waves. The method is capable of capturing the shock of the underlying problem without oscillation or smearing, even without any penalization of the entropy condition, total variation, and/or artificial viscosity.

math.NA↗

arXiv2023

Adaptive Least-Squares Methods for Convection-Dominated Diffusion-Reaction Problems

This paper studies adaptive least-squares finite element methods for convection-dominated diffusion-reaction problems. The least-squares methods are based on the first-order system of the primal and dual variables with various ways of imposing outflow boundary conditions. The coercivity of the homogeneous least-squares functionals are established, and the a priori error estimates of the least-squares methods are obtained in a norm that incorporates the streamline derivative. All methods have the same convergence rate provided that meshes in the layer regions are fine enough. To increase computational accuracy and reduce computational cost, adaptive least-squares methods are implemented and numerical results are presented for some test problems.

math.NA↗

arXiv2022

Residual-Quantile Adjustment for Adaptive Training of Physics-informed Neural Network

Adaptive training methods for Physics-informed neural network (PINN) require dedicated constructions of the distribution of weights assigned at each training sample. To efficiently seek such an optimal weight distribution is not a simple task and most existing methods choose the adaptive weights based on approximating the full distribution or the maximum of residuals. In this paper, we show that the bottleneck in the adaptive choice of samples for training efficiency is the behavior of the tail distribution of the numerical residual. Thus, we propose the Residual-Quantile Adjustment (RQA) method for a better weight choice for each training sample. After initially setting the weights proportional to the $p$-th power of the residual, our RQA method reassign all weights above $q$-quantile ($90\%$ for example) to the median value, so that the weight follows a quantile-adjusted distribution derived from the residuals. This iterative reweighting technique, on the other hand, is also very easy to implement. Experiment results show that the proposed method can outperform several adaptive methods on various partial differential equation (PDE) problems.

cs.LG↗

arXiv2022

Self-adaptive deep neural network: Numerical approximation to functions and PDEs

Designing an optimal deep neural network for a given task is important and challenging in many machine learning applications. To address this issue, we introduce a self-adaptive algorithm: the adaptive network enhancement (ANE) method, written as loops of the form train, estimate and enhance. Starting with a small two-layer neural network (NN), the step train is to solve the optimization problem at the current NN; the step estimate is to compute a posteriori estimator/indicators using the solution at the current NN; the step enhance is to add new neurons to the current NN. Novel network enhancement strategies based on the computed estimator/indicators are developed in this paper to determine how many new neurons and when a new layer should be added to the current NN. The ANE method provides a natural process for obtaining a good initialization in training the current NN; in addition, we introduce an advanced procedure on how to initialize newly added neurons for a better approximation. We demonstrate that the ANE method can automatically design a nearly minimal NN for learning functions exhibiting sharp transitional layers as well as discontinuous solutions of hyperbolic partial differential equations.

math.NA↗

arXiv2022

Least-Squares ReLU Neural Network (LSNN) Method For Scalar Nonlinear Hyperbolic Conservation Law

We introduced the least-squares ReLU neural network (LSNN) method for solving the linear advection-reaction problem with discontinuous solution and showed that the method outperforms mesh-based numerical methods in terms of the number of degrees of freedom. This paper studies the LSNN method for scalar nonlinear hyperbolic conservation law. The method is a discretization of an equivalent least-squares (LS) formulation in the set of neural network functions with the ReLU activation function. Evaluation of the LS functional is done by using numerical integration and conservative finite volume scheme. Numerical results of some test problems show that the method is capable of approximating the discontinuous interface of the underlying problem automatically through the free breaking lines of the ReLU neural network. Moreover, the method does not exhibit the common Gibbs phenomena along the discontinuous interface.

math.NA↗

arXiv2022

Learn Quasi-stationary Distributions of Finite State Markov Chain

We propose a reinforcement learning (RL) approach to compute the expression of quasi-stationary distribution. Based on the fixed-point formulation of quasi-stationary distribution, we minimize the KL-divergence of two Markovian path distributions induced by the candidate distribution and the true target distribution. To solve this challenging minimization problem by gradient descent, we apply the reinforcement learning technique by introducing the reward and value functions. We derive the corresponding policy gradient theorem and design an actor-critic algorithm to learn the optimal solution and the value function. The numerical examples of finite state Markov chain are tested to demonstrate the new method.

cs.LG↗

arXiv2022

Adaptive Two-Layer ReLU Neural Network: I. Best Least-squares Approximation

In this paper, we introduce adaptive neuron enhancement (ANE) method for the best least-squares approximation using two-layer ReLU neural networks (NNs). For a given function f(x), the ANE method generates a two-layer ReLU NN and a numerical integration mesh such that the approximation accuracy is within the prescribed tolerance. The ANE method provides a natural process for obtaining a good initialization which is crucial for training nonlinear optimization problems. Numerical results of the ANE method are presented for functions of two variables exhibiting either intersecting interface singularities or sharp interior layers.

math.NA↗

arXiv2021

Hybrid A Posteriori Error Estimators for Conforming Finite Element Approximations to Stationary Convection-Diffusion-Reaction equations

We consider the a posteriori error estimation for convection-diffusion-reaction equations in both diffusion-dominated and convection/reaction-dominated regimes. We present an explicit hybrid estimator, which, in each regime, is proved to be reliable and efficient with constants independent of the parameters in the underlying problem. For convection-dominated problems, the norm introduced by Verf{ü}rth \cite{verf2005confusion} is used to measure the approximation error. Various numerical experiments are performed to (1) demonstrate the robustness of the hybrid estimator; (2) show that the hybrid estimator is more accurate than the explicit residual estimator and is less sensitive to the size of reaction, even though both of them are robust.

math.NA↗

arXiv2021

Adaptive Two-Layer ReLU Neural Network: II. Ritz Approximation to Elliptic PDEs

In this paper, we study adaptive neuron enhancement (ANE) method for solving self-adjoint second-order elliptic partial differential equations (PDEs). The ANE method is a self-adaptive method generating a two-layer spline NN and a numerical integration mesh such that the approximation accuracy is within the prescribed tolerance. Moreover, the ANE method provides a natural process for obtaining a good initialization which is crucial for training nonlinear optimization problem. The underlying PDE is discretized by the Ritz method using a two-layer spline neural network based on either the primal or dual formulations that minimize the respective energy or complimentary functionals. Essential boundary conditions are imposed weakly through the functionals with proper norms. It is proved that the Ritz approximation is the best approximation in the energy norm; moreover, effect of numerical integration for the Ritz approximation is analyzed as well. Two estimators for adaptive neuron enhancement method are introduced, one is the so-called recovery estimator and the other is the least-squares estimator. Finally, numerical results for diffusion problems with either corner or intersecting interface singularities are presented.

math.NA↗

arXiv2021

Least-Squares ReLU Neural Network (LSNN) Method For Linear Advection-Reaction Equation

This paper studies least-squares ReLU neural network method for solving the linear advection-reaction problem with discontinuous solution. The method is a discretization of an equivalent least-squares formulation in the set of neural network functions with the ReLU activation function. The method is capable of approximating the discontinuous interface of the underlying problem automatically through the free hyper-planes of the ReLU neural network and, hence, outperforms mesh-based numerical methods in terms of the number of degrees of freedom. Numerical results of some benchmark test problems show that the method can not only approximate the solution with the least number of parameters, but also avoid the common Gibbs phenomena along the discontinuous interface. Moreover, a three-layer ReLU neural network is necessary and sufficient in order to well approximate a discontinuous solution with an interface in $\mathbb{R}^2$ that is not a straight line.

math.NA↗

arXiv2020

Deep least-squares methods: an unsupervised learning-based numerical method for solving elliptic PDEs

This paper studies an unsupervised deep learning-based numerical approach for solving partial differential equations (PDEs). The approach makes use of the deep neural network to approximate solutions of PDEs through the compositional construction and employs least-squares functionals as loss functions to determine parameters of the deep neural network. There are various least-squares functionals for a partial differential equation. This paper focuses on the so-called first-order system least-squares (FOSLS) functional studied in [3], which is based on a first-order system of scalar second-order elliptic PDEs. Numerical results for second-order elliptic PDEs in one dimension are presented.

cs.LG↗

arXiv2020

Generalized Prager-Synge Inequality and Equilibrated Error Estimators for Discontinuous Elements

The well-known Prager-Synge identity is valid in $H^1(Ω)$ and serves as a foundation for developing equilibrated a posteriori error estimators for continuous elements. In this paper, we introduce a new inequality, that may be regarded as a generalization of the Prager-Synge identity, to be valid for piecewise $H^1(Ω)$ functions for diffusion problems. The inequality is proved to be identity in two dimensions. For nonconforming finite element approximation of arbitrary odd order, we propose a fully explicit approach that recovers an equilibrated flux in $H(div; Ω)$ through a local element-wise scheme and that recovers a gradient in $H(curl;Ω)$ through a simple averaging technique over edges. The resulting error estimator is then proved to be globally reliable and locally efficient. Moreover, the reliability and efficiency constants are independent of the jump of the diffusion coefficient regardless of its distribution.

math.NA↗

arXiv2019

A Discretization-Accurate Stopping Criterion for Iterative Solvers for Finite Element Approximation

This paper introduces a discretization-accurate stopping criterion of symmetric iterative methods for solving systems of algebraic equations resulting from the finite element approximation. The stopping criterion consists of the evaluations of the discretization and the algebraic error estimators, that are based on the respective duality error estimator and the difference of two consecutive iterates. Iterations are terminated when the algebraic estimator is of the same magnitude as the discretization estimator. Numerical results for multigrid $V(1,1)$-cycle and symmetric Gauss-Seidel iterative methods are presented for the linear finite element approximation to the Poisson equations. A large reduction in computational cost is observed compared to the standard residual-based stopping criterion.

math.NA↗

arXiv2019

A finite element method for Dirichlet boundary control of elliptic partial differential equations

This paper introduces a new variational formulation for Dirichlet boundary control problem of elliptic partial differential equations, based on observations that the state and adjoint state are related through the control on the boundary of the domain, and that such a relation may be imposed in the variational formulation of the adjoint state. Well-posedness (unique solvability and stability) of the variational problem is established in the $H^{1}(Ω)\times H_{0}^{1}(Ω)$ space for the respective state and adjoint state. A finite element method based on this formulation is analyzed. It is shown that the conforming $k-$th order finite element approximations to the state and the adjoint state, in the respective $L^{2}$ and $H^{1}$ norms converge at the rate of order $k-1/2$ on quasi-uniform mesh for conforming element of order $k$. Numerical examples are presented to validate the theory.

math.NA↗

arXiv2016

Robust A Posteriori Error Estimation for Finite Element Approximation to H(curl) Problem

In this paper, we introduce a novel a posteriori error estimator for the conforming finite element approximation to the H(curl) problem with inhomogeneous media and with the right-hand side only in L^2. The estimator is of the recovery type. Independent with the current approximation to the primary variable (the electric field), an auxiliary variable (the magnetizing field) is recovered in parallel by solving a similar H(curl) problem. An alternate way of recovery is presented as well by localizing the error flux. The estimator is then defined as the sum of the modified element residual and the residual of the constitutive equation defining the auxiliary variable. It is proved that the estimator is approximately equal to the true error in the energy norm without the quasi-monotonicity assumption. Finally, we present numerical results for two H(curl) interface problems.

math.NA↗

arXiv2016

Improved ZZ A Posteriori Error Estimators for Diffusion Problems: Conforming Linear Elements

In \cite{CaZh:09}, we introduced and analyzed an improved Zienkiewicz-Zhu (ZZ) estimator for the conforming linear finite element approximation to elliptic interface problems. The estimator is based on the piecewise "constant" flux recovery in the $H(div;Ω)$ conforming finite element space. This paper extends the results of \cite{CaZh:09} to diffusion problems with full diffusion tensor and to the flux recovery both in piecewise constant and piecewise linear $H(div)$ space.

math.NA↗

arXiv2016

Finite Element Methods for Interface Problems: Robust Residual-Based A Posteriori Error Estimates

For elliptic interface problems, this paper studies residual-based a posteriori error estimations for various finite element approximations. For the conforming and the Raviart-Thomas mixed elements in two-dimension and for the Crouzeix-Raviart nonconforming and the discontinuous Galerkin elements in both two- and three-dimensions, the global reliability bounds are established with constants independent of the jump of the diffusion coefficient. Moreover, we obtain these estimates with no assumption on the distribution of the diffusion coefficient.

math.NA↗