arXiv subjects
Publications and source records attributed to Jinchao Xu.
For the Hodge--Laplace equation in finite element exterior calculus, we introduce several families of discontinuous Galerkin methods in the extended Galerkin framework. For contractible domains, this framework utilizes seven fields and provides a unifying inf-sup analysis with respect to all discretization and penalty parameters. It is shown that the proposed methods can be hybridized as a reduced two-field formulation.
We prove some new results concerning the approximation rate of neural networks with general activation functions. Our first result concerns the rate of approximation of a two layer neural network with a polynomially-decaying non-sigmoidal activation function. We extend the dimension independent approximation rates previously obtained to this new class of activation functions. Our second result gives a weaker, but still dimension independent, approximation rate for a larger class of activation functions, removing the polynomial decay assumption. This result applies to any bounded, integrable activation function. Finally, we show that a stratified sampling approach can be used to improve the approximation rate for polynomially decaying activation functions under mild additional assumptions.
This paper studies numerical solutions for parameterized partial differential equations (P-PDEs) with deep learning (DL). P-PDEs arise in many important application areas and the computational cost using traditional numerical schemes can be exorbitant, especially when the parameters fall into a particular range and the underlying PDE is required to be solved with high accuracy. Recently, solving PDEs with DL has become an emerging field. Existing works demonstrate great potentials of the DL based approach in speeding up numerical solutions of PDEs. However, there is still limited research on the DL approach for P-PDEs. If we directly apply existing supervised learning models to P-PDEs, the models need to be constantly fine-tuned or retrained when the parameters change. This drastically limits the applicability and utility of these models in practice. To resolve this issue, we propose a meta-learning-based method that can efficiently solve P-PDEs with a wide range of parameters without retraining. Our key observation is to regard training a solver for the P-PDE with a given set of parameters as a learning task. Then, training a solver for the P-PDEs with varied parameters can be viewed as a multi-task learning problem, to which meta-learning is one of the most effective approaches. This new perspective can be applied to many existing PDE solvers. As an example, we adopt the Multigrid Network (MgNet) as the base solver. To achieve multi-task learning, we introduce a new hypernetwork, called Meta-NN, in MgNet and refer to the entire network as the Meta-MgNet. Meta-NN takes the differential operators and the right-hand-side of the underlying P-PDEs as inputs and generates appropriate smoothers for MgNet which can significantly affect the convergent speed. Finally, extensive numerical experiments demonstrate that Meta-MgNet is more efficient in solving P-PDEs than the MG methods and MgNet.
We study a family of $H^m$-conforming piecewise polynomials based on artificial neural network, named as the finite neuron method (FNM), for numerical solution of $2m$-th order partial differential equations in $\mathbb{R}^d$ for any $m,d \geq 1$ and then provide convergence analysis for this method. Given a general domain $Ω\subset\mathbb R^d$ and a partition $\mathcal T_h$ of $Ω$, it is still an open problem in general how to construct conforming finite element subspace of $H^m(Ω)$ that have adequate approximation properties. By using techniques from artificial neural networks, we construct a family of $H^m$-conforming set of functions consisting of piecewise polynomials of degree $k$ for any $k\ge m$ and we further obtain the error estimate when they are applied to solve elliptic boundary value problem of any order in any dimension. For example, the following error estimates between the exact solution $u$ and finite neuron approximation $u_N$ are obtained. $$ \|u-u_N\|_{H^m(Ω)}=\mathcal O(N^{-{1\over 2}-{1\over d}}). $$ Discussions will also be given on the difference and relationship between the finite neuron method and finite element methods (FEM). For example, for finite neuron method, the underlying finite element grids are not given a priori and the discrete solution can only be obtained by solving a non-linear and non-convex optimization problem. Despite of many desirable theoretical properties of the finite neuron method analyzed in the paper, its practical value is a subject of further investigation since the aforementioned underlying non-linear and non-convex optimization problem can be expensive and challenging to solve. For completeness and also convenience to readers, some basic known results and their proofs are also included in this manuscript.
In this paper, we propose and analyze an abstract stabilized mixed finite element framework that can be applied to nonlinear incompressible elasticity problems. In the abstract stabilized framework, we prove that any mixed finite element method that satisfies the discrete inf-sup condition can be modified so that it is stable and optimal convergent as long as the mixed continuous problem is stable. Furthermore, we apply the abstract stabilized framework to nonlinear incompressible elasticity problems and present numerical experiments to verify the theoretical results.
In this paper, we propose a constrained linear data-feature mapping model as an interpretable mathematical model for image classification using convolutional neural network (CNN) such as the ResNet. From this viewpoint, we establish the detailed connections in a technical level between the traditional iterative schemes for constrained linear system and the architecture for the basic blocks of ResNet. Under these connections, we propose some natural modifications of ResNet type models which will have less parameters but still maintain almost the same accuracy as these corresponding original models. Some numerical experiments are shown to demonstrate the validity of this constrained learning data-feature mapping assumption.
Compressed Sensing using $\ell_1$ regularization is among the most powerful and popular sparsification technique in many applications, but why has it not been used to obtain sparse deep learning model such as convolutional neural network (CNN)? This paper is aimed to provide an answer to this question and to show how to make it work. We first demonstrate that the commonly used stochastic gradient decent (SGD) and variants training algorithm is not an appropriate match with $\ell_1$ regularization and then replace it with a different training algorithm based on a regularized dual averaging (RDA) method. RDA was originally designed specifically for convex problem, but with new theoretical insight and algorithmic modifications (using proper initialization and adaptivity), we have made it an effective match with $\ell_1$ regularization to achieve a state-of-the-art sparsity for CNN compared to other weight pruning methods without compromising accuracy (achieving 95\% sparsity for ResNet18 on CIFAR-10, for example).
In this paper we study the linear systems arising from discretized poroelasticity problems. We formulate one block preconditioner for the two-filed Biot model and several preconditioners for the classical three-filed Biot model under the unified relationship framework between well-posedness and preconditioners. By the unified theory, we show all the considered preconditioners are uniformly optimal with respect to material and discretization parameters. Numerical tests demonstrate the robustness of these preconditioners.
A general analysis framework is presented in this paper for many different types of finite element methods (including various discontinuous Galerkin methods). For second order elliptic equation, this framework employs $4$ different discretization variables, $u_h, \bm{p}_h, \check u_h$ and $\check p_h$, where $u_h$ and $\bm{p}_h$ are for approximation of $u$ and $\bm{p}=-α\nabla u$ inside each element, and $ \check u_h$ and $\check p_h$ are for approximation of residual of $u$ and $\bm{p} \cdot \bm{n}$ on the boundary of each element. The resulting 4-field discretization is proved to satisfy inf-sup conditions that are uniform with respect to all discretization and penalization parameters. As a result, most existing finite element and discontinuous Galerkin methods can be analyzed using this general framework by making appropriate choices of discretization spaces and penalization parameters.
In this paper we give an estimate on the asymptotic behavior of eigenvalues of discretized elliptic boundary values problems. We first prove a simple min-max principle for selfadjoint operators on a Hilbert space. Then we show two sided bounds on the $k$-th eigenvalue of the discrete Laplacian by the $k$-th eigenvalue of the continuous Laplacian operator under the assumption that the finite element mesh is quasi-uniform. Combining this result with the well-known Weyl's law, we show that the $k$-th eigenvalue of the discretized isotropic elliptic operators, spectrally equivalent to the discretized Laplacian, is $\mathcal O\left(k^{2/d}\right)$. Finally, we show how these results can be used to obtain an error estimate for finite element approximations of elliptic eigenvalue problems.
Pulse feeling, representing the tactile arterial palpation of the heartbeat, has been widely used in traditional Chinese medicine (TCM) to diagnose various diseases. The quantitative relationship between the pulse wave and health conditions however has not been investigated in modern medicine. In this paper, we explored the correlation between pulse pressure wave (PPW), rather than the pulse key features in TCM, and pregnancy by using deep learning technology. This computational approach shows that the accuracy of pregnancy detection by the PPW is 84% with an AUC of 91%. Our study is a proof of concept of pulse diagnosis and will also motivate further sophisticated investigations on pulse waves.
This paper is devoted to the construction and analysis of the finite element approximations for the $H(D)$ convection-diffusion problems, where $D$ can be chosen as ${\rm grad}$, ${\rm curl}$ or ${\rm div}$ in 3D case. An essential feature of these constructions is to properly average the PDE coefficients on the sub-simplexes. The schemes are of the class of exponential fitting methods that result in special upwind schemes when the diffusion coefficient approaches to zero. Their well-posedness are established for sufficiently small mesh size assuming that the convection-diffusion problems are uniquely solvable. Convergence of first order is derived under minimal smoothness of the solution. Some numerical examples are given to demonstrate the robustness and effectiveness for general convection-diffusion problems.
We develop a unified model, known as MgNet, that simultaneously recovers some convolutional neural networks (CNN) for image classification and multigrid (MG) methods for solving discretized partial differential equations (PDEs). This model is based on close connections that we have observed and uncovered between the CNN and MG methodologies. For example, pooling operation and feature extraction in CNN correspond directly to restriction operation and iterative smoothers in MG, respectively. As the solution space is often the dual of the data space in PDEs, the analogous concept of feature space and data space (which are dual to each other) is introduced in CNN. With such connections and new concept in the unified model, the function of various convolution operations and pooling used in CNN can be better understood. As a result, modified CNN models (with fewer weights and hyper parameters) are developed that exhibit competitive and sometimes better performance in comparison with existing CNN models when applied to both CIFAR-10 and CIFAR-100 data sets.
In this paper, we study a mixed discontinuous Galerkin (MDG) method to solve linear elasticity problem with arbitrary order discontinuous finite element spaces in $d$-dimension ($d=2,3$). This method uses polynomials of degree $k+1$ for the stress and of degree $k$ for the displacement ($k\geq 0$). The mixed DG scheme is proved to be well-posed under proper norms. Specifically, we prove that, for any $k \geq 0$, the $H({\rm div})$-like error estimate for the stress and $L^2$ error estimate for the displacement are optimal. We further establish the optimal $L^2$ error estimate for the stress provided that the $\mathcal{P}_{k+2}-\mathcal{P}_{k+1}^{-1}$ Stokes pair is stable and $k \geq d$. We also provide numerical results of MDG showing that the orders of convergence are actually sharp.
In this paper, we consider the iterative method of subspace corrections with random ordering. We prove identities for the expected convergence rate, which can provide sharp estimates for the error reduction per iteration. We also study the fault-tolerant feature of the randomized successive subspace correction method by simply rejecting all the corrections when error occurs and show that the results iterative method converges with probability one. Moreover, we also provide sharp estimates on the expected convergence rate for the fault-tolerant, randomized, subspace correction method.
In this paper, we provide a number of new estimates on the stability and convergence of both hybrid discontinuous Galerkin (HDG) and weak Galerkin (WG) methods. By using the standard Brezzi theory on mixed methods, we carefully define appropriate norms for the various discretization variables and then establish that the stability and error estimates hold uniformly with respect to stabilization and discretization parameters. As a result, by taking appropriate limit of the stabilization parameters, we show that the HDG method converges to a primal conforming method and the WG method converge to a mixed conforming method.
In this paper, we investigate the relationship between deep neural networks (DNN) with rectified linear unit (ReLU) function as the activation function and continuous piecewise linear (CPWL) functions, especially CPWL functions from the simplicial linear finite element method (FEM). We first consider the special case of FEM. By exploring the DNN representation of its nodal basis functions, we present a ReLU DNN representation of CPWL in FEM. We theoretically establish that at least $2$ hidden layers are needed in a ReLU DNN to represent any linear finite element functions in $Ω\subseteq \mathbb{R}^d$ when $d\ge2$. Consequently, for $d=2,3$ which are often encountered in scientific and engineering computing, the minimal number of two hidden layers are necessary and sufficient for any CPWL function to be represented by a ReLU DNN. Then we include a detailed account on how a general CPWL in $\mathbb R^d$ can be represented by a ReLU DNN with at most $\lceil\log_2(d+1)\rceil$ hidden layers and we also give an estimation of the number of neurons in DNN that are needed in such a representation. Furthermore, using the relationship between DNN and FEM, we theoretically argue that a special class of DNN models with low bit-width are still expected to have an adequate representation power in applications. Finally, as a proof of concept, we present some numerical results for using ReLU DNNs to solve a two point boundary problem to demonstrate the potential of applying DNN for numerical solution of partial differential equations.
We prove generalized Gaffney inequalities and the discrete compactness for finite element differential forms on $s$-regular domains, including general Lipschitz domains. In computational electromagnetism, special cases of these results have been established for edge elements with weakly imposed divergence-free conditions and used in the analysis of nonlinear and eigenvalue problems. In this paper, we generalize these results to discrete differential forms, not necessarily with strongly or weakly imposed constraints. The analysis relies on a new Hodge mapping and its approximation property. As an application, we show $L^{p}$ estimates for several finite element approximations of the scalar and vector Laplacian problems.