arXiv subjects

Jichang Xiao

Publications and source records attributed to Jichang Xiao.

3 recordsLinked to original sources

arXiv2024

Generalization Error Analysis of Deep Backward Dynamic Programming for Solving Nonlinear PDEs

We explore the application of the quasi-Monte Carlo (QMC) method in deep backward dynamic programming (DBDP) (Hure et al. 2020) for numerically solving high-dimensional nonlinear partial differential equations (PDEs). Our study focuses on examining the generalization error as a component of the total error in the DBDP framework, discovering that the rate of convergence for the generalization error is influenced by the choice of sampling methods. Specifically, for a given batch size $m$, the generalization error under QMC methods exhibits a convergence rate of $O(m^{-1+\varepsilon})$, where $\varepsilon$ can be made arbitrarily small. This rate is notably more favorable than that of the traditional Monte Carlo (MC) methods, which is $O(m^{-1/2+\varepsilon})$. Our theoretical analysis shows that the generalization error under QMC methods achieves a higher order of convergence than their MC counterparts. Numerical experiments demonstrate that QMC indeed surpasses MC in delivering solutions that are both more precise and stable.

math.NA↗

arXiv2024

Error analysis for empirical risk minimization over clipped ReLU networks in solving linear Kolmogorov partial differential equations

Deep learning algorithms have been successfully applied to numerically solve linear Kolmogorov partial differential equations~(PDEs). A recent research shows that if the initial functions are bounded, the empirical risk minimization (ERM) over clipped ReLU networks generalizes well for solving the linear Kolmogorov PDE. In this paper, we propose to use a truncation technique to extend the generalization results for polynomially growing initial functions. Specifically, we prove that under an assumption, the sample size required to achieve an generalization error within $\varepsilon$ with a confidence level $\varrho$ grows polynomially in the size of the clipped neural networks and $(\varepsilon^{-1},\varrho^{-1})$, which means that the curse of dimensionality is broken. Moreover, we verify that the required assumptions hold for Black-Scholes PDEs and heat equations which are two important cases of linear Kolmogorov PDEs. For the approximation error, under certain assumptions, we establish approximation results for clipped ReLU neural networks when approximating the solution of Kolmogorov PDEs. Consequently, we establish that the ERM over artificial neural networks indeed overcomes the curse of dimensionality for a larger class of linear Kolmogorov PDEs.

math.NA↗

arXiv2024

Deep Learning Based on Randomized Quasi-Monte Carlo Method for Solving Linear Kolmogorov Partial Differential Equation

Deep learning algorithms have been widely used to solve linear Kolmogorov partial differential equations~(PDEs) in high dimensions, where the loss function is defined as a mathematical expectation. We propose to use the randomized quasi-Monte Carlo (RQMC) method instead of the Monte Carlo (MC) method for computing the loss function. In theory, we decompose the error from empirical risk minimization~(ERM) into the generalization error and the approximation error. Notably, the approximation error is independent of the sampling methods. We prove that the convergence order of the mean generalization error for the RQMC method is $O(n^{-1+ε})$ for arbitrarily small $ε>0$, while for the MC method it is $O(n^{-1/2+ε})$ for arbitrarily small $ε>0$. Consequently, we find that the overall error for the RQMC method is asymptotically smaller than that for the MC method as $n$ increases. Our numerical experiments show that the algorithm based on the RQMC method consistently achieves smaller relative $L^{2}$ error than that based on the MC method.

math.NA↗