arXiv subjects

Jianfeng Lu

Publications and source records attributed to Jianfeng Lu.

At least 163 records · Page 9Linked to original sources

arXiv2021

A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equations

This paper concerns the a priori generalization analysis of the Deep Ritz Method (DRM) [W. E and B. Yu, 2017], a popular neural-network-based method for solving high dimensional partial differential equations. We derive the generalization error bounds of two-layer neural networks in the framework of the DRM for solving two prototype elliptic PDEs: Poisson equation and static Schrödinger equation on the $d$-dimensional unit hypercube. Specifically, we prove that the convergence rates of generalization errors are independent of the dimension $d$, under the a priori assumption that the exact solutions of the PDEs lie in a suitable low-complexity space called spectral Barron space. Moreover, we give sufficient conditions on the forcing term and the potential function which guarantee that the solutions are spectral Barron functions. We achieve this by developing a new solution theory for the PDEs on the spectral Barron space, which can be viewed as an analog of the classical Sobolev regularity theory for PDEs.

math.NA↗

arXiv2021

Incorporating Orientations into End-to-end Driving Model for Steering Control

In this paper, we present a novel end-to-end deep neural network model for autonomous driving that takes monocular image sequence as input, and directly generates the steering control angle. Firstly, we model the end-to-end driving problem as a local path planning process. Inspired by the environmental representation in the classical planning algorithms(i.e. the beam curvature method), pixel-wise orientations are fed into the network to learn direction-aware features. Next, to handle the imbalanced distribution of steering values in training datasets, we propose an improvement on a cost-sensitive loss function named SteeringLoss2. Besides, we also present a new end-to-end driving dataset, which provides corresponding LiDAR and image sequences, as well as standard driving behaviors. Our dataset includes multiple driving scenarios, such as urban, country, and off-road. Numerous experiments are conducted on both public available LiVi-Set and our own dataset, and the results show that the model using our proposed methods can predict steering angle accurately.

cs.RO↗

arXiv2021

On explicit $L^2$-convergence rate estimate for piecewise deterministic Markov processes in MCMC algorithms

We establish $L^2$-exponential convergence rate for three popular piecewise deterministic Markov processes for sampling: the randomized Hamiltonian Monte Carlo method, the zigzag process, and the bouncy particle sampler. Our analysis is based on a variational framework for hypocoercivity, which combines a Poincaré-type inequality in time-augmented state space and a standard $L^2$ energy estimate. Our analysis provides explicit convergence rate estimates, which are more quantitative than existing results.

math.PR↗

arXiv2021

Symmetry Breaking in Density Functional Theory due to Dirac Exchange for a Hydrogen Molecule

We study symmetry breaking in the mean field solutions to the 2 electron hydrogen molecule within Kohn Sham (KS) local spin density function theory with Dirac exchange (the XLDA model). This simplified model shows behavior related to that of the (KS) spin density functional theory (SDFT) predictions in condensed and molecular systems. The Kohn Sham solutions to the constrained SDFT variation problem undergo spontaneous symmetry breaking as the relative strength of the non-convex exchange term increases. This results in the change of the molecular ground state from a paramagnetic state to an antiferromagnetic ground states and a stationary symmetric delocalized 1st excited state. We further characterize the limiting behavior of the minimizer when the strength of the exchange term goes to infinity. This leads to further bifurcations and highly localized states with varying character. The stability of the various solution classes is demonstrated by Hessian analysis. Finite element numerical results provide support for the formal conjectures.

math-ph↗

arXiv2021

A Grid-free Approach for Simulating Sweep and Cyclic Voltammetry

We present a new computational approach to simulate linear sweep and cyclic voltammetry experiments that does not require a discretized grid in space to quantify diffusion. By using a Green's function solution coupled to a standard implicit ordinary differential equation solver, we are able to simulate current and redox species concentrations using only a small grid in time. As a result, where benchmarking is possible, we find that the current method is faster (and quantitatively identical) to established techniques. The present algorithm should help open the door to studying adsorption effects in inner sphere electrochemistry.

physics.chem-ph↗

arXiv2021

Numerical methods for stochastic differential equations based on Gaussian mixture

We develop in this work a numerical method for stochastic differential equations (SDEs) with weak second order accuracy based on Gaussian mixture. Unlike the conventional higher order schemes for SDEs based on Itô-Taylor expansion and iterated Itô integrals, the proposed scheme approximates the probability measure $μ(X^{n+1}|X^n=x_n)$ by a mixture of Gaussians. The solution at next time step $X^{n+1}$ is then drawn from the Gaussian mixture with complexity linear in the dimension $d$. This provides a new general strategy to construct efficient high weak order numerical schemes for SDEs.

math.NA↗

arXiv2021

Neural Collapse with Cross-Entropy Loss

We consider the variational problem of cross-entropy loss with $n$ feature vectors on a unit hypersphere in $\mathbb{R}^d$. We prove that when $d \geq n - 1$, the global minimum is given by the simplex equiangular tight frame, which justifies the neural collapse behavior. We also prove that as $n \rightarrow \infty$ with fixed $d$, the minimizing points will distribute uniformly on the hypersphere and show a connection with the frame potential of Benedetto & Fickus.

cs.LG↗

arXiv2021

Complexity of randomized algorithms for underdamped Langevin dynamics

We establish an information complexity lower bound of randomized algorithms for simulating underdamped Langevin dynamics. More specifically, we prove that the worst $L^2$ strong error is of order $Ω(\sqrt{d}\, N^{-3/2})$, for solving a family of $d$-dimensional underdamped Langevin dynamics, by any randomized algorithm with only $N$ queries to $\nabla U$, the driving Brownian motion and its weighted integration, respectively. The lower bound we establish matches the upper bound for the randomized midpoint method recently proposed by Shen and Lee [NIPS 2019], in terms of both parameters $N$ and $d$.

math.NA↗

arXiv2020

Analysis of a fourth order exponential PDE arising from a crystal surface jump process with Metropolis-type transition rates

We analytically and numerically study a fourth order PDE modeling rough crystal surface diffusion on the macroscopic level. We discuss existence of solutions globally in time and long time dynamics for the PDE model. The PDE, originally derived by the second author, is the continuum limit of a microscopic model of the surface dynamics, given by a Markov jump process with Metropolis type transition rates. We outline the convergence argument, which depends on a simplifying assumption on the local equilibrium measure that is valid in the high temperature regime. We provide numerical evidence for the convergence of the microscopic model to the PDE in this regime.

math.AP↗

arXiv2020

A Universal Approximation Theorem of Deep Neural Networks for Expressing Probability Distributions

This paper studies the universal approximation property of deep neural networks for representing probability distributions. Given a target distribution $π$ and a source distribution $p_z$ both defined on $\mathbb{R}^d$, we prove under some assumptions that there exists a deep neural network $g:\mathbb{R}^d\rightarrow \mathbb{R}$ with ReLU activation such that the push-forward measure $(\nabla g)_\# p_z$ of $p_z$ under the map $\nabla g$ is arbitrarily close to the target measure $π$. The closeness are measured by three classes of integral probability metrics between probability distributions: $1$-Wasserstein distance, maximum mean distance (MMD) and kernelized Stein discrepancy (KSD). We prove upper bounds for the size (width and depth) of the deep neural network in terms of the dimension $d$ and the approximation error $\varepsilon$ with respect to the three discrepancies. In particular, the size of neural network can grow exponentially in $d$ when $1$-Wasserstein distance is used as the discrepancy, whereas for both MMD and KSD the size of neural network only depends on $d$ at most polynomially. Our proof relies on convergence estimates of empirical measures under aforementioned discrepancies and semi-discrete optimal transport.

cs.LG↗

arXiv2020

MPNet: Masked and Permuted Pre-training for Language Understanding

BERT adopts masked language modeling (MLM) for pre-training and is one of the most successful pre-training models. Since BERT neglects dependency among predicted tokens, XLNet introduces permuted language modeling (PLM) for pre-training to address this problem. However, XLNet does not leverage the full position information of a sentence and thus suffers from position discrepancy between pre-training and fine-tuning. In this paper, we propose MPNet, a novel pre-training method that inherits the advantages of BERT and XLNet and avoids their limitations. MPNet leverages the dependency among predicted tokens through permuted language modeling (vs. MLM in BERT), and takes auxiliary position information as input to make the model see a full sentence and thus reducing the position discrepancy (vs. PLM in XLNet). We pre-train MPNet on a large-scale dataset (over 160GB text corpora) and fine-tune on a variety of down-streaming tasks (GLUE, SQuAD, etc). Experimental results show that MPNet outperforms MLM and PLM by a large margin, and achieves better results on these tasks compared with previous state-of-the-art pre-trained methods (e.g., BERT, XLNet, RoBERTa) under the same model setting. The code and the pre-trained models are available at: https://github.com/microsoft/MPNet.

cs.CL↗

arXiv2020

Fast localization of eigenfunctions via smoothed potentials

We study the problem of predicting highly localized low-lying eigenfunctions $(-Δ+V) ϕ= λϕ$ in bounded domains $Ω\subset \mathbb{R}^d$ for rapidly varying potentials $V$. Filoche & Mayboroda introduced the function $1/u$, where $(-Δ+ V)u=1$, as a suitable regularization of $V$ from whose minima one can predict the location of eigenfunctions with high accuracy. We proposed a fast method that produces a landscapes that is exceedingly similar, can be used for the same purposes and can be computed very efficiently: the computation time on an $n \times n$ grid, for example, is merely $\mathcal{O}(n^2 \log{n})$, the cost of two FFTs.

math.NA↗

arXiv2020

Random Coordinate Underdamped Langevin Monte Carlo

The Underdamped Langevin Monte Carlo (ULMC) is a popular Markov chain Monte Carlo sampling method. It requires the computation of the full gradient of the log-density at each iteration, an expensive operation if the dimension of the problem is high. We propose a sampling method called Random Coordinate ULMC (RC-ULMC), which selects a single coordinate at each iteration to be updated and leaves the other coordinates untouched. We investigate the computational complexity of RC-ULMC and compare it with the classical ULMC for strongly log-concave probability distributions. We show that RC-ULMC is always cheaper than the classical ULMC, with a significant cost reduction when the problem is highly skewed and high dimensional. Our complexity bound for RC-ULMC is also tight in terms of dimension dependence.

stat.ML↗

arXiv2020

Global optimality of softmax policy gradient with single hidden layer neural networks in the mean-field regime

We study the problem of policy optimization for infinite-horizon discounted Markov Decision Processes with softmax policy and nonlinear function approximation trained with policy gradient algorithms. We concentrate on the training dynamics in the mean-field regime, modeling e.g., the behavior of wide single hidden layer neural networks, when exploration is encouraged through entropy regularization. The dynamics of these models is established as a Wasserstein gradient flow of distributions in parameter space. We further prove global optimality of the fixed points of this dynamics under mild conditions on their initialization.

cs.LG↗

arXiv2020

Variational training of neural network approximations of solution maps for physical models

A novel solve-training framework is proposed to train neural network in representing low dimensional solution maps of physical models. Solve-training framework uses the neural network as the ansatz of the solution map and train the network variationally via loss functions from the underlying physical models. Solve-training framework avoids expensive data preparation in the traditional supervised training procedure, which prepares labels for input data, and still achieves effective representation of the solution map adapted to the input data distribution. The efficiency of solve-training framework is demonstrated through obtaining solutions maps for linear and nonlinear elliptic equations, and maps from potentials to ground states of linear and nonlinear Schrödinger equations.

math.NA↗

arXiv2020

Random Coordinate Langevin Monte Carlo

Langevin Monte Carlo (LMC) is a popular Markov chain Monte Carlo sampling method. One drawback is that it requires the computation of the full gradient at each iteration, an expensive operation if the dimension of the problem is high. We propose a new sampling method: Random Coordinate LMC (RC-LMC). At each iteration, a single coordinate is randomly selected to be updated by a multiple of the partial derivative along this direction plus noise, and all other coordinates remain untouched. We investigate the total complexity of RC-LMC and compare it with the classical LMC for log-concave probability distributions. When the gradient of the log-density is Lipschitz, RC-LMC is less expensive than the classical LMC if the log-density is highly skewed for high dimensional problems, and when both the gradient and the Hessian of the log-density are Lipschitz, RC-LMC is always cheaper than the classical LMC, by a factor proportional to the square root of the problem dimension. In the latter case, our estimate of complexity is sharp with respect to the dimension.

stat.ML↗

arXiv2020

The Iterated Projected Position Algorithm for Constructing Exponentially Localized Generalized Wannier Functions for Periodic and Non-Periodic Insulators in Two Dimensions and Higher

Localized bases play an important role in understanding electronic structure. In periodic insulators, a natural choice of localized basis is given by the Wannier functions which depend a choice of unitary transform known as a gauge transformation. Over the past few decades, there have been many works which have focused on optimizing the choice of gauge so that the corresponding Wannier functions are maximally localized or reflect some symmetry of the underlying system. In this work, we consider fully non-periodic materials where the usual Wannier functions are not well defined and gauge optimization is impossible. To tackle the problem of calculating exponentially localized generalized Wannier functions in both periodic and non-periodic system we discuss the "Iterated Projected Position (IPP)" algorithm. The IPP algorithm is based on matrix diagonalization and therefore unlike optimization based approaches it does not require initialization and cannot get stuck at a local minimum. Furthermore, the IPP algorithm is guaranteed by a rigorous analysis to produce exponentially localized functions under certain mild assumptions. We numerically demonstrate that the IPP algorithm can be used to calculate exponentially localized bases for the Haldane model, the Kane-Mele model (in both $\mathbb{Z}_2$ invariant even and $\mathbb{Z}_2$ invariant odd phases), and the $p_x + i p_y$ model on a quasi-crystal lattice.

math-ph↗

arXiv2020

Optimal Orbital Selection for Full Configuration Interaction (OptOrbFCI): Pursuing the Basis Set Limit under a Budget

Full configuration interaction (FCI) solvers are limited to small basis sets due to their expensive computational costs. An optimal orbital selection for FCI (OptOrbFCI) is proposed to boost the power of existing FCI solvers to pursue the basis set limit under a computational budget. The optimization problem coincides with that of the complete active space SCF method (CASSCF), while OptOrbFCI is algorithmically quite different. OptOrbFCI effectively finds an optimal rotation matrix via solving a constrained optimization problem directly to compress the orbitals of large basis sets to one with a manageable size, conducts FCI calculations only on rotated orbital sets, and produces a variational ground-state energy and its wave function. Coupled with coordinate descent full configuration interaction (CDFCI), we demonstrate the efficiency and accuracy of the method on the carbon dimer and nitrogen dimer under basis sets up to cc-pV5Z. We also benchmark the binding curve of the nitrogen dimer under the cc-pVQZ basis set with 28 selected orbitals, which provide consistently lower ground-state energies than the FCI results under the cc-pVDZ basis set. The dissociation energy in this case is found to be of higher accuracy.

physics.chem-ph↗