arXiv subjects

Jianfeng Lu

Publications and source records attributed to Jianfeng Lu.

At least 217 records · Page 12Linked to original sources

arXiv2019

Coordinate descent full configuration interaction

We develop an efficient algorithm, coordinate descent FCI (CDFCI), for the electronic structure ground state calculation in the configuration interaction framework. CDFCI solves an unconstrained non-convex optimization problem, which is a reformulation of the full configuration interaction eigenvalue problem, via an adaptive coordinate descent method with a deterministic compression strategy. CDFCI captures and updates appreciative determinants with different frequencies proportional to their importance. We show that CDFCI produces accurate variational energy for both static and dynamic correlation by benchmarking the binding curve of nitrogen dimer in the cc-pVDZ basis with $10^{-3}$ mHa accuracy. We also demonstrate the efficiency and accuracy of CDFCI for strongly correlated chromium dimer in the Ahlrichs VDZ basis and produces state-of-the-art variational energy.

physics.chem-ph↗

arXiv2019

A stochastic version of Stein Variational Gradient Descent for efficient sampling

We propose in this work RBM-SVGD, a stochastic version of Stein Variational Gradient Descent (SVGD) method for efficiently sampling from a given probability measure and thus useful for Bayesian inference. The method is to apply the Random Batch Method (RBM) for interacting particle systems proposed by Jin et al to the interacting particle systems in SVGD. While keeping the behaviors of SVGD, it reduces the computational cost, especially when the interacting kernel has long range. Numerical examples verify the efficiency of this new version of SVGD.

stat.ML↗

arXiv2019

Gradient flow structure and exponential decay of the sandwiched Rényi divergence for primitive Lindblad equations with GNS-detailed balance

We study the entropy production of the sandwiched Rényi divergence under the primitive Lindblad equation with GNS-detailed balance. We prove that the Lindblad equation can be identified as the gradient flow of the sandwiched Rényi divergence of any order $α \in (0, \infty)$. This extends a previous result by Carlen and Maas [Journal of Functional Analysis, 273(5), 1810-1869] for the quantum relative entropy (i.e., $α = 1$). Moreover, we show that the sandwiched Rényi divergence of any order $α \in (0, \infty)$ decays exponentially fast under the time-evolution of such a Lindblad equation.

math.FA↗

arXiv2019

Quadrature Points via Heat Kernel Repulsion

We discuss the classical problem of how to pick $N$ weighted points on a $d-$dimensional manifold so as to obtain a reasonable quadrature rule $$ \frac{1}{|M|}\int_{M}{f(x) dx} \simeq \frac{1}{N} \sum_{n=1}^{N}{a_i f(x_i)}.$$ This problem, naturally, has a long history; the purpose of our paper is to propose selecting points and weights so as to minimize the energy functional $$ \sum_{i,j =1}^{N}{ a_i a_j \exp\left(-\frac{d(x_i,x_j)^2}{4t}\right) } \rightarrow \min, \quad \mbox{where}~t \sim N^{-2/d},$$ $d(x,y)$ is the geodesic distance and $d$ is the dimension of the manifold. This yields point sets that are theoretically guaranteed, via spectral theoretic properties of the Laplacian $-Δ$, to have good properties. One nice aspect is that the energy functional is universal and independent of the underlying manifold; we show several numerical examples.

math.NA↗

arXiv2019

Generating Adversarial Examples With Conditional Generative Adversarial Net

Recently, deep neural networks have significant progress and successful application in various fields, but they are found vulnerable to attack instances, e.g., adversarial examples. State-of-art attack methods can generate attack images by adding small perturbation to the source image. These attack images can fool the classifier but have little impact to human. Therefore, such attack instances are difficult to generate by searching the feature space. How to design an effective and robust generating method has become a spotlight. Inspired by adversarial examples, we propose two novel generative models to produce adaptive attack instances directly, in which conditional generative adversarial network is adopted and distinctive strategy is designed for training. Compared with the common method, such as Fast Gradient Sign Method, our models can reduce the generating cost and improve robustness and has about one fifth running time for producing attack instance.

cs.LG↗

arXiv2019

Simulated Tempering Method in the Infinite Switch Limit with Adaptive Weight Learning

We investigate the theoretical foundations of the simulated tempering method and use our findings to design efficient algorithms. Employing a large deviation argument first used for replica exchange molecular dynamics [Plattner et al., J. Chem. Phys. 135:134111 (2011)], we demonstrate that the most efficient approach to simulated tempering is to vary the temperature infinitely rapidly. In this limit, we can replace the equations of motion for the temperature and physical variables by averaged equations for the latter alone, with the forces rescaled according to a position-dependent function defined in terms of temperature weights. The averaged equations are similar to those used in Gao's integrated-over-temperature method, except that we show that it is better to use a continuous rather than a discrete set of temperatures. We give a theoretical argument for the choice of the temperature weights as the reciprocal partition function, thereby relating simulated tempering to Wang-Landau sampling. Finally, we describe a self-consistent algorithm for simultaneously sampling the canonical ensemble and learning the weights during simulation. This algorithm is tested on a system of harmonic oscillators as well as a continuous variant of the Curie-Weiss model, where it is shown to perform well and to accurately capture the second-order phase transition observed in this model.

physics.chem-ph↗

arXiv2018

Randomized sampling for basis functions construction in generalized finite element methods

In the framework of generalized finite element methods for elliptic equations with rough coefficients, efficiency and accuracy of the numerical method depend critically on the use of appropriate basis functions. This work explores several random sampling strategies that construct approximations to the optimal set of basis functions of a given dimension, and proposes a quantitative criterion to analyze and compare these sampling strategies. Numerical evidence shows that the best results are achieved by two strategies, Random Gaussian and Smooth boundary sampling.

math.OC↗

arXiv2018

Weakly supervised segment annotation via expectation kernel density estimation

Since the labelling for the positive images/videos is ambiguous in weakly supervised segment annotation, negative mining based methods that only use the intra-class information emerge. In these methods, negative instances are utilized to penalize unknown instances to rank their likelihood of being an object, which can be considered as a voting in terms of similarity. However, these methods 1) ignore the information contained in positive bags, 2) only rank the likelihood but cannot generate an explicit decision function. In this paper, we propose a voting scheme involving not only the definite negative instances but also the ambiguous positive instances to make use of the extra useful information in the weakly labelled positive bags. In the scheme, each instance votes for its label with a magnitude arising from the similarity, and the ambiguous positive instances are assigned soft labels that are iteratively updated during the voting. It overcomes the limitations of voting using only the negative bags. We also propose an expectation kernel density estimation (eKDE) algorithm to gain further insight into the voting mechanism. Experimental results demonstrate the superiority of our scheme beyond the baselines.

cs.CV↗

arXiv2018

Hybrid Self-Attention Network for Machine Translation

The encoder-decoder is the typical framework for Neural Machine Translation (NMT), and different structures have been developed for improving the translation performance. Transformer is one of the most promising structures, which can leverage the self-attention mechanism to capture the semantic dependency from global view. However, it cannot distinguish the relative position of different tokens very well, such as the tokens located at the left or right of the current token, and cannot focus on the local information around the current token either. To alleviate these problems, we propose a novel attention mechanism named Hybrid Self-Attention Network (HySAN) which accommodates some specific-designed masks for self-attention network to extract various semantic, such as the global/local information, the left/right part context. Finally, a squeeze gate is introduced to combine different kinds of SANs for fusion. Experimental results on three machine translation tasks show that our proposed framework outperforms the Transformer baseline significantly and achieves superior results over state-of-the-art NMT systems.

cs.CL↗

arXiv2018

On Pointwise Products of Elliptic Eigenfunctions

We consider eigenfunctions of Schrödinger operators on a $d-$dimensional bounded domain $Ω$ (or a $d-$dimensional compact manifold $Ω$) with Dirichlet conditions. These operators give rise to a sequence of eigenfunctions $(ϕ_n)_{n \in \mathbb{N}}$. We study the subspace of all pointwise products $$ A_n = \mbox{span} \left\{ ϕ_i(x) ϕ_j(x): 1 \leq i,j \leq n\right\} \subseteq L^2(Ω).$$ Clearly, that vector space has dimension $\mbox{dim}(A_n) = n(n+1)/2$. We prove that products $ϕ_i ϕ_j$ of eigenfunctions are simple in a certain sense: for any $\varepsilon > 0$, there exists a low-dimensional vector space $B_n$ that almost contains all products. More precisely, denoting the orthogonal projection $Π_{B_n}:L^2(Ω) \rightarrow B_n$, we have $$ \forall~1 \leq i,j \leq n~ \qquad \|ϕ_iϕ_j - Π_{B_n}( ϕ_i ϕ_j) \|_{L^2} \leq \varepsilon$$ and the size of the space $\mbox{dim}(B_n)$ is relatively small $$ \mbox{dim}(B_n) \lesssim \left( \frac{1}{\varepsilon} \max_{1 \leq i \leq n} \|ϕ_i\|_{L^{\infty}} \right)^d n.$$ In the generic delocalized setting, this bound grows linearly up to logarithmic factors: pointwise products of eigenfunctions are low-rank. This has implications, among other things, for the validity of fast algorithms in electronic structure computations.

math.SP↗

arXiv2018

Approximating Pointwise Products of Laplacian Eigenfunctions

We consider Laplacian eigenfunctions on a $d-$dimensional bounded domain $M$ (or a $d-$dimensional compact manifold $M$) with Dirichlet conditions. These operators give rise to a sequence of eigenfunctions $(e_\ell)_{\ell \in \mathbb{N}}$. We study the subspace of all pointwise products $$ A_n = \mbox{span} \left\{ e_i(x) e_j(x): 1 \leq i,j \leq n\right\} \subseteq L^2(M).$$ Clearly, that vector space has dimension $\mbox{dim}(A_n) = n(n+1)/2$. We prove that products $e_i e_j$ of eigenfunctions are simple in a certain sense: for any $\varepsilon > 0$, there exists a low-dimensional vector space $B_n$ that almost contains all products. More precisely, denoting the orthogonal projection $Π_{B_n}:L^2(M) \rightarrow B_n$, we have $$ \forall~1 \leq i,j \leq n~ \qquad \|e_ie_j - Π_{B_n}( e_i e_j) \|_{L^2} \leq \varepsilon$$ and the size of the space $\mbox{dim}(B_n)$ is relatively small: for every $δ> 0$, $$ \mbox{dim}(B_n) \lesssim_{M,δ} \varepsilon^{-δ} n^{1+δ}.$$ We obtain the same sort of bounds for products of arbitrary length, as well for approximation in $H^{-1}$ norm. Pointwise products of eigenfunctions are low-rank. This has implications, among other things, for the validity of fast algorithms in electronic structure computations.

math.AP↗

arXiv2018

Exponential decay of Rényi divergence under Fokker-Planck equations

We prove the exponential convergence to the equilibrium, quantified by Rényi divergence, of the solution of the Fokker-Planck equation with drift given by the gradient of a strictly convex potential. This extends the classical exponential decay result on the relative entropy for the same equation.

math.AP↗

arXiv2018

Scaling limit of the Stein variational gradient descent: the mean field regime

We study an interacting particle system in $\mathbf{R}^d$ motivated by Stein variational gradient descent [Q. Liu and D. Wang, NIPS 2016], a deterministic algorithm for sampling from a given probability density with unknown normalization. We prove that in the large particle limit the empirical measure of the particle system converges to a solution of a non-local and nonlinear PDE. We also prove global existence, uniqueness and regularity of the solution to the limiting PDE. Finally, we prove that the solution to the PDE converges to the unique invariant solution in long time limit.

math.AP↗

arXiv2018

Learning interacting particle systems: diffusion parameter estimation for aggregation equations

In this article, we study the parameter estimation of interacting particle systems subject to the Newtonian aggregation and Brownian diffusion. Specifically, we construct an estimator $\widehatν$ with partial observed data to approximate the diffusion parameter $ν$, and the estimation error is achieved. Furthermore, we extend this result to general aggregation equations with a bounded Lipschitz interaction field.

math.AP↗

arXiv2018

Analysis of multiscale integrators for multiple attractors and irreversible Langevin samplers

We study multiscale integrator numerical schemes for a class of stiff stochastic differential equations (SDEs). We consider multiscale SDEs with potentially multiple attractors that behave as diffusions on graphs as the stiffness parameter goes to its limit. Classical numerical discretization schemes, such as the Euler-Maruyama scheme, become unstable as the stiffness parameter converges to its limit and appropriate multiscale integrators can correct for this. We rigorously establish the convergence of the numerical method to the related diffusion on graph, identifying the appropriate choice of discretization parameters. Theoretical results are supplemented by numerical studies on the problem of the recently developing area of introducing irreversibility in Langevin samplers in order to accelerate convergence to equilibrium.

math.NA↗

arXiv2018

Stop memorizing: A data-dependent regularization framework for intrinsic pattern learning

Deep neural networks (DNNs) typically have enough capacity to fit random data by brute force even when conventional data-dependent regularizations focusing on the geometry of the features are imposed. We find out that the reason for this is the inconsistency between the enforced geometry and the standard softmax cross entropy loss. To resolve this, we propose a new framework for data-dependent DNN regularization, the Geometrically-Regularized-Self-Validating neural Networks (GRSVNet). During training, the geometry enforced on one batch of features is simultaneously validated on a separate batch using a validation loss consistent with the geometry. We study a particular case of GRSVNet, the Orthogonal-Low-rank Embedding (OLE)-GRSVNet, which is capable of producing highly discriminative features residing in orthogonal low-rank subspaces. Numerical experiments show that OLE-GRSVNet outperforms DNNs with conventional regularization when trained on real data. More importantly, unlike conventional DNNs, OLE-GRSVNet refuses to memorize random data or random labels, suggesting it only learns intrinsic patterns by reducing the memorizing capacity of the baseline DNN.

cs.CV↗

arXiv2018

Double Path Networks for Sequence to Sequence Learning

Encoder-decoder based Sequence to Sequence learning (S2S) has made remarkable progress in recent years. Different network architectures have been used in the encoder/decoder. Among them, Convolutional Neural Networks (CNN) and Self Attention Networks (SAN) are the prominent ones. The two architectures achieve similar performances but use very different ways to encode and decode context: CNN use convolutional layers to focus on the local connectivity of the sequence, while SAN uses self-attention layers to focus on global semantics. In this work we propose Double Path Networks for Sequence to Sequence learning (DPN-S2S), which leverage the advantages of both models by using double path information fusion. During the encoding step, we develop a double path architecture to maintain the information coming from different paths with convolutional layers and self-attention layers separately. To effectively use the encoded context, we develop a cross attention module with gating and use it to automatically pick up the information needed during the decoding step. By deeply integrating the two paths with cross attention, both types of information are combined and well exploited. Experiments show that our proposed method can significantly improve the performance of sequence to sequence learning over state-of-the-art systems.

cs.CL↗

arXiv2018

The full configuration interaction quantum Monte Carlo method in the lens of inexact power iteration

In this paper, we propose a general analysis framework for inexact power iteration, which can be used to efficiently solve high dimensional eigenvalue problems arising from quantum many-body problems. Under the proposed framework, we establish the convergence theorems for several recently proposed randomized algorithms, including the full configuration interaction quantum Monte Carlo (FCIQMC) and the fast randomized iteration (FRI). The analysis is consistent with numerical experiments for physical systems such as Hubbard model and small chemical molecules. We also compare the algorithms both in convergence analysis and numerical results.

math.NA↗