arXiv subjects

Lexing Ying

Publications and source records attributed to Lexing Ying.

At least 145 records · Page 8Linked to original sources

arXiv2019

Recursively Preconditioned Hierarchical Interpolative Factorization for Elliptic Partial Differential Equations

The hierarchical interpolative factorization for elliptic partial differential equations is a fast algorithm for approximate sparse matrix inversion in linear or quasilinear time. Its accuracy can degrade, however, when applied to strongly ill-conditioned problems. Here, we propose a simple modification that can significantly improve the accuracy at no additional asymptotic cost: applying a block Jacobi preconditioner before each level of skeletonization. This dramatically limits the impact of the underlying system conditioning and enables the construction of robust and highly efficient preconditioners even at quite modest compression tolerances. Numerical examples demonstrate the performance of the new approach.

math.NA↗

arXiv2019

Analytical low-rank compression via proxy point selection

It has been known in potential theory that, for some kernels matrices corresponding to well-separated point sets, fast analytical low-rank approximation can be achieved via the use of proxy points. This proxy point method gives a surprisingly convenient way of explicitly writing out approximate basis matrices for a kernel matrix. However, this elegant strategy is rarely known or used in the numerical linear algebra community. It still needs clear algebraic understanding of the theoretical background. Moreover, rigorous quantifications of the approximation errors and reliable criteria for the selection of the proxy points are still missing. In this work, we use contour integration to clearly justify the idea in terms of a class of important kernels. We further provide comprehensive accuracy analysis for the analytical compression and show how to choose nearly optimal proxy points. The analytical compression is then combined with fast rank-revealing factorizations to get compact low-rank approximations and also to select certain representative points. We provide the error bounds for the resulting overall low-rank approximation. This work thus gives a fast and reliable strategy for compressing those kernel matrices. Furthermore, it provides an intuitive way of understanding the proxy point method and bridges the gap between this useful analytical strategy and practical low-rank approximations. Some numerical examples help to further illustrate the ideas.

math.NA↗

arXiv2019

A multiscale neural network based on hierarchical nested bases

In recent years, deep learning has led to impressive results in many fields. In this paper, we introduce a multi-scale artificial neural network for high-dimensional non-linear maps based on the idea of hierarchical nested bases in the fast multipole method and the $\mathcal{H}^2$-matrices. This approach allows us to efficiently approximate discretized nonlinear maps arising from partial differential equations or integral equations. It also naturally extends our recent work based on the generalization of hierarchical matrices [Fan et al. arXiv:1807.01883] but with a reduced number of parameters. In particular, the number of parameters of the neural network grows linearly with the dimension of the parameter space of the discretized PDE. We demonstrate the properties of the architecture by approximating the solution maps of non-linear Schr{ö}dinger equation, the radiative transfer equation, and the Kohn-Sham map.

math.NA↗

arXiv2018

SwitchNet: a neural network model for forward and inverse scattering problems

We propose a novel neural network architecture, SwitchNet, for solving the wave equation based inverse scattering problems via providing maps between the scatterers and the scattered field (and vice versa). The main difficulty of using a neural network for this problem is that a scatterer has a global impact on the scattered wave field, rendering typical convolutional neural network with local connections inapplicable. While it is possible to deal with such a problem using a fully connected network, the number of parameters grows quadratically with the size of the input and output data. By leveraging the inherent low-rank structure of the scattering problems and introducing a novel switching layer with sparse connections, the SwitchNet architecture uses much fewer parameters and facilitates the training process. Numerical experiments show promising accuracy in learning the forward and inverse maps between the scatterers and the scattered wave field.

math.NA↗

arXiv2018

BCR-Net: a neural network based on the nonstandard wavelet form

This paper proposes a novel neural network architecture inspired by the nonstandard form proposed by Beylkin, Coifman, and Rokhlin in [Communications on Pure and Applied Mathematics, 44(2), 141-183]. The nonstandard form is a highly effective wavelet-based compression scheme for linear integral operators. In this work, we first represent the matrix-vector product algorithm of the nonstandard form as a linear neural network where every scale of the multiresolution computation is carried out by a locally connected linear sub-network. In order to address nonlinear problems, we propose an extension, called BCR-Net, by replacing each linear sub-network with a deeper and more powerful nonlinear one. Numerical results demonstrate the efficiency of the new architecture by approximating nonlinear maps that arise in homogenization theory and stochastic computation.

math.NA↗

arXiv2018

Sparsifying preconditioner for the time-harmonic Maxwell's equations

This paper presents the sparsifying preconditioner for the time-harmonic Maxwell's equations in the integral formulation. Following the work on sparsifying preconditioner for the Lippmann-Schwinger equation, this paper generalizes that approach from the scalar wave case to the vector case. The key idea is to construct a sparse approximation to the dense system by minimizing the non-local interactions in the integral equation, which allows for applying sparse linear solvers to reduce the computational cost. When combined with the standard GMRES solver, the number of preconditioned iterations remains small and essentially independent of the frequency. This suggests that, when the sparsifying preconditioner is adopted, solving the dense integral system can be done as efficiently as solving the sparse system from PDE discretization.

math.NA↗

arXiv2018

Convex relaxation approaches for strictly correlated density functional theory

In this paper, we introduce methods from convex optimization to solve the multimarginal transport type problems arise in the context of density functional theory. Convex relaxations are used to provide outer approximation to the set of $N$-representable 2-marginals and 3-marginals, which in turn provide lower bounds to the energy. We further propose rounding schemes to obtain upper bound to the energy. Numerical experiments demonstrate a gap of the order of $10^{-3}$ to $10^{-2}$ between the upper and lower bounds. The Kantorovich potential of the multi-marginal transport problem is also approximated with a similar accuracy.

math.OC↗

arXiv2018

An entropic fourier method for the Boltzmann equation

We propose an entropic Fourier method for the numerical discretization of the Boltzmann collision operator. The method, which is obtained by modifying a Fourier Galerkin method to match the form of the discrete velocity method, can be viewed both as a discrete velocity method and as a Fourier method. As a discrete velocity method, it preserves the positivity of the solution and satisfies a discrete version of the H-theorem. As a Fourier method, it allows one to readily apply the FFT-based fast algorithms. A second-order convergence rate is validated by numerical experiments

math.NA↗

arXiv2018

Solving parametric PDE problems with artificial neural networks

The curse of dimensionality is commonly encountered in numerical partial differential equations (PDE), especially when uncertainties have to be modeled into the equations as random coefficients. However, very often the variability of physical quantities derived from a PDE can be captured by a few features on the space of the coefficient fields. Based on such an observation, we propose using a neural-network (NN) based method to parameterize the physical quantity of interest as a function of input coefficients. The representability of such quantity using a neural-network can be justified by viewing the neural-network as performing time evolution to find the solutions to the PDE. We further demonstrate the simplicity and accuracy of the approach through notable examples of PDEs in engineering and physics.

math.NA↗

arXiv2018

Solving for high dimensional committor functions using artificial neural networks

In this note we propose a method based on artificial neural network to study the transition between states governed by stochastic processes. In particular, we aim for numerical schemes for the committor function, the central object of transition path theory, which satisfies a high-dimensional Fokker-Planck equation. By working with the variational formulation of such partial differential equation and parameterizing the committor function in terms of a neural network, approximations can be obtained via optimizing the neural network weights using stochastic algorithms. The numerical examples show that moderate accuracy can be achieved for high-dimensional problems.

cs.LG↗

arXiv2018

Fast algorithms for integral formulations of steady-state radiative transfer equation

We investigate integral formulations and fast algorithms for the steady-state radiative transfer equation with isotropic and anisotropic scattering. When the scattering term is a smooth convolution on the unit sphere, a model reduction step in the angular domain using the Fourier transformation in 2D and the spherical harmonic transformation in 3D significantly reduces the number of degrees of freedoms. The resulting Fourier coefficients or spherical harmonic coefficients satisfy a Fredholm integral equation of the second kind. We study the uniqueness of the equation and proved an a priori estimate. For a homogeneous medium, the integral equation can be solved efficiently using the FFT and iterative methods. For an inhomogeneous medium, the recursive skeletonization factorization method is applied instead. Numerical simulations demonstrate the efficiency of the proposed algorithms in both homogeneous and inhomogeneous cases and for both transport and diffusion regimes.

math.NA↗

arXiv2017

Sparsify and sweep: an efficient preconditioner for the Lippmann-Schwinger equation

This paper presents an efficient preconditioner for the Lippmann-Schwinger equation that combines the ideas of the sparsifying and the sweeping preconditioners. Following first the idea of the sparsifying preconditioner, this new preconditioner starts by transforming the dense linear system of the Lippmann-Schwinger equation into a nearly sparse system. The key novelty is a newly designed perfectly matched layer (PML) stencil for the boundary degrees of freedoms. The resulting sparse system gives rise to fairly accurate solutions and hence can be viewed as an accurate discretization of the Helmholtz equation. This new PML stencil also paves the way for applying the moving PML sweeping preconditioner to invert the resulting sparse system approximately. When combined with the standard GMRES solver, this new preconditioner for the Lippmann-Schwinger equation takes only a few iterations to converge for both 2D and 3D problems, where the iteration numbers are almost independent of the frequency. To the best of our knowledge, this is the first method that achieves near-linear cost to solve the 3D Lippmann-Schwinger equation in high frequency cases.

math.NA↗

arXiv2017

Fast spatial Gaussian process maximum likelihood estimation via skeletonization factorizations

Maximum likelihood estimation for parameter-fitting given observations from a Gaussian process in space is a computationally-demanding task that restricts the use of such methods to moderately-sized datasets. We present a framework for unstructured observations in two spatial dimensions that allows for evaluation of the log-likelihood and its gradient (i.e., the score equations) in $\tilde O(n^{3/2})$ time under certain assumptions, where $n$ is the number of observations. Our method relies on the skeletonization procedure described by Martinsson & Rokhlin in the form of the recursive skeletonization factorization of Ho & Ying. Combining this with an adaptation of the matrix peeling algorithm of Lin et al. for constructing $\mathcal{H}$-matrix representations of black-box operators, we obtain a framework that can be used in the context of any first-order optimization routine to quickly and accurately compute maximum-likelihood estimates.

stat.ME↗

arXiv2017

Computing localized representations of the kohn-sham subspace via randomization and refinement

Localized representation of the Kohn-Sham subspace plays an important role in quantum chemistry and materials science. The recently developed selected columns of the density matrix (SCDM) method [J. Chem. Theory Comput. 11, 1463, 2015] is a simple and robust procedure for finding a localized representation of a set of Kohn-Sham orbitals from an insulating system. The SCDM method allows the direct construction of a well conditioned (or even orthonormal) and localized basis for the Kohn-Sham subspace. The SCDM algorithm avoids the use of an optimization procedure and does not depend on any adjustable parameters. The most computationally expensive step of the SCDM method is a column pivoted QR factorization that identifies the important columns for constructing the localized basis set. In this paper, we develop a two stage approximate column selection strategy to find the important columns at much lower computational cost. We demonstrate the effectiveness of this process using the dissociation process of a BH$_3$NH$_3$ molecule, an alkane chain and a supercell with 256 water molecules. Numerical results for the large collection of water molecules show that two stage localization procedure can be more than 30 times faster than the original SCDM algorithm and compares favorably with the popular Wannier90 package.

math.NA↗

arXiv2017

Localized Sparsifying Preconditioner for Periodic Indefinite Systems

This paper introduces the localized sparsifying preconditioner for the pseudospectral approximations of indefinite systems on periodic structures. The work is built on top of the recently proposed sparsifying preconditioner with two major modifications. First, the local potential information is utilized to improve the accuracy of the preconditioner. Second, an FFT based method to compute the local stencil is proposed to reduce the setup time of the algorithm. Numerical results show that the iteration number of this improved method grows only mildly as the problem size grows, which implies that solving pseudospectral approximation systems is computationally as efficient as solving sparse systems, up to a mildly growing factor.

math.NA↗

arXiv2017

Robust and efficient multi-way spectral clustering

We present a new algorithm for spectral clustering based on a column-pivoted QR factorization that may be directly used for cluster assignment or to provide an initial guess for k-means. Our algorithm is simple to implement, direct, and requires no initial guess. Furthermore, it scales linearly in the number of nodes of the graph and a randomized variant provides significant computational gains. Provided the subspace spanned by the eigenvectors used for clustering contains a basis that resembles the set of indicator vectors on the clusters, we prove that both our deterministic and randomized algorithms recover a basis close to the indicators in Frobenius norm. We also experimentally demonstrate that the performance of our algorithm tracks recent information theoretic bounds for exact recovery in the stochastic block model. Finally, we explore the performance of our algorithm when applied to a real world graph.

math.NA↗

arXiv2017

Distributed-memory Hierarchical Interpolative Factorization

The hierarchical interpolative factorization (HIF) offers an efficient way for solving or preconditioning elliptic partial differential equations. By exploiting locality and low-rank properties of the operators, the HIF achieves quasi-linear complexity for factorizing the discrete positive definite elliptic operator and linear complexity for solving the associated linear system. In this paper, the distributed-memory HIF (DHIF) is introduced as a parallel and distributed-memory implementation of the HIF. The DHIF organizes the processes in a hierarchical structure and keep the communication as local as possible. The computation complexity is $O\left(\frac{N\log N}{P}\right)$ and $O\left(\frac{N}{P}\right)$ for constructing and applying the DHIF, respectively, where $N$ is the size of the problem and $P$ is the number of processes. The communication complexity is $O\left(\sqrt{P}\log^3 P\right)α+ O\left(\frac{N^{2/3}}{\sqrt{P}}\right)β$ where $α$ is the latency and $β$ is the inverse bandwidth. Extensive numerical examples are performed on the NERSC Edison system with up to 8192 processes. The numerical results agree with the complexity analysis and demonstrate the efficiency and scalability of the DHIF.

math.NA↗

arXiv2017

A recursive skeletonization factorization based on strong admissibility

We introduce the strong recursive skeletonization factorization (RS-S), a new approximate matrix factorization based on recursive skeletonization for solving discretizations of linear integral equations associated with elliptic partial differential equations in two and three dimensions (and other matrices with similar hierarchical rank structure). Unlike previous skeletonization-based factorizations, RS-S uses a simple modification of skeletonization, strong skeletonization, which compresses only far-field interactions. This leads to an approximate factorization in the form of a product of many block unit-triangular matrices that may be used as a preconditioner or moderate-accuracy direct solver, with dramatically reduced rank growth. We further combine the strong skeletonization procedure with alternating near-field compression to obtain the hybrid recursive skeletonization factorization (RS-WS), a modification of RS-S that exhibits reduced storage cost in many settings. Under suitable rank assumptions both RS-S and RS-WS exhibit linear computational complexity, which we demonstrate with a number of numerical examples.

math.NA↗