arXiv subjects

Yousef Saad

Publications and source records attributed to Yousef Saad.

49 records · Page 3Linked to original sources

arXiv2017

Low-rank Label Propagation for Semi-supervised Learning with 100 Millions Samples

The success of semi-supervised learning crucially relies on the scalability to a huge amount of unlabelled data that are needed to capture the underlying manifold structure for better classification. Since computing the pairwise similarity between the training data is prohibitively expensive in most kinds of input data, currently, there is no general ready-to-use semi-supervised learning method/tool available for learning with tens of millions or more data points. In this paper, we adopted the idea of two low-rank label propagation algorithms, GLNP (Global Linear Neighborhood Propagation) and Kernel Nyström Approximation, and implemented the parallelized version of the two algorithms accelerated with Nesterov's accelerated projected gradient descent for Big-data Label Propagation (BigLP). The parallel algorithms are tested on five real datasets ranging from 7000 to 10,000,000 in size and a simulation dataset of 100,000,000 samples. In the experiments, the implementation can scale up to datasets with 100,000,000 samples and hundreds of features and the algorithms also significantly improved the prediction accuracy when only a very small percentage of the data is labeled. The results demonstrate that the BigLP implementation is highly scalable to big data and effective in utilizing the unlabeled data for semi-supervised learning.

cs.LG↗

arXiv2016

Fast estimation of approximate matrix ranks using spectral densities

In many machine learning and data related applications, it is required to have the knowledge of approximate ranks of large data matrices at hand. In this paper, we present two computationally inexpensive techniques to estimate the approximate ranks of such large matrices. These techniques exploit approximate spectral densities, popular in physics, which are probability density distributions that measure the likelihood of finding eigenvalues of the matrix at a given point on the real line. Integrating the spectral density over an interval gives the eigenvalue count of the matrix in that interval. Therefore the rank can be approximated by integrating the spectral density over a carefully selected interval. Two different approaches are discussed to estimate the approximate rank, one based on Chebyshev polynomials and the other based on the Lanczos algorithm. In order to obtain the appropriate interval, it is necessary to locate a gap between the eigenvalues that correspond to noise and the relevant eigenvalues that contribute to the matrix rank. A method for locating this gap and selecting the interval of integration is proposed based on the plot of the spectral density. Numerical experiments illustrate the performance of these techniques on matrices from typical applications.

math.NA↗

arXiv2015

A Thick-Restart Lanczos algorithm with polynomial filtering for Hermitian eigenvalue problems

Polynomial filtering can provide a highly effective means of computing all eigenvalues of a real symmetric (or complex Hermitian) matrix that are located in a given interval, anywhere in the spectrum. This paper describes a technique for tackling this problem by combining a Thick-Restart version of the Lanczos algorithm with deflation (`locking') and a new type of polynomial filters obtained from a least-squares technique. The resulting algorithm can be utilized in a `spectrum-slicing' approach whereby a very large number of eigenvalues and associated eigenvectors of the matrix are computed by extracting eigenpairs located in different sub-intervals independently from one another.

math.NA↗

arXiv2015

Low-rank correction methods for algebraic domain decomposition preconditioners

This paper presents a parallel preconditioning method for distributed sparse linear systems, based on an approximate inverse of the original matrix, that adopts a general framework of distributed sparse matrices and exploits the domain decomposition method and low-rank corrections. The domain decomposition approach decouples the matrix and once inverted, a low-rank approximation is applied by exploiting the Sherman-Morrison-Woodbury formula, which yields two variants of the preconditioning methods. The low-rank expansion is computed by the Lanczos procedure with reorthogonalizations. Numerical experiments indicate that, when combined with Krylov subspace accelerators, this preconditioner can be efficient and robust for solving symmetric sparse linear systems. Comparisons with other distributed-memory preconditioning methods are presented.

math.NA↗

arXiv2015

Schur Complement based domain decomposition preconditioners with Low-rank corrections

This paper introduces a robust preconditioner for general sparse symmetric matrices, that is based on low-rank approximations of the Schur complement in a Domain Decomposition (DD) framework. In this "Schur Low Rank" (SLR) preconditioning approach, the coefficient matrix is first decoupled by DD, and then a low-rank correction is exploited to compute an approximate inverse of the Schur complement associated with the interface points. The method avoids explicit formation of the Schur complement matrix. We show the feasibility of this strategy for a model problem, and conduct a detailed spectral analysis for the relationship between the low-rank correction and the quality of the preconditioning. Numerical experiments on general matrices illustrate the robustness and efficiency of the proposed approach.

math.NA↗

arXiv2014

Approximating spectral densities of large matrices

In physics, it is sometimes desirable to compute the so-called \emph{Density Of States} (DOS), also known as the \emph{spectral density}, of a real symmetric matrix $A$. The spectral density can be viewed as a probability density distribution that measures the likelihood of finding eigenvalues near some point on the real line. The most straightforward way to obtain this density is to compute all eigenvalues of $A$. But this approach is generally costly and wasteful, especially for matrices of large dimension. There exists alternative methods that allow us to estimate the spectral density function at much lower cost. The major computational cost of these methods is in multiplying $A$ with a number of vectors, which makes them appealing for large-scale problems where products of the matrix $A$ with arbitrary vectors are relatively inexpensive. This paper defines the problem of estimating the spectral density carefully, and discusses how to measure the accuracy of an approximate spectral density. It then surveys a few known methods for estimating the spectral density, and proposes some new variations of existing methods. All methods are discussed from a numerical linear algebra point of view.

math.NA↗

arXiv2014

Efficient estimation of eigenvalue counts in an interval

Estimating the number of eigenvalues located in a given interval of a large sparse Hermitian matrix is an important problem in certain applications and it is a prerequisite of eigensolvers based on a divide-and-conquer paradigm. Often an exact count is not necessary and methods based on stochastic estimates can be utilized to yield rough approximations. This paper examines a number of techniques tailored to this specific task. It reviews standard approaches and explores new ones based on polynomial and rational approximation filtering combined with a stochastic procedure.

math.NA↗

arXiv2014

Fast updating algorithms for latent semantic indexing

This paper discusses a few algorithms for updating the approximate Singular Value Decomposition (SVD) in the context of information retrieval by Latent Semantic Indexing (LSI) methods. A unifying framework is considered which is based on Rayleigh-Ritz projection methods. First, a Rayleigh-Ritz approach for the SVD is discussed and it is then used to interpret the Zha--Simon algorithms [SIAM J. Scient. Comput. vol. 21 (1999), pp. 782-791]. This viewpoint leads to a few alternatives whose goal is to reduce computational cost and storage requirement by projection techniques that utilize subspaces of much smaller dimension. Numerical experiments show that the proposed algorithms yield accuracies comparable to those obtained from standard ones at a much lower computational cost.

math.NA↗

arXiv2013

Graph partitioning using matrix values for preconditioning symmetric positive definite systems

Prior to the parallel solution of a large linear system, it is required to perform a partitioning of its equations/unknowns. Standard partitioning algorithms are designed using the considerations of the efficiency of the parallel matrix-vector multiplication, and typically disregard the information on the coefficients of the matrix. This information, however, may have a significant impact on the quality of the preconditioning procedure used within the chosen iterative scheme. In the present paper, we suggest a spectral partitioning algorithm, which takes into account the information on the matrix coefficients and constructs partitions with respect to the objective of enhancing the quality of the nonoverlapping additive Schwarz (block Jacobi) preconditioning for symmetric positive definite linear systems. For a set of test problems with large variations in magnitudes of matrix coefficients, our numerical experiments demonstrate a noticeable improvement in the convergence of the resulting solution scheme when using the new partitioning approach.

math.NA↗

arXiv2011

Lanczos-based Low-Rank Correction Method for Solving the Dyson Equation in Inhomogenous Dynamical Mean-Field Theory

Inhomogeneous dynamical mean-field theory has been employed to solve many interesting strongly interacting problems from transport in multilayered devices to the properties of ultracold atoms in a trap. The main computational step, especially for large systems, is the problem of calculating the inverse of a large sparse matrix to solve Dyson's equation and determine the local Green's function at each lattice site from the corresponding local self-energy. We present a new efficient algorithm, the Lanczos-based low-rank algorithm, for the calculation of the inverse of a large sparse matrix which yields this local (imaginary time) Green's function. The Lanczos-based low-rank algorithm is based on a domain decomposition viewpoint, but avoids explicit calculation of Schur complements and relies instead on low-rank matrix approximations derived from the Lanczos algorithm, for solving the Dyson equation. We report at least a 25-fold improvement of performance compared to explicit decomposition (such as sparse LU) of the matrix inverse. We also report that scaling relative to matrix sizes, of the low-rank correction method on the one hand and domain decomposition methods on the other, are comparable.

cond-mat.str-el↗

arXiv2008

Turbo charging time-dependent density-functional theory with Lanczos chains

We introduce a new implementation of time-dependent density-functional theory which allows the \emph{entire} spectrum of a molecule or extended system to be computed with a numerical effort comparable to that of a \emph{single} standard ground-state calculation. This method is particularly well suited for large systems and/or large basis sets, such as plane waves or real-space grids. By using a super-operator formulation of linearized time-dependent density-functional theory, we first represent the dynamical polarizability of an interacting-electron system as an off-diagonal matrix element of the resolvent of the Liouvillian super-operator. One-electron operators and density matrices are treated using a representation borrowed from time-independent density-functional perturbation theory, which permits to avoid the calculation of unoccupied Kohn-Sham orbitals. The resolvent of the Liouvillian is evaluated through a newly developed algorithm based on the non-symmetric Lanczos method. Each step of the Lanczos recursion essentially requires twice as many operations as a single step of the iterative diagonalization of the unperturbed Kohn-Sham Hamiltonian. Suitable extrapolation of the Lanczos coefficients allows for a dramatic reduction of the number of Lanczos steps necessary to obtain well converged spectra, bringing such number down to hundreds (or a few thousands, at worst) in typical plane-wave pseudopotential applications. The resulting numerical workload is only a few times larger than that needed by a ground-state Kohn-Sham calculation for a same system. Our method is demonstrated with the calculation of the spectra of benzene, C$_{60}$ fullerene, and of chlorofyll a.

cond-mat.mtrl-sci↗

arXiv2007

Parallel Self-Consistent-Field Calculations via Chebyshev-Filtered Subspace Acceleration

Solving the Kohn-Sham eigenvalue problem constitutes the most computationally expensive part in self-consistent density functional theory (DFT) calculations. In a previous paper, we have proposed a nonlinear Chebyshev-filtered subspace iteration method, which avoids computing explicit eigenvectors except at the first SCF iteration. The method may be viewed as an approach to solve the original nonlinear Kohn-Sham equation by a nonlinear subspace iteration technique, without emphasizing the intermediate linearized Kohn-Sham eigenvalue problem. It reaches self-consistency within a similar number of SCF iterations as eigensolver-based approaches. However, replacing the standard diagonalization at each SCF iteration by a Chebyshev subspace filtering step results in a significant speedup over methods based on standard diagonalization. Here, we discuss an approach for implementing this method in multi-processor, parallel environment. Numerical results are presented to show that the method enables to perform a class of highly challenging DFT calculations that were not feasible before.

cond-mat.mtrl-sci↗

arXiv2006

The Evolution of Magnetism in Iron from the Atom to the Bulk

The evolution of the magnetic moment in iron clusters containing 20 to 400 atoms is investigated using first-principles numerical calculations based on density-functional theory and real-space pseudopotentials. Three families of clusters are studied, characterized by the arrangement of atoms: icosahedral, body-centered cubic centered on an atom site, and body-centered cubic centered on the bridge between two neighboring atoms. We find an overall decrease of magnetic moment as the clusters grow in size towards the bulk limit. Clusters with faceted surfaces are predicted to have magnetic moment lower than other clusters with similar size. As a result, the magnetic moment is observed to decrease as function of size in a non-monotonic manner, which explains measurements performed at low temperatures.

cond-mat.mtrl-sci↗