arXiv subjects

Qi Luan

Publications and source records attributed to Qi Luan.

10 recordsLinked to original sources

arXiv2023

Fast Approximation of Polynomial Zeros and Matrix Eigenvalues

We approximate the d complex zeros of a univariate polynomial p(x) of a degree d or those zeros that lie in a fixed region of interest on the complex plane such as a disc or a square. Our divide and conquer algorithm of STOC 1995 supports solution of this problem in optimal Boolean time (up to a poly-logarithmic factor), that is, runs nearly as fast as one can access the coefficients of p with the precision necessary to support required accuracy of the output. That record complexity has not been matched by any other algorithm yet, but our root-finder of 1995 is quite involved and has never been implemented. We present alternative nearly optimal root-finders based on our novel variants of the classical subdivision iterations. Unlike our predecessor of 1995, we require randomization of Las Vegas type, allowing us to detect any output error at a dominated computational cost, but our new root-finders are much simpler to implement than their predecessor of 1995. According to the results of extensive test with standard test polynomials for their preliminary version, which incorporates only a part of our novel techniques, the new root-finders compete and for a large class of inputs significantly supersedes the package of root-finding subroutines MPSolve, which for decades has been user's choice package. Unlike our predecessor of 1995 and all known fast algorithms for the cited tasks of polynomial root-finding, our new algorithms can be also applied to a polynomial given by a black box oracle for its evaluation rather than by its coefficients. This makes our root-finders particularly efficient for polynomials p(x) that can be evaluated fast such as the Mandelbrot polynomials or those given by the sum of a small number of shifted monomials. Our algorithm can be readily extended to fast approximation of the eigenvalues of a matrix or a matrix polynomial.

cs.SC↗

arXiv2021

Superfast Approximate Linear Least Squares Solution of a Highly Overdetermined Linear System of Equations

With a high probability the Sarlos randomized algorithm of 2006 outputs a nearly optimal least squares solution of a highly overdeterminedlinear system of equations. We propose its simple deterministic variation which computes such a solution for a random input whp and therefore computes it deterministically for a large input class. Unlike the Sarlos original algorithm our variation performs computations at sublinear cost or, as we say, superfast, that is, by using much fewer memory cells and arithmetic operations than an input matrix has entries. Our extensive tests are in good accordance with this result.

math.NA↗

arXiv2021

CUR Low Rank Approximation at Deterministic Sublinear Cost

A matrix algorithm runs at {\em sublinear cost} if it uses much fewer memory cells and arithmetic operations than the input matrix has entries. Such algorithms are indispensable for Big Data Mining and Analysis. Quite typically in that area the input matrices are so immense that realistically one can only access a small fraction of all their entries but can access and process at sublinear cost their Low Rank Approximation {\em (LRA)}. Can, however, we compute LRA at sublinear cost? Adversary argument shows that the output of any algorithm running at sublinear cost is extremely far from LRA of the worst case input matrices and even of the matrices of small families of our Appendix, but we prove that some deterministic sublinear cost algorithms output reasonably close LRA in a memory efficient form of CUR LRA if an input matrix admits LRA and is Symmetric Positive Semidefinite or is very close to a low rank matrix. The latter result is technically simple but provides some (very limited but long overdue) support for the well-known empirical efficiency of sublinear cost LRA by means of Cross-Approximation. We demonstrate the power of application of such LRA by turning the Fast Multipole celebrated Method into Superfast Multipole Method. The design and analysis of our algorithms rely on extensive prior study of the link of LRA of a matrix to maximization of its volume.

math.NA↗

arXiv2019

Low Rank Approximation at Sublinear Cost

Low Rank Approximation (LRA) of a matrix is a hot research subject, fundamental for Matrix and Tensor Computations and Big Data Mining and Analysis. Computations with low rank matrices can be performed at sublinear cost -- by using much fewer floating-point operations (flops) than an input matrix has entries, but can we compute LRA at sublinear cost? This is routinely done in computational practice for a large class of inputs, even though any sublinear cost LRA algorithm fails most miserably on worst case matrices. To provide insight into this controversy we first accelerate some popular near-optimal random sketching LRA algorithms -- to run them at sublinear cost. Then we define two probabilistic structures in the space of input matrices and estimate that the expected spectral and Frobenius error norms for the output LRA of the accelerated algorithms stay within a reasonable factor from their optima under both models, and so these sublinear cost algorithms only fail for a very narrow input class. Our upper estimates for their output accuracy are still quite high, but under some additional semi-heuristic amendments the algorithms have consistently output accurate LRA of various synthetic and real-world matrices in our numerical tests.

math.NA↗

arXiv2019

CUR Low Rank Approximation of a Matrix at Sublinear Cost

Low rank approximation of a matrix (hereafter LRA) is a highly important area of Numerical Linear and Multilinear Algebra and Data Mining and Analysis. One can operate with an LRA at sublinear cost -- by using much fewer memory cells and flops than an input matrix M has entries. For worst case inputs one cannot compute even a reasonably close LRA at sublinear cost, but in computational practice accurate LRAs, even in their memory efficient form of CUR LRAs, are routinely obtained at sublinear cost for large and important classes of matrices, in particular by means of Cross-Approximation iterations, which specialize Alternating Direction techniques to LRA. We identify some classes of matrices for which CUR LRA are computed at sublinear cost as well as some sublinear cost LRA algorithms that are empirically accurate for large classes of inputs. Some of our techniques and concepts can be of independent interests.

math.NA↗

arXiv2019

Iterative Refinement and Oversampling for Low Rank Approximation

Iterative refinement is particularly popular for numerical solution of linear systems of equations. We extend it to Low Rank Approximation of a matrix (LRA) and observe close link of the resulting algorithm to oversampling techniques, commonly used in randomized LRA algorithms. We elaborate upon this link and revisit oversampling and some efficient randomized LRA algorithms. Applied with sparse sketch matrices they run significantly faster and in particular yield Very Low Rank Approximation (VLRA) at sublinear cost, using much fewer scalars and flops than the input matrix has entries. This is achieved at the price of deterioration of output accuracy, but according to our formal and empirical study subsequent oversampling improves accuracy to near-optimal level under the spectral norm for a large sub-class of matrices with fast decaying spectra of singular values.

math.NA↗

arXiv2019

Superfast iterative refinement of low rank approximation of a matrix based on ALS method and random sampling

A matrix algorithm runs superfast (aka at sublinear cost) if it involves much fewer flops and memory cells than an input matrix has entries. Big Data are frequently represented by matrices of immense sizes that cannot be handled directly but can be approximated with low rank matrices, with which one can operate superfast. Superfast computation of Low Rank Approximations (LRA) of a matrix, however, can be a challenge. Any superfast LRA algorithm fails miserably on worst case matrices. Fortunately, they rarely appear in computational practice. For an important example, superfast Adaptive Cross--Approximation iterations have consistently output accurate LRA of a large and important class of matrices during the decades of their worldwide application. Furthermore, they output CUR LRA, which is a special attractive form of LRA, widely applied in data analysis. %VP Adequate formal support for that empirical behavior, however, is still a challenge. Encouraged by success of these iterations we present a superfast randomized iterative refinement of a crude CUR LRA by means of combining random sampling with the Alternating Least Squares Method, which reduces this refinement to recursive solution of generalized Linear Least Squares Problems (LLSPs). We prove monotone convergence of our iterations with a high probability to a near-optimal LRA of an input matrix under some specified assumptions on it and on a crude approximate solution to an initial generalized LLSP. In our numerical tests two or three iterations of our algorithm have consistently and significantly improved crude initial LRAs of real world inputs.

math.NA↗

arXiv2018

Superfast Low Rank Approximation

Low rank approximation of a matrix (LRA) is a highly important area of Numerical Linear and Multilinear Algebra and Data Mining and Analysis. One can operate with an LRA superfast -- by using much fewer memory cells and flops than an input matrix has entries. Can we, however, compute an LRA of a matrix superfast? YES and NO. For worst case inputs, any LRA algorithm fails miserably unless it involves all input entries, but in computational practice worst case inputs seem to appear rarely, and accurate LRA are routinely computed superfast for large and important classes of matrices, in particular in the memory efficient form of CUR, widely used in data analysis. We advance formal study of this YES and NO coexistence by proving novel universal upper bounds on the spectral output error norms of all CUR LRA algorithms and, under a fixed probabilistic structure in the space of input matrices, on both spectral and Frobenius error norms of nearly all sketching LRA algorithms. These bounds imply that superfast LRA algorithms of the two kinds fail miserably only for a very narrow input class. Furthermore, in our numerical tests such superfast algorithms were consistently much more accurate than our upper estimates ensure and usually were reasonably close to optimal.

math.NA↗

arXiv2017

Superfast CUR Matrix Algorithms, Their Pre-Processing and Extensions

We study superfast algorithms that computes low rank approximation of a matrix (hereafter referred to as LRA) that use much fewer memory cells and arithmetic operations than the input matrix has entries. We first specify a family of 2mn matrices of size m*n such that for almost 50% of them any superfast LRA algorithm fails to improve the poor trivial approximation by the matrix filled with zeros, but then we prove that the class of all such hard inputs is narrow - the cross-approximation (hereafter {C-A}) superfast iterations as well as some more primitive superfast algorithms compute reasonably accurate LRAs in their transparent CUR form (i) to any matrix allowing close LRA except for small norm perturbations of matrices of an algebraic variety of a smaller dimension, (ii) to the average matrix allowing close LRA, (iii) to the average sparse matrix allowing close LRA and (iv) with a high probability to any matrix allowing close LRA if it is pre-processed fast with a random Gaussian, SRHT or SRFT multiplier. Moreover empirically the output LRAs remain accurate when we perform the computations superfast by replacing such a multiplier with one of our sparse and structured multipliers. Our techniques, auxiliary results and extensions may be of some independent interest. We analyze C-A and other superfast algorithms twice -- based on two well-known sufficient criteria for obtaining accurate LRAs. We provide a distinct proof in the case of superfast variant of randomized algorithms of [DMM08], improve a decade-old estimate for the norm of the inverse of a Gaussian matrix, prove such an estimate also in the case of a sparse Gaussian matrix, present some novel advanced pre-processing techniques for fast and superfast computation of LRA, and extend our results to dramatic acceleration of the Fast Multipole Method (FMM) and the Conjugate Gradient algorithms.

math.NA↗

arXiv2016

Superfast Low-Rank Approximation and Least Squares Regression

Low Rank Approximation is among most fundamental subjects of numerical linear algebra having important applications to various areas of modern computing and %they range from machine learning theory and %neural networks to data mining and analysis. The known algorithms compute such approximations by using more flops than the input matrix has entries, but we prove that much fewer flops than entries are sufficient in the case of the average input ("flop" stands for "floating point arithmetic operation"). We prove this twice -- for the solutions by means of two distinct algorithms, and we analyze them by applying two different approaches. Our analysis of both algorithms is quite involved, but we devise them mostly by simplifying, combining, and ameliorating the known techniques, although we propose some technical novelties for further enhancing the performance of the popular Cross-Approximation Algorithms. They are highly efficient empirically, and we prove that they are efficient for the average input. We specify some narrow classes of hard inputs for which the presented algorithms fail with high probability even when we randomize them, but we narrow such classes further by means of preprocessing with new sparse and structured multipliers. The average complexity estimates do not cover many realistic input classes, but our formal analysis is in good accordance with the results of our tests applied to benchmark inputs from discretized PDEs and Integral quations and to random inputs. Our work should already be of practical value but also leads to research challenges. At the end we list some of them, propose two novel extensions of our progress -- to the acceleration of the Fast Multipole Method and Conjugate Gradient algorithms, and explore and slightly extend the recent techniques of Osinsky, which enhance the output accuracy of CUR Approximation.

math.NA↗