Searcharxiv⌕ Search

arXiv subjects

Victor Y. Pan

Publications and source records attributed to Victor Y. Pan.

At least 37 records · Page 2Linked to original sources
arXiv2018

Superfast Low-Rank Approximation and Least Squares Regression

Low Rank Approximation is among most fundamental subjects of numerical linear algebra having important applications to various areas of modern computing and %they range from machine learning theory and %neural networks to data mining and analysis. The known algorithms compute such approximations by using more flops than the input matrix has entries, but we prove that much fewer flops than entries are sufficient in the case of the average input ("flop" stands for "floating point arithmetic operation"). We prove this twice -- for the solutions by means of two distinct algorithms, and we analyze them by applying two different approaches. Our analysis of both algorithms is quite involved, but we devise them mostly by simplifying, combining, and ameliorating the known techniques, although we propose some technical novelties for further enhancing the performance of the popular Cross-Approximation Algorithms. They are highly efficient empirically, and we prove that they are efficient for the average input. We specify some narrow classes of hard inputs for which the presented algorithms fail with high probability even when we randomize them, but we narrow such classes further by means of preprocessing with new sparse and structured multipliers. The average complexity estimates do not cover many realistic input classes, but our formal analysis is in good accordance with the results of our tests applied to benchmark inputs from discretized PDEs and Integral quations and to random inputs. Our work should already be of practical value but also leads to research challenges. At the end we list some of them, propose two novel extensions of our progress -- to the acceleration of the Fast Multipole Method and Conjugate Gradient algorithms, and explore and slightly extend the recent techniques of Osinsky, which enhance the output accuracy of CUR Approximation.

math.NA↗
arXiv2018

Implementation of a Near-Optimal Complex Root Clustering Algorithm

We describe Ccluster, a software for computing natural $ε$-clusters of complex roots in a given box of the complex plane. This algorithm from Becker et al.~(2016) is near-optimal when applied to the benchmark problem of isolating all complex roots of an integer polynomial. It is one of the first implementations of a near-optimal algorithm for complex roots. We describe some low level techniques for speeding up the algorithm. Its performance is compared with the well-known MPSolve library and Maple.

cs.MS↗
arXiv2018

Fast Feasible and Unfeasible Matrix Multiplication

Fast matrix-by-matrix multiplication (hereafter MM) is a highly recognized research subject. The record upper bound 3 of 1968 on the exponent of the complexity MM decreased below 2.38 by 1987, applies to celebrated problems in many areas of computing, and is extensively cited in the Theory of Computing. Further decrease of the exponent remains a celebrated challenge. Acceleration of MM in the Practice of Computing is a distinct challenge, because all known algorithms supporting the exponents below 2.7733 improve straightforward MM only for unfeasible MM of immense size, greatly exceeding the sizes of interest nowadays and in any foreseeable future. We first survey the mainstream study of the acceleration of MM of unbounded sizes, cover the progress in decreasing the exponents of MM, comment on its impact on the theory and practice of computing, and recall various fundamental concepts and techniques supporting fast MM and naturally introduced in that study by 1980. Then we demonstrate how the curse of recursion naturally entered the game of decreasing the record exponents. Finally we cover the State of the Art of efficient feasible MM, including some most efficient known techniques and algorithms as well as various issues of numerical and symbolic implementation. We hope that our review will help motivate and properly focus further effort in this highly important area.

cs.DS↗
arXiv2018

Superfast CUR Matrix Algorithms, Their Pre-Processing and Extensions

We study superfast algorithms that computes low rank approximation of a matrix (hereafter referred to as LRA) that use much fewer memory cells and arithmetic operations than the input matrix has entries. We first specify a family of 2mn matrices of size m*n such that for almost 50% of them any superfast LRA algorithm fails to improve the poor trivial approximation by the matrix filled with zeros, but then we prove that the class of all such hard inputs is narrow - the cross-approximation (hereafter {C-A}) superfast iterations as well as some more primitive superfast algorithms compute reasonably accurate LRAs in their transparent CUR form (i) to any matrix allowing close LRA except for small norm perturbations of matrices of an algebraic variety of a smaller dimension, (ii) to the average matrix allowing close LRA, (iii) to the average sparse matrix allowing close LRA and (iv) with a high probability to any matrix allowing close LRA if it is pre-processed fast with a random Gaussian, SRHT or SRFT multiplier. Moreover empirically the output LRAs remain accurate when we perform the computations superfast by replacing such a multiplier with one of our sparse and structured multipliers. Our techniques, auxiliary results and extensions may be of some independent interest. We analyze C-A and other superfast algorithms twice -- based on two well-known sufficient criteria for obtaining accurate LRAs. We provide a distinct proof in the case of superfast variant of randomized algorithms of [DMM08], improve a decade-old estimate for the norm of the inverse of a Gaussian matrix, prove such an estimate also in the case of a sparse Gaussian matrix, present some novel advanced pre-processing techniques for fast and superfast computation of LRA, and extend our results to dramatic acceleration of the Fast Multipole Method (FMM) and the Conjugate Gradient algorithms.

math.NA↗
arXiv2017

Fast Approximate Computations with Cauchy Matrices and Polynomials

Multipoint polynomial evaluation and interpolation are fundamental for modern symbolic and numerical computing. The known algorithms solve both problems over any field of constants in nearly linear arithmetic time, but the cost grows to quadratic for numerical solution. We fix this discrepancy: our new numerical algorithms run in nearly linear arithmetic time. At first we restate our goals as the multiplication of an n-by-n Vandermonde matrix by a vector and the solution of a Vandermonde linear system of n equations. Then we transform the matrix into a Cauchy structured matrix with some special features. By exploiting them, we approximate the matrix by a generalized hierarchically semiseparable matrix, which is a structured matrix of a different class. Finally we accelerate our solution to the original problems by applying Fast Multipole Method to the latter matrix. Our resulting numerical algorithms run in nearly optimal arithmetic time when they perform the above fundamental computations with polynomials, Vandermonde matrices, transposed Vandermonde matrices, and a large class of Cauchy and Cauchy-like matrices. Some of our techniques may be of independent interest.

math.NA↗
arXiv2017

Real Polynomial Root-finding by Means of Matrix and Polynomial Iterations

Univariate polynomial root-finding is a classical subject, still important for modern computing. Frequently one seeks just the real roots of a polynomial with real coefficients. They can be approximated at a low computational cost if the polynomial has no nonreal roots, but for high degree polynomials, nonreal roots are typically much more numerous than the real ones. The challenge is known for a long time, and the subject has been intensively studied. Nevertheless, we produce some novel ideas and techniques and obtain dramatic acceleration of the known algorithms. In order to achieve our progress we exploit the correlation between the computations with matrices and polynomials, randomized matrix computations, and complex plane geometry, extend the techniques of the matrix sign iterations, and use the structure of the companion matrix of the input polynomial. The results of our extensive tests with benchmark polynomials and random matrices are quite encouraging. In particular in our tests the number of iterations required for convergence of our algorithms grew very slowly (if at all) as we increased the degree of the univariate input polynomials and the dimension of the input matrices from 64 to 1024.

cs.SC↗
arXiv2017

Enhancing the Power of Cardinal's Algorithm

Cardinal's factorization algorithm of 1996 splits a univariate polynomial into two factors with root sets separated by the imaginary axis, which is an important goal itself and a basic step toward root-finding. The novelty of the algorithm and its potential power have been well recognized by experts immediately, but by 2016, that is, two decades later, its practical value still remains nil, particularly because of the high computational cost of performing its final stage by means of computing approximate greatest common divisor of two polynomials. We briefly recall Cardinal's algorithm and its difficulties, amend it based on some works performed since 1996, extend its power to splitting out factors of a more general class, and reduce the final stage of the algorithm to quite manageable computations with structured matrices. Some of our techniques can be of independent interest for matrix computations.

math.NA↗
arXiv2016

New Results on CGR/CUR Approximation of a Matrix

CUR and low-rank approximations are among most fundamental subjects of numerical linear algebra, with a wide range of applications to a variety of highly important areas of modern computing, which range from the machine learning theory and neural networks to data mining and analysis. We first dramatically accelerate computation of such approximations for the average input matrix, then show some narrow classes of hard inputs for our algorithms, and finally point out a tentative direction to narrowing such classes further by means of pre-processing with quasi Gaussian structured multipliers. Our extensive numerical tests with a variety of real world inputs for regularization from Singular Matrix Database have consistently produced reasonably close CUR approximations at a low computational cost. There is a variety of efficient applications of our results and our techniques to important subjects of matrix computations. Our study provides new insights and enable design of faster algorithms for low-rank approximation by means of sampling and oversampling, for Gaussian elimination with no pivoting and block Gaussian elimination, and for the approximation of railing singular spaces associated with the $ν$ smallest singular values of a matrix having numerical nullity. We conclude the paper with novel extensions of our acceleration to the Fast Multipole Method and the Conjugate Gradient Algorithms. (This report is an outline of our paper in arXiv:1611.01391.)

math.NA↗
arXiv2016

Accelerated Approximation of the Complex Roots and Factors of a Univariate Polynomial

The algorithms of Pan (1995) and(2002) approximate the roots of a complex univariate polynomial in nearly optimal arithmetic and Boolean time but require precision of computing that exceeds the degree of the polynomial. This causes numerical stability problems when the degree is large. We observe, however, that such a difficulty disappears at the initial stage of the algorithms, and in our present paper we extend this stage to root-finding within a nearly optimal arithmetic and Boolean complexity bounds provided that some mild initial isolation of the roots of the input polynomial has been ensured. Furthermore our algorithm is nearly optimal for the approximation of the roots isolated in a fixed disc, square or another region on the complex plane rather than all complex roots of a polynomial. Moreover the algorithm can be applied to a polynomial given by a black box for its evaluation (even if its coefficients are not known); it promises to be of practical value for polynomial root-finding and factorization, the latter task being of interest on its own right. We also provide a new support for a winding number algorithm, which enables extension of our progress to obtaining mild initial approximations to the roots. We conclude with summarizing our algorithms and their extension to the approximation of isolated multiple roots and root clusters.

cs.SC↗
arXiv2016

Low-rank Approximation of a Matrix: Novel Insights, New Progress, and Extensions

Low-rank approximation of a matrix by means of random sampling has been consistently efficient in its empirical studies by many scientists who applied it with various sparse and structured multipliers, but adequate formal support for this empirical phenomenon has been missing so far. Our novel insight into the subject leads to such an elusive formal support and promises significant acceleration of the known algorithms for some fundamental problems of matrix computations and data mining and analysis. Our formal results and our numerical tests are in good accordance with each other. We also outline extensions of low-rank approximation algorithms and of our progress to the Least Squares Regression, the Fast Multipole Method, and the Conjugate Gradient algorithms.

math.NA↗
arXiv2016

New Studies of Randomized Augmentation and Additive Preprocessing

1. A standard Gaussian random matrix has full rank with probability 1 and is well-conditioned with a probability quite close to 1 and converging to 1 fast as the matrix deviates from square shape and becomes more rectangular. 2. If we append sufficiently many standard Gaussian random rows or columns to any normalized matrix A, then the augmented matrix has full rank with probability 1 and is well-conditioned with a probability close to 1, even if the matrix A is rank deficient or ill-conditioned. 3. We specify and prove these properties of augmentation and extend them to additive preprocessing, that is, to adding a product of two rectangular Gaussian matrices. 4. By applying our randomization techniques to a matrix that has numerical rank r, we accelerate the known algorithms for the approximation of its leading and trailing singular spaces associated with its r largest and with all its remaining singular values, respectively. 5. Our algorithms use much fewer random parameters and run much faster when various random sparse and structured preprocessors replace Gaussian. Empirically the outputs of the resulting algorithms is as accurate as the outputs under Gaussian preprocessing. 6. Our novel duality techniques provides formal support, so far missing, for these empirical observations and opens door to derandomization of our preprocessing and to further acceleration and simplification of our algorithms by using more efficient sparse and structured preprocessors. 7. Our techniques and our progress can be applied to various other fundamental matrix computations such as the celebrated low-rank approximation of a matrix by means of random sampling.

math.NA↗
arXiv2015

How Bad Are Vandermonde Matrices?

The work on the estimation of the condition numbers of Vandermonde matrices, motivated by applications to interpolation and quadrature, can be traced back at least to the 1970s. Empirical study has shown consistently that Vandermonde matrices tend to be badly ill-conditioned, with a narrow class of notable exceptions, such as the matrices of the discrete Fourier transform (hereafter referred to as DFT). So far formal support for this empirical observation, however, has been limited to the matrices defined by the real set of knots. We prove that, more generally, any Vandermonde matrix of a large size is badly ill-conditioned unless its knots are more or less equally spaced on or about the circle $C(0,1)=\{x:\,|x|=1\}$. The matrices of DFT are perfectly conditioned, being defined by a cyclic sequence of knots, equally spaced on that circle, but we prove that even a slight modification of the knots into the so called quasi-cyclic sequence on this circle defines badly ill-conditioned Vandermonde matrices. Likewise we prove that the half-size leading block of a large DFT matrix is badly ill-conditioned. (This result was motivated by an application to pre-conditioning of an input matrix for Gaussian elimination with no pivoting.) Our analysis involves the Ekkart--Young theorem, the Vandermonde-to-Cauchy transformation of matrix structure, our new inversion formula for a Cauchy matrix, and low-rank approximation of its large submatrices.

math.NA↗
arXiv2015

Polynomial Root Isolation by Means of Root Radii Approximation

Univariate polynomial root-finding is a classical subject, still important for modern computing. Frequently one seeks just the real roots of a real coefficient polynomial. They can be approximated at a low computational cost if the polynomial has no nonreal roots, but for high degree polynomials, nonreal roots are typically much more numerous than the real ones. The challenge is known for long time, and the subject has been intensively studied. The Boolean cost bounds for the refinement of the simple and isolated real roots have been decreased to nearly optimal, but the success has been more limited at the stage of the isolation of real roots. We obtain substantial progress by applying the algorithm of of 1982 by Schoenhage for the approximation of the root radii, that is, the distances of the roots to the origin. Namely we isolate the simple and well-conditioned real roots of a polynomial at the Boolean cost dominated by the nearly optimal bounds for the refinement of such roots. We also extend our algorithm to the isolation of complex, possibly multiple, roots and root clusters staying within the same (nearly optimal) asymptotic Boolean cost bound. Our numerical tests with benchmark polynomials performed with the IEEE standard double precision show that our nearly optimal real root-finder is practically promising. Our techniques are simple, and their power and application range may increase in combination with the known efficient methods.

math.NA↗
arXiv2015

Matrix Multiplication, Trilinear Decompositions, APA Algorithms, and Summation

Matrix multiplication (hereafter we use the acronym MM) is among the most fundamental operations of modern computations. The efficiency of its performance depends on various factors, in particular vectorization, data movement and arithmetic complexity of the computations, but here we focus just on the study of the arithmetic cost and the impact of this study on other areas of modern computing. In the early 1970s it was expected that the straightforward cubic time algorithm for MM will soon be accelerated to enable MM in nearly quadratic arithmetic time, with some far fetched implications. While pursuing this goal the mainstream research had its focus on the decrease of the classical exponent 3 of the complexity of MM towards its lower bound 2, disregarding the growth of the input size required to support this decrease. Eventually, surprising combinations of novel ideas and sophisticated techniques enabled the decrease of the exponent to its benchmark value of about 2.38, but the supporting MM algorithms improved the straightforward one only for the inputs of immense sizes. Meanwhile, the communication complexity, rather than the arithmetic complexity, has become the bottleneck of computations in linear algebra. This development may seem to undermine the value of the past and future research aimed at the decrease of the arithmetic cost of MM, but we feel that the study should be reassessed rather than closed and forgotten. We review the old and new work in this area in the present day context, recall some major techniques introduced in the study of MM, discuss their impact on the modern theory and practice of computations for MM and beyond MM, and link one of these techniques to some simple algorithms for inner product and summation.

cs.DS↗
arXiv2014

Random Multipliers Numerically Stabilize Gaussian and Block Gaussian Elimination: Proofs and an Extension to Low-rank Approximation

We prove that standard Gaussian random multipliers are expected to numerically stabilize both Gaussian elimination with no pivoting and block Gaussian elimination. Moreover we prove that such a multiplier (even without the customary oversampling) is expected to support low-rank approximation of a matrix. Our test results are in good accordance with this analysis. Empirically random circulant or Toeplitz multipliers are as efficient as Gaussian ones, but their formal support is more problematic.

math.NA↗
arXiv2014

Better Late Than Never: Filling a Void in the History of Fast Matrix Multiplication and Tensor Decompositions

Multilinear and tensor decompositions are a popular tool in linear and multilinear algebra and have a wide range of important applications to modern computing. Our paper of 1972 presented the first nontrivial application of such decompositions to fundamental matrix computations and was also a landmark in the history of the acceleration of matrix multiplication. Published in 1972 in Russian, it has never been translated into English. It has been very rarely cited in the Western literature on matrix multiplication and never in the works on multilinear and tensor decompositions. This motivates us to present its translation into English, together with our brief comments on its impact on the two fields.

math.NA↗
arXiv2014

Novel Approach to Real Polynomial Root-finding and Matrix Eigen-solving

Univariate polynomial root-finding is both classical and important for modern computing. Frequently one seeks just the real roots of a polynomial with real coefficients. They can be approximated at a low computational cost if the polynomial has no nonreal roots, but typically nonreal roots are much more numerous than the real ones. We dramatically accelerate the known algorithms in this case by exploiting the correlation between the computations with matrices and polynomials, extending the techniques of the matrix sign iteration, and exploiting the structure of the companion matrix of the input polynomial. We extend some of the proposed techniques to the approximation of the real eigenvalues of a real nonsymmetric matrix.

math.NA↗