arXiv subjects

Felipe Cucker

Publications and source records attributed to Felipe Cucker.

At least 19 recordsLinked to original sources

arXiv2023

When can you trust feature selection? -- I: A condition-based analysis of LASSO and generalised hardness of approximation

The arrival of AI techniques in computations, with the potential for hallucinations and non-robustness, has made trustworthiness of algorithms a focal point. However, trustworthiness of the many classical approaches are not well understood. This is the case for feature selection, a classical problem in the sciences, statistics, machine learning etc. Here, the LASSO optimisation problem is standard. Despite its widespread use, it has not been established when the output of algorithms attempting to compute support sets of minimisers of LASSO in order to do feature selection can be trusted. In this paper we establish how no (randomised) algorithm that works on all inputs can determine the correct support sets (with probability $> 1/2$) of minimisers of LASSO when reading approximate input, regardless of precision and computing power. However, we define a LASSO condition number and design an efficient algorithm for computing these support sets provided the input data is well-posed (has finite condition number) in time polynomial in the dimensions and logarithm of the condition number. For ill-posed inputs the algorithm runs forever, hence, it will never produce a wrong answer. Furthermore, the algorithm computes an upper bound for the condition number when this is finite. Finally, for any algorithm defined on an open set containing a point with infinite condition number, there is an input for which the algorithm will either run forever or produce a wrong answer. Our impossibility results stem from generalised hardness of approximation -- within the Solvability Complexity Index (SCI) hierarchy framework -- that generalises the classical phenomenon of hardness of approximation.

math.OC↗

arXiv2023

When can you trust feature selection? -- II: On the effects of random data on condition in statistics and optimisation

In Part I, we defined a LASSO condition number and developed an algorithm -- for computing support sets (feature selection) of the LASSO minimisation problem -- that runs in polynomial time in the number of variables and the logarithm of the condition number. The algorithm is trustworthy in the sense that if the condition number is infinite, the algorithm will run forever and never produce an incorrect output. In this Part II article, we demonstrate how finite precision algorithms (for example algorithms running floating point arithmetic) will fail on open sets when the condition number is large -- but still finite. This augments Part I's result: If an algorithm takes inputs from an open set that includes at least one point with an infinite condition number, it fails to compute the correct support set for all inputs within that set. Hence, for any finite precision algorithm working on open sets for the LASSO problem with random inputs, our LASSO condition number -- as a random variable -- will estimate the probability of success/failure of the algorithm. We show that a finite precision version of our algorithm works on traditional Gaussian data for LASSO with high probability. The algorithm is trustworthy, specifically, in the random cases where the algorithm fails, it will not produce an output. Finally, we demonstrate classical random ensembles for which the condition number will be large with high probability, and hence where any finite precision algorithm on open sets will fail. We show numerically how commercial software fails on these cases.

math.OC↗

arXiv2021

Functional norms, condition numbers and numerical algorithms in algebraic geometry

In numerical linear algebra, a well-established practice is to choose a norm that exploits the structure of the problem at hand in order to optimize accuracy or computational complexity. In numerical polynomial algebra, a single norm (attributed to Weyl) dominates the literature. This article initiates the use of $L_p$ norms for numerical algebraic geometry, with an emphasis on $L_{\infty}$. This classical idea yields strong improvements in the analysis of the number of steps performed by numerous iterative algorithms. In particular, we exhibit three algorithms where, despite the complexity of computing $L_{\infty}$-norm, the use of $L_p$-norms substantially reduces computational complexity: a subdivision-based algorithm in real algebraic geometry for computing the homology of semialgebraic sets, a well-known meshing algorithm in computational geometry, and the computation of zeros of systems of complex quadratic polynomials (a particular case of Smale's 17th problem).

math.NA↗

arXiv2020

Rigid continuation paths II. Structured polynomial systems

This work studies the average complexity of solving structured polynomial systems that are characterized by a low evaluation cost, as opposed to the dense random model previously used. Firstly, we design a continuation algorithm that computes, with high probability, an approximate zero of a polynomial system given only as black-box evaluation program. Secondly, we introduce a universal model of random polynomial systems with prescribed evaluation complexity L. Combining both, we show that we can compute an approximate zero of a random structured polynomial system with n equations of degree at most {\delta} in n variables with only poly(n, {\delta}) L operations with high probability. This exceeds the expectations implicit in Smale's 17th problem.

math.NA↗

arXiv2020

On the Complexity of the Plantinga-Vegter Algorithm

We introduce tools from numerical analysis and high dimensional probability for precision control and complexity analysis of subdivision-based algorithms in computational geometry. We combine these tools with the continuous amortization framework from exact computation. We use these tools on a well-known example from the subdivision family: the adaptive subdivision algorithm due to Plantinga and Vegter. The only existing complexity estimate on this rather fast algorithm was an exponential worst-case upper bound for its interval arithmetic version. We go beyond the worst-case by considering both average and smoothed analysis, and prove polynomial time complexity estimates for both interval arithmetic and finite-precision versions of the Plantinga-Vegter algorithm.

cs.CG↗

arXiv2019

On local analysis

We extend to Gaussian distributions a result providing smoothed analysis estimates for condition numbers given as relativized distances to illposedness. We also introduce a notion of local analysis meant to capture the behavior of these condition numbers around a point.

math.NA↗

arXiv2019

Computing the Homology of Semialgebraic Sets. II: General formulas

We describe and analyze a numerical algorithm for computing the homology (Betti numbers and torsion coefficients) of semialgebraic sets given by Boolean formulas. The algorithm works in weak exponential time. This means that outside a subset of data having exponentially small measure, the cost of the algorithm is single exponential in the size of the data. This extends the previous work of the authors in arXiv:1807.06435 to arbitrary semialgebraic sets. All previous algorithms proposed for this problem have doubly exponential complexity.

cs.CG↗

arXiv2019

Plantinga-Vegter algorithm takes average polynomial time

We exhibit a condition-based analysis of the adaptive subdivision algorithm due to Plantinga and Vegter. The first complexity analysis of the PV Algorithm is due to Burr, Gao and Tsigaridas who proved a $O\big(2^{\tau d^{4}\log d}\big)$ worst-case cost bound for degree $d$ plane curves with maximum coefficient bit-size $\tau$. This exponential bound, it was observed, is in stark contrast with the good performance of the algorithm in practice. More in line with this performance, we show that, with respect to a broad family of measures, the expected time complexity of the PV Algorithm is bounded by $O(d^7)$ for real, degree $d$, plane curves. We also exhibit a smoothed analysis of the PV Algorithm that yields similar complexity estimates. To obtain these results we combine robust probabilistic techniques coming from geometric functional analysis with condition numbers and the continuous amortization paradigm introduced by Burr, Krahmer and Yap. We hope this will motivate a fruitful exchange of ideas between the different approaches to numerical computation.

cs.CG↗

arXiv2018

Computing the Homology of Semialgebraic Sets I: Lax Formulas

We describe and analyze an algorithm for computing the homology (Betti numbers and torsion coefficients) of closed semialgebraic sets given by Boolean formulas without negations over lax polynomial inequalities. The algorithm works in weak exponential time. This means that outside a subset of data having exponentially small measure, the cost of the algorithm is single exponential in the size of the data. All previous algorithms solving this problem have doubly exponential complexity (and this is so for almost all input data). Our algorithm thus represents an exponential acceleration over state-of-the-art algorithms for all input data outside a set that vanishes exponentially fast.

cs.CG↗

arXiv2017

Computing the homology of basic semialgebraic sets in weak exponential time

We describe and analyze an algorithm for computing the homology (Betti numbers and torsion coefficients) of basic semialgebraic sets which works in weak exponential time. That is, out of a set of exponentially small measure in the space of data the cost of the algorithm is exponential in the size of the data. All algorithms previously proposed for this problem have a complexity which is doubly exponential (and this is so for almost all data).

cs.CG↗

arXiv2016

Computing the Homology of Real Projective Sets

We describe and analyze a numerical algorithm for computing the homology (Betti numbers and torsion coefficients) of real projective varieties. Here numerical means that the algorithm is numerically stable (in a sense to be made precise). Its cost depends on the condition of the input as well as on its size and is singly exponential in the number of variables (the dimension of the ambient space) and polynomial in the condition and the degrees of the defining polynomials. In addition, we show that outside of an exceptional set of measure exponentially small in the size of the data, the algorithm takes exponential time.

math.AG↗

arXiv2015

On the condition of characteristic polynomials

We prove that the expectation of the logarithm of the condition number of each of the zeros of the characteristic polynomial of a complex standard Gaussian matrix is ${\Omega}(n)$. This may provide an explanation for the common wisdom in numerical linear algebra that advises against computing eigenvalues via root-finding for characteristic polynomials.

math.NA↗

arXiv2015

Condition length and complexity for the solution of polynomial systems

Smale's 17th problem asks for an algorithm which finds an approximate zero of polynomial systems in average polynomial time (see Smale 2000). The main progress on Smale's problem is Beltr\'an-Pardo (2011) and B\"urgisser-Cucker (2010). In this paper we will improve on both approaches and we prove an important intermediate result. Our main results are Theorem 1 on the complexity of a randomized algorithm which improves the result of Beltr\'an-Pardo (2011), Theorem 2 on the average of the condition number of polynomial systems which improves the estimate found in B\"urgisser-Cucker (2010), and Theorem 3 on the complexity of finding a single zero of polynomial systems. This last Theorem is the main result of B\"urgisser-Cucker (2010). We give a proof of it relying only on homotopy methods, thus removing the need for the elimination theory methods used in B\"urgisser-Cucker (2010). We build on methods developed in Armentano et al. (2015).

math.NA↗

arXiv2015

A stable, polynomial-time algorithm for the eigenpair problem

We describe algorithms for computing eigenpairs (eigenvalue-eigenvector pairs) of a complex $n\times n$ matrix $A$. These algorithms are numerically stable, strongly accurate, and theoretically efficient (i.e., polynomial-time). We do not believe they outperform in practice the algorithms currently used for this computational problem. The merit of our paper is to give a positive answer to a long-standing open problem in numerical linear algebra.

math.NA↗

arXiv2014

A stable, polynomial-time algorithm for the eigenpair problem

We describe algorithms for computing eigenpairs (eigenvalue--eigenvector) of a complex $n\times n$ matrix $A$. These algorithms are numerically stable, strongly accurate, and theoretically efficient (i.e., polynomial-time). We do not believe they outperform in practice the algorithms currently used for this computational problem. The merit of our paper is to give a positive answer to a long-standing open problem in numerical linear algebra.

math.NA↗

arXiv2014

A Theory of Complexity, Condition and Roundoff

We develop a theory of complexity for numerical computations that takes into account the condition of the input data and allows for roundoff in the computations. We follow the lines of the theory developed by Blum, Shub, and Smale for computations over R (which in turn followed those of the classical, discrete, complexity theory as laid down by Cook, Karp, and Levin among others). In particular, we focus on complexity classes of decision problems and paramount among them, on appropriate versions of the classes P, NP and EXP of polynomial, nondeterministic polynomial, and exponential time, respectively. We prove some basic relationships between these complexity classes and exhibit natural NP-complete problems.

cs.CC↗

arXiv2013

Smoothed analysis of componentwise condition numbers for sparse matrices

We perform a smoothed analysis of the componentwise condition numbers for determinant computation, matrix inversion, and linear equations solving for sparse n times n matrices. The bounds we obtain for the ex- pectations of the logarithm of these condition numbers are, in all three cases, of the order O(log n). As a consequence, small bounds on the smoothed loss of accuracy for triangular linear systems follow.

math.NA↗

arXiv2012

Fast Computation of Zeros of Polynomial Systems with Bounded Degree under Finite-precision

A solution for Smale's 17th problem, for the case of systems with bounded degree was recently given. This solution, an algorithm computing approximate zeros of complex polynomial systems in average polynomial time, assumed infinite precision. In this paper we describe a finite-precision version of this algorithm. Our main result shows that this version works within the same time bounds and requires a precision which, on the average, amounts to a polynomial amount of bits in the mantissa of the intervening floating-point numbers.

math.NA↗