arXiv subjects

Simon Foucart

Publications and source records attributed to Simon Foucart.

33 records · Page 2Linked to original sources

arXiv2020

Learning from Non-Random Data in Hilbert Spaces: An Optimal Recovery Perspective

The notion of generalization in classical Statistical Learning is often attached to the postulate that data points are independent and identically distributed (IID) random variables. While relevant in many applications, this postulate may not hold in general, encouraging the development of learning frameworks that are robust to non-IID data. In this work, we consider the regression problem from an Optimal Recovery perspective. Relying on a model assumption comparable to choosing a hypothesis class, a learner aims at minimizing the worst-case error, without recourse to any probabilistic assumption on the data. We first develop a semidefinite program for calculating the worst-case error of any recovery map in finite-dimensional Hilbert spaces. Then, for any Hilbert space, we show that Optimal Recovery provides a formula which is user-friendly from an algorithmic point-of-view, as long as the hypothesis class is linear. Interestingly, this formula coincides with kernel ridgeless regression in some cases, proving that minimizing the average error and worst-case error can yield the same solution. We provide numerical experiments in support of our theoretical findings.

cs.LG↗

arXiv2020

Instances of Computational Optimal Recovery: Dealing with Observation Errors

When attempting to recover functions from observational data, one naturally seeks to do so in an optimal manner with respect to some modeling assumption. With a focus put on the worst-case setting, this is the standard goal of Optimal Recovery. The distinctive twists here are the consideration of inaccurate data through some boundedness models and the emphasis on computational realizability. Several scenarios are unraveled through the efficient constructions of optimal recovery maps: local optimality under linearly or semidefinitely describable models, global optimality for the estimation of linear functionals under approximability models, and global near-optimality under approximability models in the space of continuous functions.

math.OC↗

arXiv2020

Instances of Computational Optimal Recovery: Refined Approximability Models

Models based on approximation capabilities have recently been studied in the context of Optimal Recovery. These models, however, are not compatible with overparametrization, since model- and data-consistent functions could then be unbounded. This drawback motivates the introduction of refined approximability models featuring an added boundedness condition. Thus, two new models are proposed in this article: one where the boundedness applies to the target functions (first type) and one where the boundedness applies to the approximants (second type). For both types of model, optimal maps for the recovery of linear functionals are first described on an abstract level before their efficient constructions are addressed. By exploiting techniques from semidefinite programming, these constructions are explicitly carried out on a common example involving polynomial subspaces of $\mathcal{C}[-1,1]$.

math.OC↗

arXiv2020

Finer Metagenomic Reconstruction via Biodiversity Optimization

When analyzing communities of microorganisms from their sequenced DNA, an important task is taxonomic profiling: enumerating the presence and relative abundance of all organisms, or merely of all taxa, contained in the sample. This task can be tackled via compressive-sensing-based approaches, which favor communities featuring the fewest organisms among those consistent with the observed DNA data. Despite their successes, these parsimonious approaches sometimes conflict with biological realism by overlooking organism similarities. Here, we leverage a recently developed notion of biological diversity that simultaneously accounts for organism similarities and retains the optimization strategy underlying compressive-sensing-based approaches. We demonstrate that minimizing biological diversity still produces sparse taxonomic profiles and we experimentally validate superiority to existing compressive-sensing-based approaches. Despite showing that the objective function is almost never convex and often concave, generally yielding NP-hard problems, we exhibit ways of representing organism similarities for which minimizing diversity can be performed via a sequence of linear programs guaranteed to decrease diversity. Better yet, when biological similarity is quantified by $k$-mer co-occurrence (a popular notion in bioinformatics), minimizing diversity actually reduces to one linear program that can utilize multiple $k$-mer sizes to enhance performance. In proof-of-concept experiments, we verify that the latter procedure can lead to significant gains when taxonomically profiling a metagenomic sample, both in terms of reconstruction accuracy and computational performance. Reproducible code is available at https://github.com/dkoslicki/MinimizeBiologicalDiversity.

q-bio.GN↗

arXiv2019

Weighted matrix completion from non-random, non-uniform sampling patterns

We study the matrix completion problem when the observation pattern is deterministic and possibly non-uniform. We propose a simple and efficient debiased projection scheme for recovery from noisy observations and analyze the error under a suitable weighted metric. We introduce a simple function of the weight matrix and the sampling pattern that governs the accuracy of the recovered matrix. We derive theoretical guarantees that upper bound the recovery error and nearly matching lower bounds that showcase optimality in several regimes. Our numerical experiments demonstrate the computational efficiency and accuracy of our approach, and show that debiasing is essential when using non-uniform sampling patterns.

cs.IT↗

arXiv2019

Jointly Low-Rank and Bisparse Recovery: Questions and Partial Answers

We investigate the problem of recovering jointly $r$-rank and $s$-bisparse matrices from as few linear measurements as possible, considering arbitrary measurements as well as rank-one measurements. In both cases, we show that $m \asymp r s \ln(en/s)$ measurements make the recovery possible in theory, meaning via a nonpractical algorithm. In case of arbitrary measurements, we investigate the possibility of achieving practical recovery via an iterative-hard-thresholding algorithm when $m \asymp r s^γ\ln(en/s)$ for some exponent $γ> 0$. We show that this is feasible for $γ= 2$, and that the proposed analysis cannot cover the case $γ\leq 1$. The precise value of the optimal exponent $γ\in [1,2]$ is the object of a question, raised but unresolved in this paper, about head projections for the jointly low-rank and bisparse structure. Some related questions are partially answered in passing. For rank-one measurements, we suggest on arcane grounds an iterative-hard-thresholding algorithm modified to exploit the nonstandard restricted isometry property obeyed by this type of measurements.

math.NA↗

arXiv2019

Approximability Models and Optimal System Identification

This article considers the problem of optimally recovering stable linear time-invariant systems observed via linear measurements made on their transfer functions. A common modeling assumption is replaced here by the related assumption that the transfer functions belong to a model set described by approximation capabilities. Capitalizing on recent optimal-recovery results relative to such approximability models, we construct some optimal algorithms and characterize the optimal performance for the identification and evaluation of transfer functions in the framework of the Hardy Hilbert space and of the disc algebra. In particular, we determine explicitly the optimal recovery performance for frequency measurements taken at equispaced points on an inner circle or on the torus.

math.OC↗

arXiv2019

Computation of Chebyshev Polynomials for Union of Intervals

Chebyshev polynomials of the first and second kind for a set K are monic polynomials with minimal L $\infty$-and L 1-norm on K, respectively. This articles presents numerical procedures based on semidefinite programming to compute these polynomials in case K is a finite union of compact intervals. For Chebyshev polynomials of the first kind, the procedure makes use of a characterization of polynomial nonnegativity. It can incorporate additional constraints, e.g. that all the roots of the polynomial lie in K. For Chebyshev polynomials of the second kind, the procedure exploits the method of moments. Key words and phrases: Chebyshev polynomials of the first kind, Chebyshev polynomials of the second kind, nonnegative polynomials, method of moments, semidefinite programming.

math.OC↗

arXiv2019

One-Bit Sensing of Low-Rank and Bisparse Matrices

This note studies the worst-case recovery error of low-rank and bisparse matrices as a function of the number of one-bit measurements used to acquire them. First, by way of the concept of consistency width, precise estimates are given on how fast the recovery error can in theory decay. Next, an idealized recovery method is proved to reach the fourth-root of the optimal decay rate for Gaussian sensing schemes. This idealized method being impractical, an implementable recovery algorithm is finally proposed in the context of factorized Gaussian sensing schemes. It is shown to provide a recovery error decaying as the sixth-root of the optimal rate.

cs.IT↗

arXiv2018

Iterative Hard Thresholding for Low-Rank Recovery from Rank-One Projections

A novel algorithm for the recovery of low-rank matrices acquired via compressive linear measurements is proposed and analyzed. The algorithm, a variation on the iterative hard thresholding algorithm for low-rank recovery, is designed to succeed in situations where the standard rank-restricted isometry property fails, e.g. in case of subexponential unstructured measurements or of subgaussian rank-one measurements. The stability and robustness of the algorithm are established based on distinctive matrix-analytic ingredients and its performance is substantiated numerically.

math.NA↗

arXiv2018

Determining Projection Constants of Univariate Polynomial Spaces

The long-standing problem of minimal projections is addressed from a computational point of view. Techniques to determine bounds on the projection constants of univariate polynomial spaces are presented. The upper bound, produced by a linear program, and the lower bound, produced by a semidefinite program exploiting the method of moments, are often close enough to deduce the projection constant with reasonable accuracy. The implementation of these programs makes it possible to find the projection constant of several three-dimensional spaces with five digits of accuracy, as well as the projection constants of the spaces of cubic, quartic, and quintic polynomials with four digits of accuracy. Beliefs about uniqueness and shape-preservation of minimal projections are contested along the way.

math.NA↗

arXiv2016

One-Bit Compressive Sensing of Dictionary-Sparse Signals

One-bit compressive sensing has extended the scope of sparse recovery by showing that sparse signals can be accurately reconstructed even when their linear measurements are subject to the extreme quantization scenario of binary samples---only the sign of each linear measurement is maintained. Existing results in one-bit compressive sensing rely on the assumption that the signals of interest are sparse in some fixed orthonormal basis. However, in most practical applications, signals are sparse with respect to an overcomplete dictionary, rather than a basis. There has already been a surge of activity to obtain recovery guarantees under such a generalized sparsity model in the classical compressive sensing setting. Here, we extend the one-bit framework to this important model, providing a unified theory of one-bit compressive sensing under dictionary sparsity. Specifically, we analyze several different algorithms---based on convex programming and on hard thresholding---and show that, under natural assumptions on the sensing matrix (satisfied by Gaussian matrices), these algorithms can efficiently recover analysis-dictionary-sparse signals in the one-bit model.

cs.IT↗

arXiv2014

Exponential decay of reconstruction error from binary measurements of sparse signals

Binary measurements arise naturally in a variety of statistical and engineering applications. They may be inherent to the problem---e.g., in determining the relationship between genetics and the presence or absence of a disease---or they may be a result of extreme quantization. In one-bit compressed sensing it has recently been shown that the number of one-bit measurements required for signal estimation mirrors that of unquantized compressed sensing. Indeed, $s$-sparse signals in $\mathbb{R}^n$ can be estimated (up to normalization) from $Ω(s \log (n/s))$ one-bit measurements. Nevertheless, controlling the precise accuracy of the error estimate remains an open challenge. In this paper, we focus on optimizing the decay of the error as a function of the oversampling factor $λ:= m/(s \log(n/s))$, where $m$ is the number of measurements. It is known that the error in reconstructing sparse signals from standard one-bit measurements is bounded below by $Ω(λ^{-1})$. Without adjusting the measurement procedure, reducing this polynomial error decay rate is impossible. However, we show that an adaptive choice of the thresholds used for quantization may lower the error rate to $e^{-Ω(λ)}$. This improves upon guarantees for other methods of adaptive thresholding as proposed in Sigma-Delta quantization. We develop a general recursive strategy to achieve this exponential decay and two specific polynomial-time algorithms which fall into this framework, one based on convex programming and one on hard thresholding. This work is inspired by the one-bit compressed sensing model, in which the engineer controls the measurement procedure. Nevertheless, the principle is extendable to signal reconstruction problems in a variety of binary statistical models as well as statistical estimation problems like logistic regression.

cs.IT↗

arXiv2013

On the Hermite spline conjecture and its connection to k-monotone densities

The k-monotone classes of densities defined on (0, \infty) have been known in the mathematical literature but were for the first time considered from a statistical point of view by Balabdaoui and Wellner (2007, 2010). In these works, the authors generalized the results established for monotone (k=1) and convex (k=2) densities by giving a characterization of the Maximum Likelihood and Least Square estimators (MLE and LSE) and deriving minimax bounds for rates of convergence. For k strictly larger than 2, the pointwise asymptotic behavior of the MLE and LSE studied by Balabdaoui and Wellner (2007) would show that the MLE and LSE attain the minimax lower bounds in a local pointwise sense. However, the theory assumes that a certain conjecture about the approximation error of a Hermite spline holds true. The main goal of the present note is to show why such a conjecture cannot be true. We also suggest how to bypass the conjecture and rebuild the key proofs in the limit theory of the estimators.

math.ST↗

arXiv2010

The Gelfand widths of $\ell_p$-balls for $0<p\leq 1$

We provide sharp lower and upper bounds for the Gelfand widths of $\ell_p$-balls in the $N$-dimensional $\ell_q^N$-space for $0<p\leq 1$ and $p<q \leq 2$. Such estimates are highly relevant to the novel theory of compressive sensing, and our proofs rely on methods from this area.

math.FA↗