Searcharxiv⌕ Search

arXiv subjects

Shahar Mendelson

Publications and source records attributed to Shahar Mendelson.

At least 37 records · Page 2Linked to original sources
arXiv2020

Approximating $L_p$ unit balls via random sampling

Let $X$ be an isotropic random vector in $R^d$ that satisfies that for every $v \in S^{d-1}$, $\| \|_{L_q} \leq L \| \|_{L_p}$ for some $q \geq 2p$. We show that for $0<\varepsilon<1$, a set of $N = c(p,q,\varepsilon) d$ random points, selected independently according to $X$, can be used to construct a $1 \pm \varepsilon$ approximation of the $L_p$ unit ball endowed on $R^d$ by $X$. Moreover, $c(p,q,\varepsilon) \leq c^p \varepsilon^{-2}\log(2/\varepsilon)$; when $q=2p$ the approximation is achieved with probability at least $1-2\exp(-cN \varepsilon^2/\log^2(2/\varepsilon))$ and if $q$ is much larger than $p$---say, $q=4p$, the approximation is achieved with probability at least $1-2\exp(-cN \varepsilon^2)$. In particular, when $X$ is a log-concave random vector, this estimate improves the previous state-of-the-art---that $N=c^\prime(p,\varepsilon) d^{p/2}\log d$ random points are enough, and that the approximation is valid with constant probability.

math.PR↗
arXiv2020

Extending the scope of the small-ball method

The small-ball method was introduced as a way of obtaining a high probability, isomorphic lower bound on the quadratic empirical process, under weak assumptions on the indexing class. The key assumption was that class members satisfy a uniform small-ball estimate: that $Pr(|f| \geq κ\|f\|_{L_2}) \geq δ$ for given constants $κ$ and $δ$. Here we extend the small-ball method and obtain a high probability, almost-isometric (rather than isomorphic) lower bound on the quadratic empirical process. The scope of the result is considerably wider than the small-ball method: there is no need for class members to satisfy a uniform small-ball condition, and moreover, motivated by the notion of tournament learning procedures, the result is stable under a `majority vote'.

stat.ML↗
arXiv2020

Learning bounded subsets of $L_p$

We study learning problems in which the underlying class is a bounded subset of $L_p$ and the target $Y$ belongs to $L_p$. Previously, minimax sample complexity estimates were known under such boundedness assumptions only when $p=\infty$. We present a sharp sample complexity estimate that holds for any $p > 4$. It is based on a learning procedure that is suited for heavy-tailed problems.

stat.ML↗
arXiv2019

On the geometry of polytopes generated by heavy-tailed random vectors

We study the geometry of centrally-symmetric random polytopes, generated by $N$ independent copies of a random vector $X$ taking values in $\mathbb{R}^n$. We show that under minimal assumptions on $X$, for $N \gtrsim n$ and with high probability, the polytope contains a deterministic set that is naturally associated with the random vector---namely, the polar of a certain floating body. This solves the long-standing question on whether such a random polytope contains a canonical body. Moreover, by identifying the floating bodies associated with various random vectors we recover the estimates that have been obtained previously, and thanks to the minimal assumptions on $X$ we derive estimates in cases that had been out of reach, involving random polytopes generated by heavy-tailed random vectors (e.g., when $X$ is $q$-stable or when $X$ has an unconditional structure). Finally, the structural results are used for the study of a fundamental question in compressive sensing---noise blind sparse recovery.

math.PR↗
arXiv2019

Mean estimation and regression under heavy-tailed distributions--a survey

We survey some of the recent advances in mean estimation and regression function estimation. In particular, we describe sub-Gaussian mean estimators for possibly heavy-tailed data both in the univariate and multivariate settings. We focus on estimators based on median-of-means techniques but other methods such as the trimmed mean and Catoni's estimator are also reviewed. We give detailed proofs for the cornerstone results. We dedicate a section on statistical learning problems--in particular, regression function estimation--in the presence of possibly heavy-tailed data.

math.ST↗
arXiv2019

Stable recovery and the coordinate small-ball behaviour of random vectors

Recovery procedures in various application in Data Science are based on \emph{stable point separation}. In its simplest form, stable point separation implies that if $f$ is "far away" from $0$, and one is given a random sample $(f(Z_i))_{i=1}^m$ where a proportional number of the sample points may be corrupted by noise, that information is still enough to exhibit that $f$ is far from $0$. Stable point separation is well understood in the context of iid sampling, and to explore it for general sampling methods we introduce a new notion---the \emph{coordinate small-ball} of a random vector $X$. Roughly put, this feature captures the number of "relatively large coordinates" of $(| |)_{i=1}^m$, where $T:\mathbb{R}^n \to \mathbb{R}^m$ is an arbitrary linear operator and $(u_i)_{i=1}^m$ is any fixed orthonormal basis of $\mathbb{R}^m$. We show that under the bare-minimum assumptions on $X$, and with high probability, many of the values $| |$ are at least of the order $\|T\|_{S_2}/\sqrt{m}$. As a result, the "coordinate structure" of $TX$ exhibits the typical Euclidean norm of $TX$ and does so in a stable way. One outcome of our analysis is that random sub-sampled convolutions satisfy stable point separation under minimal assumptions on the generating random vector---a fact that was known previously only in a highly restrictive setup, namely, for random vectors with iid subgaussian coordinates.

math.PR↗
arXiv2019

Robust covariance estimation under $L_4-L_2$ norm equivalence

Let $X$ be a centered random vector taking values in $\mathbb{R}^d$ and let $Σ= \mathbb{E}(X\otimes X)$ be its covariance matrix. We show that if $X$ satisfies an $L_4-L_2$ norm equivalence, there is a covariance estimator $\hatΣ$ that exhibits the optimal performance one would expect had $X$ been a gaussian vector. The procedure also improves the current state-of-the-art regarding high probability bounds in the subgaussian case (sharp results were only known in expectation or with constant probability). In both scenarios the new bound does not depend explicitly on the dimension $d$, but rather on the effective rank of the covariance matrix $Σ$.

math.ST↗
arXiv2019

On the geometry of random polytopes

We present a simple proof to a fact recently established in [5]: let $ξ$ be a symmetric random variable that has variance $1$, let $Γ=(ξ_{ij})$ be an $N \times n$ random matrix whose entries are independent copies of $ξ$, and set $X_1,...,X_N$ to be the rows of $Γ$. Then under minimal assumptions on $ξ$ and as long as $N \geq c_1n$, $$ c_2 \bigl(B_\infty^n \cap \sqrt{\log(eN/n)} B_2^n \bigr) \subset {\rm absconv}(X_1,...,X_N) $$ with high probability.

math.FA↗
arXiv2018

Robust one-bit compressed sensing with partial circulant matrices

We present optimal sample complexity estimates for one-bit compressed sensing problems in a realistic scenario: the procedure uses a structured matrix (a randomly sub-sampled circulant matrix) and is robust to analog pre-quantization noise as well as to adversarial bit corruptions in the quantization process. Our results imply that quantization is not a statistically expensive procedure in the presence of nontrivial analog noise: recovery requires the same sample size one would have needed had the measurement matrix been Gaussian and the noisy analog measurements been given as data.

cs.IT↗
arXiv2018

Concentration of the spectral norm of Erdős-Rényi random graphs

We present results on the concentration properties of the spectral norm $\|A_p\|$ of the adjacency matrix $A_p$ of an Erdős-Rényi random graph $G(n,p)$. First we consider the Erdős-Rényi random graph process and prove that $\|A_p\|$ is uniformly concentrated over the range $p\in [C\log n/n,1]$. The analysis is based on delocalization arguments, uniform laws of large numbers, together with the entropy method to prove concentration inequalities. As an application of our techniques we prove sharp sub-Gaussian moment inequalities for $\|A_p\|$ for all $p\in [c\log^3n/n,1]$ that improve the general bounds of Alon, Krivelevich, and Vu (2001) and some of the more recent results of Erdős et al. (2013). Both results are consistent with the asymptotic result of Füredi and Komlós (1981) that holds for fixed $p$ as $n\to \infty$.

math.PR↗
arXiv2018

Non-Gaussian Hyperplane Tessellations and Robust One-Bit Compressed Sensing

We show that a tessellation generated by a small number of random affine hyperplanes can be used to approximate Euclidean distances between any two points in an arbitrary bounded set $T$, where the random hyperplanes are generated by subgaussian or heavy-tailed normal vectors and uniformly distributed shifts. We derive quantitative bounds on the number of hyperplanes needed for constructing such tessellations in terms of natural metric complexity measures of $T$ and the desired approximation error. Our work extends significantly prior results in this direction, which were restricted to Gaussian hyperplane tessellations of subsets of the Euclidean unit sphere. As an application, we obtain new reconstruction results in memoryless one-bit compressed sensing with non-Gaussian measurement matrices. We show that by quantizing at uniformly distributed thresholds, it is possible to accurately reconstruct low-complexity signals from a small number of one-bit quantized measurements, even if the measurement vectors are drawn from a heavy-tailed distribution. Our reconstruction results are uniform in nature and robust in the presence of pre-quantization noise on the analog measurements as well as adversarial bit corruptions in the quantization process. Moreover we show that if the measurement matrix is subgaussian then accurate recovery can be achieved via a convex program.

cs.IT↗
arXiv2018

Near-optimal mean estimators with respect to general norms

We study the problem of estimating the mean of a random vector in $\mathbb{R}^d$ based on an i.i.d.\ sample, when the accuracy of the estimator is measured by a general norm on $\mathbb{R}^d$. We construct an estimator (that depends on the norm) that achieves an essentially optimal accuracy/confidence tradeoff under the only assumption that the random vector has a well-defined covariance matrix. The estimator is based on the construction of a uniform median-of-means estimator in a class of real valued functions that may be of independent interest.

math.ST↗
arXiv2018

Generalized Dual Sudakov Minoration via Dimension Reduction - A Program

We propose a program for establishing a conjectural extension to the class of (origin-symmetric) log-concave probability measures $μ$, of the classical dual Sudakov Minoration on the expectation of the supremum of a Gaussian process: \begin{equation} \label{eq:abstract} M(Z_p(μ), C \int ||x||_K dμ\cdot K) \leq \exp(C p) \;\;\, \forall p \geq 1 . \end{equation} Here $K$ is an origin-symmetric convex body, $Z_p(μ)$ is the $L_p$-centroid body associated to $μ$, $M(A,B)$ is the packing-number of $B$ in $A$, and $C > 0$ is a universal constant. The Program consists of first establishing a Weak Generalized Dual Sudakov Minoration, involving the dimension $n$ of the ambient space, which is then self-improved to a dimension-free estimate after applying a dimension-reduction step. The latter step may be thought of as a conjectural "small-ball one-sided" variant of the Johnson--Lindenstrauss dimension-reduction lemma. We establish the Weak Generalized Dual Sudakov Minoration for a variety of log-concave probability measures and convex bodies (for instance, this step is fully resolved assuming a positive answer to the Slicing Problem). The Separation Dimension-Reduction step is fully established for ellipsoids and, up to logarithmic factors in the dimension, for cubes, resulting in a corresponding Generalized (regular) Dual Sudakov Minoration estimate for these bodies and arbitrary log-concave measures, which are shown to be (essentially) best-possible. Along the way, we establish a regular version of (\ref{eq:abstract}) for all $p \geq n$ and provide a new direct proof of Sudakov Minoration via The Program.

math.FA↗
arXiv2018

Approximating the covariance ellipsoid

We explore ways in which the covariance ellipsoid ${\cal B}=\{v \in \mathbb{R}^d : \mathbb{E} ^2 \leq 1\}$ of a centred random vector $X$ in $\mathbb{R}^d$ can be approximated by a simple set. The data one is given for constructing the approximating set consists of $X_1,...,X_N$ that are independent and distributed as $X$. We present a general method that can be used to construct such approximations and implement it for two types of approximating sets. We first construct a (random) set ${\cal K}$ defined by a union of intersections of slabs $H_{z,α}=\{v \in \mathbb{R}^d : | | \leq α\}$ (and therefore ${\cal K}$ is actually the output of a neural network with two hidden layers). The slabs are generated using $X_1,...,X_N$, and under minimal assumptions on $X$ (e.g., $X$ can be heavy-tailed) it suffices that $N = c_1d η^{-4}\log(2/η)$ to ensure that $(1-η) {\cal K} \subset {\cal B} \subset (1+η){\cal K}$. In some cases (e.g., if $X$ is rotation invariant and has marginals that are well behaved in some weak sense), a smaller sample size suffices: $N = c_1dη^{-2}\log(2/η)$. We then show that if the slabs are replaced by randomly generated ellipsoids defined using $X_1,...,X_N$, the same degree of approximation is true when $N \geq c_2dη^{-2}\log(2/η)$. The construction we use is based on the small-ball method.

stat.ML↗
arXiv2018

An optimal unrestricted learning procedure

We study learning problems involving arbitrary classes of functions $F$, distributions $X$ and targets $Y$. Because proper learning procedures, i.e., procedures that are only allowed to select functions in $F$, tend to perform poorly unless the problem satisfies some additional structural property (e.g., that $F$ is convex), we consider unrestricted learning procedures that are free to choose functions outside the given class. We present a new unrestricted procedure that is optimal in a very strong sense: the required sample complexity is essentially the best one can hope for, and the estimate holds for (almost) any problem, including heavy-tailed situations. Moreover, the sample complexity coincides with the what one would expect if $F$ were convex, even when $F$ is not. And if $F$ is convex, the procedure turns out to be proper. Thus, the unrestricted procedure is actually optimal in both realms, for convex classes as a proper procedure and for arbitrary classes as an unrestricted procedure.

stat.ML↗
arXiv2018

Improved bounds for sparse recovery from subsampled random convolutions

We study the recovery of sparse vectors from subsampled random convolutions via $\ell_1$-minimization. We consider the setup in which both the subsampling locations as well as the generating vector are chosen at random. For a subgaussian generator with independent entries, we improve previously known estimates: if the sparsity $s$ is small enough, i.e., $s \lesssim \sqrt{n/\log(n)}$, we show that $m \gtrsim s \log(en/s)$ measurements are sufficient to recover $s$-sparse vectors in dimension $n$ with high probability, matching the well-known condition for recovery from standard Gaussian measurements. If $s$ is larger, then essentially $m \geq s \log^2(s) \log(\log(s)) \log(n)$ measurements are sufficient, again improving over previous estimates. Our results are shown via the so-called robust null space property which is weaker than the standard restricted isometry property. Our method of proof involves a novel combination of small ball estimates with chaining techniques {which should be of independent interest.

cs.IT↗
arXiv2017

A remark on "Robust machine learning by median-of-means"

We explore the recent results announced in "Robust machine learning by median-of-means: theory and practice" by G. Lecué and M. Lerasle. We show that these results are, in fact, almost obvious outcomes of the machinery developed in [4] for the study of tournament procedures.

math.ST↗