arXiv subjects

Shahar Mendelson

Publications and source records attributed to Shahar Mendelson.

At least 19 recordsLinked to original sources

arXiv2026

On the structure of marginals in high dimensions

Let $G, G_1,\dots,G_N$ be independent copies of a standard gaussian random vector in $\mathbb{R}^d$ and denote by $\Gamma = \sum_{i=1}^N \langle G_i,\cdot\rangle e_i$ the standard gaussian ensemble. We show that, for any set $A\subset S^{d-1}$, with exponentially high probability, \[ \sup_{x\in A} \frac{1}{N}\sum_{i=1}^N \big| (\Gamma x)^\sharp_i - q_i\big| \le c \frac{ \mathbb{E} \sup_{x\in A} \langle G,x\rangle + \log^2N }{\sqrt N }. \] Here each $q_i$ is the $\frac{i}{N+1}$-quantile of the standard normal distribution and $(\Gamma x)^\sharp $ denotes the monotone increasing rearrangement of the vector $\Gamma x$. The estimate is sharp up to a possible logarithmic factor and significantly extends previously known bounds. Moreover, we show that similar estimates hold in much greater generality: after replacing the gaussian quantiles by the appropriate ones, the same phenomenon persists for a broad class of random vectors.

math.PR↗

arXiv2025

Uniform mean estimation via generic chaining

We introduce an empirical functional $\Psi$ that is an optimal uniform mean estimator: Let $F\subset L_2(\mu)$ be a class of mean zero functions, $u$ is a real valued function, and $X_1,\dots,X_N$ are independent, distributed according to $\mu$. We show that under minimal assumptions, with $\mu^{\otimes N}$ exponentially high probability, \[ \sup_{f\in F} |\Psi(X_1,\dots,X_N,f) - \mathbb{E} u(f(X))| \leq c R(F) \frac{ \mathbb{E} \sup_{f\in F } |G_f| }{\sqrt N}, \] where $(G_f)_{f\in F}$ is the gaussian processes indexed by $F$ and $R(F)$ is an appropriate notion of `diameter' of the class $\{u(f(X)) : f\in F\}$. The fact that such a bound is possible is surprising, and it leads to the solution of various key problems in high dimensional probability and high dimensional statistics. The construction is based on combining Talagrand's generic chaining mechanism with optimal mean estimation procedures for a single real-valued random variable.

math.PR↗

arXiv2025

Do we really need the Rademacher complexities?

We study the fundamental problem of learning with respect to the squared loss in a convex class. The state-of-the-art sample complexity estimates in this setting rely on Rademacher complexities, which are generally difficult to control. We prove that, contrary to prevailing belief and under minimal assumptions, the sample complexity is not governed by the Rademacher complexities but rather by the behaviour of the limiting gaussian process. In particular, all such learning problems that have the same $L_2$-structure -- even those with heavy-tailed distributions -- share the same sample complexity. This constitutes the first universality result for general convex learning problems. The proof is based on a novel learning procedure, and its performance is studied by combining optimal mean estimation techniques for real-valued random variables with Talagrand's generic chaining method.

math.ST↗

arXiv2024

Can we spot a fake?

The problem of detecting fake data inspires the following seemingly simple mathematical question. Sample a data point $X$ from the standard normal distribution in $\mathbb{R}^n$. An adversary observes $X$ and corrupts it by adding a vector $rt$, where they can choose any vector $t$ from a fixed set $T$ of the adversary's ``tricks'', and where $r>0$ is a fixed radius. The adversary's choice of $t=t(X)$ may depend on the true data $X$. The adversary wants to hide the corruption by making the fake data $X+rt$ statistically indistinguishable from the real data $X$. What is the largest radius $r=r(T)$ for which the adversary can create an undetectable fake? We show that for highly symmetric sets $T$, the detectability radius $r(T)$ is approximately twice the scaled Gaussian width of $T$. The upper bound actually holds for arbitrary sets $T$ and generalizes to arbitrary, non-Gaussian distributions of real data $X$. The lower bound may fail for not highly symmetric $T$, but we conjecture that this problem can be solved by considering the focused version of the Gaussian width of $T$, which focuses on the most important directions of $T$.

math.ST↗

arXiv2024

Covariance estimation with direction dependence accuracy

We construct an estimator $\widehat{\Sigma}$ for covariance matrices of unknown, centred random vectors X, with the given data consisting of N independent measurements $X_1,...,X_N$ of X and the wanted confidence level. We show under minimal assumptions on X, the estimator performs with the optimal accuracy with respect to the operator norm. In addition, the estimator is also optimal with respect to direction dependence accuracy: $\langle \widehat{\Sigma}u,u\rangle$ is an optimal estimator for $\sigma^2(u)=\mathbb{E}\langle X,u\rangle^2$ when $\sigma^2(u)$ is ``large".

math.ST↗

arXiv2023

A uniform Dvoretzky-Kiefer-Wolfowitz inequality

We show that under minimal assumptions on a class of functions $\mathcal{H}$ defined on a probability space $(\mathcal{X},\mu)$, there is a threshold $\Delta_0$ satisfying the following: for every $\Delta\geq\Delta_0$, with probability at least $1-2\exp(-c\Delta m)$ with respect to $\mu^{\otimes m}$, \[ \sup_{h\in\mathcal{H}} \sup_{t\in\mathbb{R}} \left| \mathbb{P}(h(X)\leq t) - \frac{1}{m}\sum_{i=1}^m 1_{(-\infty,t]}(h(X_i)) \right| \leq \sqrt{\Delta};\] here $X$ is distributed according to $\mu$ and $(X_i)_{i=1}^m$ are independent copies of $X$. The value of $\Delta_0$ is determined by an unexpected complexity parameter of the class $\mathcal{H}$ that captures the set's geometry (Talagrand's $\gamma_1$-functional). The bound, the probability estimate and the value of $\Delta_0$ are all optimal up to a logarithmic factor.

math.PR↗

arXiv2023

Optimal non-gaussian Dvoretzky-Milman embeddings

We construct the first non-gaussian ensemble that yields the optimal estimate in the Dvoretzky-Milman Theorem: the ensemble exhibits almost Euclidean sections in arbitrary normed spaces of the same dimension as the gaussian embedding -- despite being very far from gaussian (in fact, it happens to be heavy-tailed).

math.FA↗

arXiv2023

Empirical approximation of the gaussian distribution in $\mathbb{R}^d$

Let $G_1,\dots,G_m$ be independent copies of the standard gaussian random vector in $\mathbb{R}^d$. We show that there is an absolute constant $c$ such that for any $A \subset S^{d-1}$, with probability at least $1-2\exp(-c\Delta m)$, for every $t\in\mathbb{R}$, \[ \sup_{x \in A} \left| \frac{1}{m}\sum_{i=1}^m 1_{ \{\langle G_i,x\rangle \leq t \}} - \mathbb{P}(\langle G,x\rangle \leq t) \right| \leq \Delta + \sigma(t) \sqrt\Delta. \] Here $\sigma(t) $ is the variance of $1_{\{\langle G,x\rangle\leq t\}}$ and $\Delta\geq \Delta_0$, where $\Delta_0$ is determined by an unexpected complexity parameter of $A$ that captures the set's geometry (Talagrand's $\gamma_1$ functional). The bound, the probability estimate, and the value of $\Delta_0$ are all (almost) optimal. We use this fact to show that if $\Gamma=\sum_{i=1}^m \langle G_i,x\rangle e_i$ is the random matrix that has $G_1,\dots,G_m$ as its rows, then the structure of $\Gamma(A)=\{\Gamma x: x\in A\}$ is far more rigid and well-prescribed than was previously expected.

math.PR↗

arXiv2023

On a variance dependent Dvoretzky-Kiefer-Wolfowitz inequality

Let $X$ be a real-valued random variable with distribution function $F$. Set $X_1,\dots, X_m$ to be independent copies of $X$ and let $F_m$ be the corresponding empirical distribution function. We show that there are absolute constants $c_0$ and $c_1$ such that if $\Delta \geq c_0\frac{\log\log m}{m}$, then with probability at least $1-2\exp(-c_1\Delta m)$, for every $t\in\mathbb{R}$ that satisfies $F(t)\in[\Delta,1-\Delta]$, \[ |F_m(t) - F(t) | \leq \sqrt{\Delta \min\{F(t),1-F(t)\} } .\] Moreover, this estimate is optimal up to the multiplicative constants $c_0$ and $c_1$.

math.PR↗

arXiv2023

Fitting an ellipsoid to a quadratic number of random points

We consider the problem $(\mathrm{P})$ of fitting $n$ standard Gaussian random vectors in $\mathbb{R}^d$ to the boundary of a centered ellipsoid, as $n, d \to \infty$. This problem is conjectured to have a sharp feasibility transition: for any $\varepsilon > 0$, if $n \leq (1 - \varepsilon) d^2 / 4$ then $(\mathrm{P})$ has a solution with high probability, while $(\mathrm{P})$ has no solutions with high probability if $n \geq (1 + \varepsilon) d^2 /4$. So far, only a trivial bound $n \geq d^2 / 2$ is known on the negative side, while the best results on the positive side assume $n \leq d^2 / \mathrm{polylog}(d)$. In this work, we improve over previous approaches using a key result of Bartl & Mendelson (2022) on the concentration of Gram matrices of random vectors under mild assumptions on their tail behavior. This allows us to give a simple proof that $(\mathrm{P})$ is feasible with high probability when $n \leq d^2 / C$, for a (possibly large) constant $C > 0$.

math.PR↗

arXiv2022

Structure preservation via the Wasserstein distance

We show that under minimal assumptions on a random vector $X\in\mathbb{R}^d$ and with high probability, given $m$ independent copies of $X$, the coordinate distribution of each vector $(\langle X_i,\theta \rangle)_{i=1}^m$ is dictated by the distribution of the true marginal $\langle X,\theta \rangle$. Specifically, we show that with high probability, \[\sup_{\theta \in S^{d-1}} \left( \frac{1}{m}\sum_{i=1}^m \left|\langle X_i,\theta \rangle^\sharp - \lambda^\theta_i \right|^2 \right)^{1/2} \leq c \left( \frac{d}{m} \right)^{1/4},\] where $\lambda^{\theta}_i = m\int_{(\frac{i-1}{m}, \frac{i}{m}]} F_{ \langle X,\theta \rangle }^{-1}(u)\,du$ and $a^\sharp$ denotes the monotone non-decreasing rearrangement of $a$. Moreover, this estimate is optimal. The proof follows from a sharp estimate on the worst Wasserstein distance between a marginal of $X$ and its empirical counterpart, $\frac{1}{m} \sum_{i=1}^m \delta_{\langle X_i, \theta \rangle}$.

math.ST↗

arXiv2022

Fast metric embedding into the Hamming cube

We consider the problem of embedding a subset of $\mathbb{R}^n$ into a low-dimensional Hamming cube in an almost isometric way. We construct a simple, data-oblivious, and computationally efficient map that achieves this task with high probability: we first apply a specific structured random matrix, which we call the double circulant matrix; using that matrix requires linear storage and matrix-vector multiplication can be performed in near-linear time. We then binarize each vector by comparing each of its entries to a random threshold, selected uniformly at random from a well-chosen interval. We estimate the number of bits required for this encoding scheme in terms of two natural geometric complexity parameters of the set - its Euclidean covering numbers and its localized Gaussian complexity. The estimate we derive turns out to be the best that one can hope for - up to logarithmic terms. The key to the proof is a phenomenon of independent interest: we show that the double circulant matrix mimics the behavior of a Gaussian matrix in two important ways. First, it maps an arbitrary set in $\mathbb{R}^n$ into a set of well-spread vectors. Second, it yields a fast near-isometric embedding of any finite subset of $\ell_2^n$ into $\ell_1^m$. This embedding achieves the same dimension reduction as a Gaussian matrix in near-linear time, under an optimal condition - up to logarithmic factors - on the number of points to be embedded. This improves a well-known construction due to Ailon and Chazelle.

math.PR↗

arXiv2022

Random embeddings with an almost Gaussian distortion

Let $X$ be a symmetric, isotropic random vector in $\mathbb{R}^m$ and let $X_1...,X_n$ be independent copies of $X$. We show that under mild assumptions on $\|X\|_2$ (a suitable thin-shell bound) and on the tail-decay of the marginals $\langle X,u\rangle$, the random matrix $A$, whose columns are $X_i/\sqrt{m}$ exhibits a Gaussian-like behaviour in the following sense: for an arbitrary subset of $T\subset \mathbb{R}^n$, the distortion $\sup_{t \in T} | \|At\|_2^2 - \|t\|_2^2 |$ is almost the same as if $A$ were a Gaussian matrix. A simple outcome of our result is that if $X$ is a symmetric, isotropic, log-concave random vector and $n \leq m \leq c_1(α)n^α$ for some $α>1$, then with high probability, the extremal singular values of $A$ satisfy the optimal estimate: $1-c_2(α) \sqrt{n/m} \leq λ_{\rm min} \leq λ_{\rm max} \leq 1+c_2(α) \sqrt{n/m}$.

math.FA↗

arXiv2022

On Monte-Carlo methods in convex stochastic optimization

We develop a novel procedure for estimating the optimizer of general convex stochastic optimization problems of the form $\min_{x\in\mathcal{X}} \mathbb{E}[F(x,ξ)]$, when the given data is a finite independent sample selected according to $ξ$. The procedure is based on a median-of-means tournament, and is the first procedure that exhibits the optimal statistical performance in heavy tailed situations: we recover the asymptotic rates dictated by the central limit theorem in a non-asymptotic manner once the sample size exceeds some explicitly computable threshold. Additionally, our results apply in the high-dimensional setup, as the threshold sample size exhibits the optimal dependence on the dimension (up to a logarithmic factor). The general setting allows us to recover recent results on multivariate mean estimation and linear regression in heavy-tailed situations and to prove the first sharp, non-asymptotic results for the portfolio optimization problem.

math.ST↗

arXiv2022

Sharp estimates on random hyperplane tessellations

We study the problem of generating a hyperplane tessellation of an arbitrary set $T$ in $\mathbb{R}^n$, ensuring that the Euclidean distance between any two points corresponds to the fraction of hyperplanes separating them up to a pre-specified error $δ$. We focus on random gaussian tessellations with uniformly distributed shifts and derive sharp bounds on the number of hyperplanes $m$ that are required. Surprisingly, our lower estimates falsify the conjecture that $m\sim \ell_*^2(T)/δ^2$, where $\ell_*^2(T)$ is the gaussian width of $T$, is optimal.

math.PR↗

arXiv2021

Column randomization and almost-isometric embeddings

The matrix $A:\mathbb{R}^n \to \mathbb{R}^m$ is $(δ,k)$-regular if for any $k$-sparse vector $x$, $$ \left| \|Ax\|_2^2-\|x\|_2^2\right| \leq δ\sqrt{k} \|x\|_2^2. $$ We show that if $A$ is $(δ,k)$-regular for $1 \leq k \leq 1/δ^2$, then by multiplying the columns of $A$ by independent random signs, the resulting random ensemble $A_ε$ acts on an arbitrary subset $T \subset \mathbb{R}^n$ (almost) as if it were gaussian, and with the optimal probability estimate: if $\ell_*(T)$ is the gaussian mean-width of $T$ and $d_T=\sup_{t \in T} \|t\|_2$, then with probability at least $1-2\exp(-c(\ell_*(T)/d_T)^2)$, $$ \sup_{t \in T} \left| \|A_εt\|_2^2-\|t\|_2^2 \right| \leq C\left(Λd_T δ\ell_*(T)+(δ\ell_*(T))^2 \right), $$ where $Λ=\max\{1,δ^2\log(nδ^2)\}$. This estimate is optimal for $0<δ\leq 1/\sqrt{\log n}$.

math.ST↗

arXiv2020

An isomorphic Dvoretzky-Milman Theorem using general random ensembles

We construct rather general random ensembles that yield the optimal (isomorphic) estimate in the Dvoretzky-Milman Theorem. This is the first construction of non gaussian/spherical ensembles that exhibit the optimal behaviour. The ensembles constructed here need not satisfy any rotation invariance and can be rather heavy-tailed.

math.FA↗

arXiv2020

Multivariate mean estimation with direction-dependent accuracy

We consider the problem of estimating the mean of a random vector based on $N$ independent, identically distributed observations. We prove the existence of an estimator that has a near-optimal error in all directions in which the variance of the one dimensional marginal of the random vector is not too small: with probability $1-δ$, the procedure returns $\whμ_N$ which satisfies that for every direction $u \in S^{d-1}$, \[ \inr{\whμ_N - μ, u}\le \frac{C}{\sqrt{N}} \left( σ(u)\sqrt{\log(1/δ)} + \left(\E\|X-\EXP X\|_2^2\right)^{1/2} \right)~, \] where $σ^2(u) = \var(\inr{X,u})$ and $C$ is a constant. To achieve this, we require only slightly more than the existence of the covariance matrix, in the form of a certain moment-equivalence assumption. The proof relies on novel bounds for the ratio of empirical and true probabilities that hold uniformly over certain classes of random variables.

math.ST↗