Searcharxiv⌕ Search

arXiv subjects

Mark Rudelson

Publications and source records attributed to Mark Rudelson.

At least 37 records · Page 2Linked to original sources

Invertibility of Sparse non-Hermitian matrices

We consider a class of sparse random matrices of the form $A_n =(ξ_{i,j}δ_{i,j})_{i,j=1}^n$, where $\{ξ_{i,j}\}$ are i.i.d.~centered random variables, and $\{δ_{i,j}\}$ are i.i.d.~Bernoulli random variables taking value $1$ with probability $p_n$, and prove a quantitative estimate on the smallest singular value for $p_n = Ω(\frac{\log n}{n})$, under a suitable assumption on the spectral norm of the matrices. This establishes the invertibility of a large class of sparse matrices. For $p_n =Ω( n^{-α})$ with some $α\in (0,1)$, we deduce that the condition number of $A_n$ is of order $n$ with probability tending to one under the optimal moment assumption on $\{ξ_{i,j}\}$. This in particular, extends a conjecture of von Neumann about the condition number to sparse random matrices with heavy-tailed entries. In the case that the random variables $\{ξ_{i,j}\}$ are i.i.d.~sub-Gaussian, we further show that a sparse random matrix is singular with probability at most $\exp(-c n p_n)$ whenever $p_n$ is above the critical threshold $p_n = Ω(\frac{ \log n}{n})$. The results also extend to the case when $\{ξ_{i,j}\}$ have a non-zero mean. We further find quantitative estimates on the smallest singular value of the adjacency matrix of a directed Erdős-Réyni graph whenever its edge connectivity probability is above the critical threshold $Ω(\frac{\log n}{n})$.

math.PR↗

Hafnians, perfect matchings and Gaussian matrices

We analyze the behavior of the Barvinok estimator of the hafnian of even dimension, symmetric matrices with nonnegative entries. We introduce a condition under which the Barvinok estimator achieves subexponential errors, and show that this condition is almost optimal. Using that hafnians count the number of perfect matchings in graphs, we conclude that Barvinok's estimator gives a polynomial-time algorithm for the approximate (up to subexponential errors) evaluation of the number of perfect matchings.

math.PR↗

High dimensional errors-in-variables models with dependent measurements

Suppose that we observe $y \in \mathbb{R}^f$ and $X \in \mathbb{R}^{f \times m}$ in the following errors-in-variables model: \begin{eqnarray*} y & = & X_0 β^* + ε\\ X & = & X_0 + W \end{eqnarray*} where $X_0$ is a $f \times m$ design matrix with independent subgaussian row vectors, $ε\in \mathbb{R}^f$ is a noise vector and $W$ is a mean zero $f \times m$ random noise matrix with independent subgaussian column vectors, independent of $X_0$ and $ε$. This model is significantly different from those analyzed in the literature in the sense that we allow the measurement error for each covariate to be a dependent vector across its $f$ observations. Such error structures appear in the science literature when modeling the trial-to-trial fluctuations in response strength shared across a set of neurons. Under sparsity and restrictive eigenvalue type of conditions, we show that one is able to recover a sparse vector $β^* \in \mathbb{R}^m$ from the model given a single observation matrix $X$ and the response vector $y$. We establish consistency in estimating $β^*$ and obtain the rates of convergence in the $\ell_q$ norm, where $q = 1, 2$ for the Lasso-type estimator, and for $q \in [1, 2]$ for a Dantzig-type conic programming estimator. We show error bounds which approach that of the regular Lasso and the Dantzig selector in case the errors in $W$ are tending to 0.

math.ST↗

On the complexity of the set of unconditional convex bodies

We show that for any $t>1$, the set of unconditional convex bodies in $\mathbb{R}^n$ contains a $t$-separated subset of cardinality at least $\exp \exp (C(t) n)$. This implies that there exists an unconditional convex body in $\mathbb{R}^n$ which cannot be approximated within the distance $d$ by a projection of a polytope with $N$ faces unless $N > \exp(c(d)n)$. We also show that for $t>2$, the cardinality of a $t$-separated set of completely symmetric bodies in $\mathbb{R}^n$ does not exceed $\exp \exp (c(t) \log^2 n)$.

math.MG↗

No-gaps delocalization for general random matrices

We prove that with high probability, every eigenvector of a random matrix is delocalized in the sense that any subset of its coordinates carries a non-negligible portion of its $\ell_2$ norm. Our results pertain to a wide class of random matrices, including matrices with independent entries, symmetric and skew-symmetric matrices, as well as some other naturally arising ensembles. The matrices can be real and complex; in the latter case we assume that the real and imaginary parts of the entries are independent.

math.PR↗

Spectral Norm of Random Kernel Matrices with Applications to Privacy

Kernel methods are an extremely popular set of techniques used for many important machine learning and data analysis applications. In addition to having good practical performances, these methods are supported by a well-developed theory. Kernel methods use an implicit mapping of the input data into a high dimensional feature space defined by a kernel function, i.e., a function returning the inner product between the images of two data points in the feature space. Central to any kernel method is the kernel matrix, which is built by evaluating the kernel function on a given sample dataset. In this paper, we initiate the study of non-asymptotic spectral theory of random kernel matrices. These are n x n random matrices whose (i,j)th entry is obtained by evaluating the kernel function on $x_i$ and $x_j$, where $x_1,...,x_n$ are a set of n independent random high-dimensional vectors. Our main contribution is to obtain tight upper bounds on the spectral norm (largest eigenvalue) of random kernel matrices constructed by commonly used kernel functions based on polynomials and Gaussian radial basis. As an application of these results, we provide lower bounds on the distortion needed for releasing the coefficients of kernel ridge regression under attribute privacy, a general privacy notion which captures a large class of privacy definitions. Kernel ridge regression is standard method for performing non-parametric regression that regularly outperforms traditional regression approaches in various domains. Our privacy distortion lower bounds are the first for any kernel technique, and our analysis assumes realistic scenarios for the input, unlike all previous lower bounds for other release problems which only hold under very restrictive input settings.

stat.ML↗

Small ball probabilities for linear images of high dimensional distributions

We study concentration properties of random vectors of the form $AX$, where $X = (X_1, ..., X_n)$ has independent coordinates and $A$ is a given matrix. We show that the distribution of $AX$ is well spread in space whenever the distributions of $X_i$ are well spread on the line. Specifically, assume that the probability that $X_i$ falls in any given interval of length $T$ is at most $p$. Then the probability that $AX$ falls in any given ball of radius $T \|A\|_{HS}$ is at most $(Cp)^{0.9 r(A)}$, where $r(A)$ denotes the stable rank of $A$ and $C$ is an absolute constant.

math.PR↗

Delocalization of eigenvectors of random matrices with independent entries

We prove that an n by n random matrix G with independent entries is completely delocalized. Suppose the entries of G have zero means, variances uniformly bounded below, and a uniform tail decay of exponential type. Then with high probability all unit eigenvectors of G have all coordinates of magnitude O(n^{-1/2}), modulo logarithmic corrections. This comes a consequence of a new, geometric, approach to delocalization for random matrices.

math.PR↗

Singular values of Gaussian matrices and permanent estimators

We present estimates on the small singular values of a class of matrices with independent Gaussian entries and inhomogeneous variance profile, satisfying a broad-connectedness condition. Using these estimates and concentration of measure for the spectrum of Gaussian matrices with independent entries, we prove that for a large class of graphs satisfying an appropriate expansion property, the Barvinok--Godsil-Gutman estimator for the permanent achieves sub-exponential errors with high probability.

math.PR↗

Hanson-Wright inequality and sub-gaussian concentration

In this expository note, we give a modern proof of Hanson-Wright inequality for quadratic forms in sub-gaussian random variables. We deduce a useful concentration inequality for sub-gaussian random vectors. Two examples are given to illustrate these results: a concentration of distances between random vectors and subspaces, and a bound on the norms of products of random and deterministic matrices.

math.PR↗

Recent developments in non-asymptotic theory of random matrices

Non-asymptotic theory of random matrices strives to investigate the spectral properties of random matrices, which are valid with high probability for matrices of a large fixed size. Results obtained in this framework find their applications in high-dimensional convexity, analysis of convergence of algorithms, as well as in random matrix theory itself. In these notes we survey some recent results in this area and describe the techniques aimed for obtaining explicit probability bounds.

math.PR↗

Invertibility of random matrices: unitary and orthogonal perturbations

We show that a perturbation of any fixed square matrix D by a random unitary matrix is well invertible with high probability. A similar result holds for perturbations by random orthogonal matrices; the only notable exception is when D is close to orthogonal. As an application, these results completely eliminate a hard-to-check condition from the Single Ring Theorem by Guionnet, Krishnapur and Zeitouni.

math.PR↗

The Power of Linear Reconstruction Attacks

We consider the power of linear reconstruction attacks in statistical data privacy, showing that they can be applied to a much wider range of settings than previously understood. Linear attacks have been studied before (Dinur and Nissim PODS'03, Dwork, McSherry and Talwar STOC'07, Kasiviswanathan, Rudelson, Smith and Ullman STOC'10, De TCC'12, Muthukrishnan and Nikolov STOC'12) but have so far been applied only in settings with releases that are obviously linear. Consider a database curator who manages a database of sensitive information but wants to release statistics about how a sensitive attribute (say, disease) in the database relates to some nonsensitive attributes (e.g., postal code, age, gender, etc). We show one can mount linear reconstruction attacks based on any release that gives: a) the fraction of records that satisfy a given non-degenerate boolean function. Such releases include contingency tables (previously studied by Kasiviswanathan et al., STOC'10) as well as more complex outputs like the error rate of classifiers such as decision trees; b) any one of a large class of M-estimators (that is, the output of empirical risk minimization algorithms), including the standard estimators for linear and logistic regression. We make two contributions: first, we show how these types of releases can be transformed into a linear format, making them amenable to existing polynomial-time reconstruction algorithms. This is already perhaps surprising, since many of the above releases (like M-estimators) are obtained by solving highly nonlinear formulations. Second, we show how to analyze the resulting attacks under various distributional assumptions on the data. Specifically, we consider a setting in which the same statistic (either a) or b) above) is released about how the sensitive attribute relates to all subsets of size k (out of a total of d) nonsensitive boolean attributes.

cs.DS↗

On approximations by projections of polytopes with few facets

We provide an affirmative answer to a problem posed by Barvinok and Veomett, showing that in general an n-dimensional convex body cannot be approximated by a projection of a section of a simplex of a sub-exponential dimension. Moreover, we establish a lower bound of the Banach-Mazur distance between n-dimensional projections of sections of an N-dimensional simplex and a certain convex symmetric body, which is sharp up to a logarithmic factor for all N>n.

math.FA↗

Row products of random matrices

We define the row product of K matrices of size d by n as a matrix of size d^K by n, whose row are entry-wise products of rows of these matrices. This construction arises in certain computer science problems. We study the question, to which extent the spectral and geometric properties of the row product of independent random matrices resemble those properties for a d^K by n matrix with independent random entries. In particular, we show that the largest and the smallest singular values of these matrices are of the same order, as long as n is significantly smaller than d^K. We also consider a problem of privately releasing the summary information about a database, and use the previous results to obtain a bound for the minimal amount of noise, which has to be added to the released data to avoid a privacy breach.

math.PR↗

Reconstruction from anisotropic random measurements

Random matrices are widely used in sparse recovery problems, and the relevant properties of matrices with i.i.d. entries are well understood. The current paper discusses the recently introduced Restricted Eigenvalue (RE) condition, which is among the most general assumptions on the matrix, guaranteeing recovery. We prove a reduction principle showing that the RE condition can be guaranteed by checking the restricted isometry on a certain family of low-dimensional subspaces. This principle allows us to establish the RE condition for several broad classes of random matrices with dependent entries, including random matrices with subgaussian rows and non-trivial covariance structure, as well as matrices with independent rows, and uniformly bounded entries.

math.ST↗

Non-asymptotic theory of random matrices: extreme singular values

The classical random matrix theory is mostly focused on asymptotic spectral properties of random matrices as their dimensions grow to infinity. At the same time many recent applications from convex geometry to functional analysis to information theory operate with random matrices in fixed dimensions. This survey addresses the non-asymptotic theory of extreme singular values of random matrices with independent entries. We focus on recently developed geometric methods for estimating the hard edge of random matrices (the smallest singular value).

math.FA↗

The smallest singular value of a random rectangular matrix

We prove an optimal estimate on the smallest singular value of a random subgaussian matrix, valid for all fixed dimensions. For an N by n matrix A with independent and identically distributed subgaussian entries, the smallest singular value of A is at least of the order \sqrt{N} - \sqrt{n-1} with high probability. A sharp estimate on the probability is also obtained.

math.PR↗