arXiv subjects

Joe Neeman

Publications and source records attributed to Joe Neeman.

At least 37 records · Page 2Linked to original sources

arXiv2017

Noise Stability is computable and low dimensional

Questions of noise stability play an important role in hardness of approximation in computer science as well as in the theory of voting. In many applications, the goal is to find an optimizer of noise stability among all possible partitions of $\mathbb{R}^n$ for $n \geq 1$ to $k$ parts with given Gaussian measures $μ_1,\ldots,μ_k$. We call a partition $ε$-optimal, if its noise stability is optimal up to an additive $ε$. In this paper, we give an explicit, computable function $n(ε)$ such that an $ε$-optimal partition exists in $\mathbb{R}^{n(ε)}$. This result has implications for the computability of certain problems in non-interactive simulation, which are addressed in a subsequent work.

math.PR↗

arXiv2017

Non interactive simulation of correlated distributions is decidable

A basic problem in information theory is the following: Let $\mathbf{P} = (\mathbf{X}, \mathbf{Y})$ be an arbitrary distribution where the marginals $\mathbf{X}$ and $\mathbf{Y}$ are (potentially) correlated. Let Alice and Bob be two players where Alice gets samples $\{x_i\}_{i \ge 1}$ and Bob gets samples $\{y_i\}_{i \ge 1}$ and for all $i$, $(x_i, y_i) \sim \mathbf{P}$. What joint distributions $\mathbf{Q}$ can be simulated by Alice and Bob without any interaction? Classical works in information theory by G{á}cs-K{ö}rner and Wyner answer this question when at least one of $\mathbf{P}$ or $\mathbf{Q}$ is the distribution on $\{0,1\} \times \{0,1\}$ where each marginal is unbiased and identical. However, other than this special case, the answer to this question is understood in very few cases. Recently, Ghazi, Kamath and Sudan showed that this problem is decidable for $\mathbf{Q}$ supported on $\{0,1\} \times \{0,1\}$. We extend their result to $\mathbf{Q}$ supported on any finite alphabet. We rely on recent results in Gaussian geometry (by the authors) as well as a new \emph{smoothing argument} inspired by the method of \emph{boosting} from learning theory and potential function arguments from complexity theory and additive combinatorics.

cs.CC↗

arXiv2016

Belief propagation, robust reconstruction and optimal recovery of block models

We consider the problem of reconstructing sparse symmetric block models with two blocks and connection probabilities $a/n$ and $b/n$ for inter- and intra-block edge probabilities, respectively. It was recently shown that one can do better than a random guess if and only if $(a-b)^2>2(a+b)$. Using a variant of belief propagation, we give a reconstruction algorithm that is optimal in the sense that if $(a-b)^2>C(a+b)$ for some constant $C$ then our algorithm maximizes the fraction of the nodes labeled correctly. Ours is the only algorithm proven to achieve the optimal fraction of nodes labeled correctly. Along the way, we prove some results of independent interest regarding robust reconstruction for the Ising model on regular and Poisson trees.

math.PR↗

arXiv2016

Information-theoretic thresholds for community detection in sparse networks

We give upper and lower bounds on the information-theoretic threshold for community detection in the stochastic block model. Specifically, consider the symmetric stochastic block model with $q$ groups, average degree $d$, and connection probabilities $c_\text{in}/n$ and $c_\text{out}/n$ for within-group and between-group edges respectively; let $λ= (c_\text{in}-c_\text{out})/(qd)$. We show that, when $q$ is large, and $λ= O(1/q)$, the critical value of $d$ at which community detection becomes possible---in physical terms, the condensation threshold---is \[ d_\text{c} = Θ\!\left( \frac{\log q}{q λ^2} \right) \, , \] with tighter results in certain regimes. Above this threshold, we show that any partition of the nodes into $q$ groups which is as `good' as the planted one, in terms of the number of within- and between-group edges, is correlated with it. This gives an exponential-time algorithm that performs better than chance; specifically, community detection becomes possible below the Kesten-Stigum bound for $q \ge 5$ in the disassortative case $λ< 0$, and for $q \ge 11$ in the assortative case $λ>0$ (similar upper bounds were obtained independently by Abbe and Sandon). Conversely, below this threshold, we show that no algorithm can label the vertices better than chance, or even distinguish the block model from an \ER\ random graph with high probability. Our lower bound on $d_\text{c}$ uses Robinson and Wormald's small subgraph conditioning method, and we also give (less explicit) results for non-symmetric stochastic block models. In the symmetric case, we obtain explicit results by using bounds on certain functions of doubly stochastic matrices due to Achlioptas and Naor; indeed, our lower bound on $d_\text{c}$ is their second moment lower bound on the $q$-colorability threshold for random graphs with a certain effective degree.

math.PR↗

arXiv2016

An interpolation proof of Ehrhard's inequality

We prove Ehrhard's inequality using interpolation along the Ornstein-Uhlenbeck semi-group. We also provide an improved Jensen inequality for Gaussian variables that might be of independent interest.

math.PR↗

arXiv2016

Noise Stability and Correlation with Half Spaces

Benjamini, Kalai and Schramm showed that a monotone function $f : \{-1,1\}^n \to \{-1,1\}$ is noise stable if and only if it is correlated with a half-space (a set of the form $\{x: \langle x, a\rangle \le b\}$). We study noise stability in terms of correlation with half-spaces for general (not necessarily monotone) functions. We show that a function $f: \{-1, 1\}^n \to \{-1, 1\}$ is noise stable if and only if it becomes correlated with a half-space when we modify $f$ by randomly restricting a constant fraction of its coordinates. Looking at random restrictions is necessary: we construct noise stable functions whose correlation with any half-space is $o(1)$. The examples further satisfy that different restrictions are correlated with different half-spaces: for any fixed half-space, the probability that a random restriction is correlated with it goes to zero. We also provide quantitative versions of the above statements, and versions that apply for the Gaussian measure on $\mathbb{R}^n$ instead of the discrete cube. Our work is motivated by questions in learning theory and a recent question of Khot and Moshkovitz.

math.PR↗

arXiv2015

A Proof Of The Block Model Threshold Conjecture

We study a random graph model named the "block model" in statistics and the "planted partition model" in theoretical computer science. In its simplest form, this is a random graph with two equal-sized clusters, with a between-class edge probability of $q$ and a within-class edge probability of $p$. A striking conjecture of Decelle, Krzkala, Moore and Zdeborová based on deep, non-rigorous ideas from statistical physics, gave a precise prediction for the algorithmic threshold of clustering in the sparse planted partition model. In particular, if $p = a/n$ and $q = b/n$, $s=(a-b)/2$ and $p=(a+b)/2$ then Decelle et al.\ conjectured that it is possible to efficiently cluster in a way correlated with the true partition if $s^2 > p$ and impossible if $s^2 < p$. By comparison, the best-known rigorous result is that of Coja-Oghlan, who showed that clustering is possible if $s^2 > C p \ln p$ for some sufficiently large $C$. In a previous work, we proved that indeed it is information theoretically impossible to to cluster if $s^2 < p$ and furthermore it is information theoretically impossible to even estimate the model parameters from the graph when $s^2 < p$. Here we complete the proof of the conjecture by providing an efficient algorithm for clustering in a way that is correlated with the true partition when $s^2 > p$. A different independent proof of the same result was recently obtained by Laurent Massoulie.

math.PR↗

arXiv2015

Preference Completion: Large-scale Collaborative Ranking from Pairwise Comparisons

In this paper we consider the collaborative ranking setting: a pool of users each provides a small number of pairwise preferences between $d$ possible items; from these we need to predict preferences of the users for items they have not yet seen. We do so by fitting a rank $r$ score matrix to the pairwise data, and provide two main contributions: (a) we show that an algorithm based on convex optimization provides good generalization guarantees once each user provides as few as $O(r\log^2 d)$ pairwise comparisons -- essentially matching the sample complexity required in the related matrix completion setting (which uses actual numerical as opposed to pairwise information), and (b) we develop a large-scale non-convex implementation, which we call AltSVM, that trains a factored form of the matrix via alternating minimization (which we show reduces to alternating SVM problems), and scales and parallelizes very well to large problem settings. It also outperforms common baselines on many moderately large popular collaborative filtering datasets in both NDCG and in other measures of ranking performance.

stat.ML↗

arXiv2015

Robust dimension free isoperimetry in Gaussian space

We prove the first robust dimension free isoperimetric result for the standard Gaussian measure $γ_n$ and the corresponding boundary measure $γ_n^+$ in $\mathbb {R}^n$. The main result in the theory of Gaussian isoperimetry (proven in the 1970s by Sudakov and Tsirelson, and independently by Borell) states that if $γ_n(A)=1/2$ then the surface area of $A$ is bounded by the surface area of a half-space with the same measure, $γ_n^+(A)\leq(2π)^{-1/2}$. Our results imply in particular that if $A\subset \mathbb {R}^n$ satisfies $γ_n(A)=1/2$ and $γ_n^+(A)\leq(2π)^{-1/2}+δ$ then there exists a half-space $B\subset \mathbb {R}^n$ such that $γ_n(AΔB)\leq C\smash{\log^{-1/2}}(1/δ)$ for an absolute constant $C$. Since the Gaussian isoperimetric result was established, only recently a robust version of the Gaussian isoperimetric result was obtained by Cianchi et al., who showed that $γ_n(AΔB)\le C(n)\sqrtδ$ for some function $C(n)$ with no effective bounds. Compared to the results of Cianchi et al., our results have optimal (i.e., no) dependence on the dimension, but worse dependence on $ δ$.

math.PR↗

arXiv2014

Standard Simplices and Pluralities are Not the Most Noise Stable

The Standard Simplex Conjecture and the Plurality is Stablest Conjecture are two conjectures stating that certain partitions are optimal with respect to Gaussian and discrete noise stability respectively. These two conjectures are natural generalizations of the Gaussian noise stability result by Borell (1985) and the Majority is Stablest Theorem (2004). Here we show that the standard simplex is not the most stable partition in Gaussian space and that Plurality is not the most stable low influence partition in discrete space for every number of parts $k \geq 3$, for every value $ρ\neq 0$ of the noise and for every prescribed measures for the different parts as long as they are not all equal to $1/k$. Our results do not contradict the original statements of the Plurality is Stablest and Standard Simplex Conjectures in their original statements concerning partitions to sets of equal measure. However, they indicate that if these conjectures are true, their veracity and their proofs will crucially rely on assuming that the sets are of equal measures, in stark contrast to Borell's result, the Majority is Stablest Theorem and many other results in isoperimetric theory. Given our results it is natural to ask for (conjectured) partitions achieving the optimum noise stability.

math.PR↗

arXiv2014

Non-Reconstructability in the Stochastic Block Model

We consider the problem of clustering (or reconstruction) in the stochastic block model, in the regime where the average degree is constant. For the case of two clusters with equal sizes, recent results by Mossel, Neeman and Sly, and by Massoulie, show that reconstructability undergoes a phase transition at the Kesten-Stigum bound of $λ_2^2 d = 1$, where $λ_2$ is the second largest eigenvalue of a related stochastic matrix and $d$ is the average degree. In this paper, we address the general case of more than two clusters and/or unbalanced cluster sizes. Our main result is a sufficient condition for clustering to be impossible, which matches the existing result for two clusters of equal sizes. A key ingredient in our result is a new connection between non-reconstructability and non-distinguishability of the block model from an Erdős-Rényi model with the same average degree. We also show that it is some times possible to reconstruct even when $λ_2^2 d < 1$. Our results provide evidence supporting a series of conjectures made by Decelle, Krzkala, Moore and Zdeborová regarding reconstructability and distinguishability of stochastic block models (but do not settle them).

math.PR↗

arXiv2014

Testing surface area with arbitrary accuracy

Recently, Kothari et al.\ gave an algorithm for testing the surface area of an arbitrary set $A \subset [0, 1]^n$. Specifically, they gave a randomized algorithm such that if $A$'s surface area is less than $S$ then the algorithm will accept with high probability, and if the algorithm accepts with high probability then there is some perturbation of $A$ with surface area at most $κ_n S$. Here, $κ_n$ is a dimension-dependent constant which is strictly larger than 1 if $n \ge 2$, and grows to $4/π$ as $n \to \infty$. We give an improved analysis of Kothari et al.'s algorithm. In doing so, we replace the constant $κ_n$ with $1 + η$ for $η> 0$ arbitrary. We also extend the algorithm to more general measures on Riemannian manifolds.

math.PR↗

arXiv2013

Spectral redemption: clustering sparse networks

Spectral algorithms are classic approaches to clustering and community detection in networks. However, for sparse networks the standard versions of these algorithms are suboptimal, in some cases completely failing to detect communities even when other algorithms such as belief propagation can do so. Here we introduce a new class of spectral algorithms based on a non-backtracking walk on the directed edges of the graph. The spectrum of this operator is much better-behaved than that of the adjacency matrix or other commonly used matrices, maintaining a strong separation between the bulk eigenvalues and the eigenvalues relevant to community structure even in the sparse case. We show that our algorithm is optimal for graphs generated by the stochastic block model, detecting communities all the way down to the theoretical limit. We also show the spectrum of the non-backtracking operator for some real-world networks, illustrating its advantages over traditional spectral clustering.

cs.SI↗

arXiv2013

A multidimensional version of noise stability

We give a multivariate generalization of Borell's noise stability theorem for Gaussian vectors. As a consequence we recover two inequalities, also due to Borell, for exit times of the Ornstein-Uhlenbeck process.

math.PR↗

arXiv2013

Robust Optimality of Gaussian Noise Stability

We prove that under the Gaussian measure, half-spaces are uniquely the most noise stable sets. We also prove a quantitative version of uniqueness, showing that a set which is almost optimally noise stable must be close to a half-space. This extends a theorem of Borell, who proved the same result but without uniqueness, and it also answers a question of Ledoux, who asked whether it was possible to prove Borell's theorem using a direct semigroup argument. Our quantitative uniqueness result has various applications in diverse fields.

math.PR↗

arXiv2012

Majority is Stablest : Discrete and SoS

The Majority is Stablest Theorem has numerous applications in hardness of approximation and social choice theory. We give a new proof of the Majority is Stablest Theorem by induction on the dimension of the discrete cube. Unlike the previous proof, it uses neither the "invariance principle" nor Borell's result in Gaussian space. The new proof is general enough to include all previous variants of majority is stablest such as "it ain't over until it's over" and "Majority is most predictable". Moreover, the new proof allows us to derive a proof of Majority is Stablest in a constant level of the Sum of Squares hierarchy.This implies in particular that Khot-Vishnoi instance of Max-Cut does not provide a gap instance for the Lasserre hierarchy.

cs.CC↗

arXiv2012

On extracting common random bits from correlated sources on large alphabets

Suppose Alice and Bob receive strings $X=(X_1,...,X_n)$ and $Y=(Y_1,...,Y_n)$ each uniformly random in $[s]^n$ but so that $X$ and $Y$ are correlated . For each symbol $i$, we have that $Y_i = X_i$ with probability $1-\eps$ and otherwise $Y_i$ is chosen independently and uniformly from $[s]$. Alice and Bob wish to use their respective strings to extract a uniformly chosen common sequence from $[s]^k$ but without communicating. How well can they do? The trivial strategy of outputting the first $k$ symbols yields an agreement probability of $(1 - \eps + \eps/s)^k$. In a recent work by Bogdanov and Mossel it was shown that in the binary case where $s=2$ and $k = k(\eps)$ is large enough then it is possible to extract $k$ bits with a better agreement probability rate. In particular, it is possible to achieve agreement probability $(k\eps)^{-1/2} \cdot 2^{-k\eps/(2(1 - \eps/2))}$ using a random construction based on Hamming balls, and this is optimal up to lower order terms. In the current paper we consider the same problem over larger alphabet sizes $s$ and we show that the agreement probability rate changes dramatically as the alphabet grows. In particular we show no strategy can achieve agreement probability better than $(1-\eps)^k (1+δ(s))^k$ where $δ(s) \to 0$ as $s \to \infty$. We also show that Hamming ball based constructions have {\em much lower} agreement probability rate than the trivial algorithm as $s \to \infty$. Our proofs and results are intimately related to subtle properties of hypercontractive inequalities.

cs.IT↗

arXiv2012

Stochastic Block Models and Reconstruction

The planted partition model (also known as the stochastic blockmodel) is a classical cluster-exhibiting random graph model that has been extensively studied in statistics, physics, and computer science. In its simplest form, the planted partition model is a model for random graphs on $n$ nodes with two equal-sized clusters, with an between-class edge probability of $q$ and a within-class edge probability of $p$. Although most of the literature on this model has focused on the case of increasing degrees (ie.\ $pn, qn \to \infty$ as $n \to \infty$), the sparse case $p, q = O(1/n)$ is interesting both from a mathematical and an applied point of view. A striking conjecture of Decelle, Krzkala, Moore and Zdeborová based on deep, non-rigorous ideas from statistical physics gave a precise prediction for the algorithmic threshold of clustering in the sparse planted partition model. In particular, if $p = a/n$ and $q = b/n$, then Decelle et al.\ conjectured that it is possible to cluster in a way correlated with the true partition if $(a - b)^2 > 2(a + b)$, and impossible if $(a - b)^2 < 2(a + b)$. By comparison, the best-known rigorous result is that of Coja-Oghlan, who showed that clustering is possible if $(a - b)^2 > C (a + b)$ for some sufficiently large $C$. We prove half of their prediction, showing that it is indeed impossible to cluster if $(a - b)^2 < 2(a + b)$. Furthermore we show that it is impossible even to estimate the model parameters from the graph when $(a - b)^2 < 2(a + b)$; on the other hand, we provide a simple and efficient algorithm for estimating $a$ and $b$ when $(a - b)^2 > 2(a + b)$. Following Decelle et al, our work establishes a rigorous connection between the clustering problem, spin-glass models on the Bethe lattice and the so called reconstruction problem. This connection points to fascinating applications and open problems.

math.PR↗