arXiv subjects

Sidhanth Mohanty

Publications and source records attributed to Sidhanth Mohanty.

30 records · Page 2Linked to original sources

arXiv2021

On statistical inference when fixed points of belief propagation are unstable

Many statistical inference problems correspond to recovering the values of a set of hidden variables from sparse observations on them. For instance, in a planted constraint satisfaction problem such as planted 3-SAT, the clauses are sparse observations from which the hidden assignment is to be recovered. In the problem of community detection in a stochastic block model, the community labels are hidden variables that are to be recovered from the edges of the graph. Inspired by ideas from statistical physics, the presence of a stable fixed point for belief propogation has been widely conjectured to characterize the computational tractability of these problems. For community detection in stochastic block models, many of these predictions have been rigorously confirmed. In this work, we consider a general model of statistical inference problems that includes both community detection in stochastic block models, and all planted constraint satisfaction problems as special cases. We carry out the cavity method calculations from statistical physics to compute the regime of parameters where detection and recovery should be algorithmically tractable. At precisely the predicted tractable regime, we give: (i) a general polynomial-time algorithm for the problem of detection: distinguishing an input with a planted signal from one without; (ii) a general polynomial-time algorithm for the problem of recovery: outputting a vector that correlates with the hidden assignment significantly better than a random guess would.

cs.DS↗

arXiv2021

Certifying solution geometry in random CSPs: counts, clusters and balance

An active topic in the study of random constraint satisfaction problems (CSPs) is the geometry of the space of satisfying or almost satisfying assignments as the function of the density, for which a precise landscape of predictions has been made via statistical physics-based heuristics. In parallel, there has been a recent flurry of work on refuting random constraint satisfaction problems, via nailing refutation thresholds for spectral and semidefinite programming-based algorithms, and also on counting solutions to CSPs. Inspired by this, the starting point for our work is the following question: what does the solution space for a random CSP look like to an efficient algorithm? In pursuit of this inquiry, we focus on the following problems about random Boolean CSPs at the densities where they are unsatisfiable but no refutation algorithm is known. 1. Counts. For every Boolean CSP we give algorithms that with high probability certify a subexponential upper bound on the number of solutions. We also give algorithms to certify a bound on the number of large cuts in a Gaussian-weighted graph, and the number of large independent sets in a random $d$-regular graph. 2. Clusters. For Boolean $3$CSPs we give algorithms that with high probability certify an upper bound on the number of clusters of solutions. 3. Balance. We also give algorithms that with high probability certify that there are no "unbalanced" solutions, i.e., solutions where the fraction of $+1$s deviates significantly from $50\%$. Finally, we also provide hardness evidence suggesting that our algorithms for counting are optimal.

cs.DS↗

arXiv2021

High-girth near-Ramanujan graphs with lossy vertex expansion

Kahale proved that linear sized sets in $d$-regular Ramanujan graphs have vertex expansion $\sim\frac{d}{2}$ and complemented this with construction of near-Ramanujan graphs with vertex expansion no better than $\frac{d}{2}$. However, the construction of Kahale encounters highly local obstructions to better vertex expansion. In particular, the poorly expanding sets are associated with short cycles in the graph. Thus, it is natural to ask whether high-girth Ramanujan graphs have improved vertex expansion. Our results are two-fold: 1. For every $d = p+1$ for prime $p$ and infinitely many $n$, we exhibit an $n$-vertex $d$-regular graph with girth $Ω(\log_{d-1} n)$ and vertex expansion of sublinear sized sets bounded by $\frac{d+1}{2}$ whose nontrivial eigenvalues are bounded in magnitude by $2\sqrt{d-1}+O\left(\frac{1}{\log n}\right)$. 2. In any Ramanujan graph with girth $C\log n$, all sets of size bounded by $n^{0.99C/4}$ have vertex expansion $(1-o_d(1))d$. The tools in analyzing our construction include the nonbacktracking operator of an infinite graph, the Ihara--Bass formula, a trace moment method inspired by Bordenave's proof of Friedman's theorem, and a method of Kahale to study dispersion of eigenvalues of perturbed graphs.

math.CO↗

arXiv2021

List Decodable Mean Estimation in Nearly Linear Time

Learning from data in the presence of outliers is a fundamental problem in statistics. Until recently, no computationally efficient algorithms were known to compute the mean of a high dimensional distribution under natural assumptions in the presence of even a small fraction of outliers. In this paper, we consider robust statistics in the presence of overwhelming outliers where the majority of the dataset is introduced adversarially. With only an $α< 1/2$ fraction of "inliers" (clean data) the mean of a distribution is unidentifiable. However, in their influential work, [CSV17] introduces a polynomial time algorithm recovering the mean of distributions with bounded covariance by outputting a succinct list of $O(1/α)$ candidate solutions, one of which is guaranteed to be close to the true distributional mean; a direct analog of 'List Decoding' in the theory of error correcting codes. In this work, we develop an algorithm for list decodable mean estimation in the same setting achieving up to constants the information theoretically optimal recovery, optimal sample complexity, and in nearly linear time up to polylogarithmic factors in dimension. Our conceptual innovation is to design a descent style algorithm on a nonconvex landscape, iteratively removing minima to generate a succinct list of solutions. Our runtime bottleneck is a saddle-point optimization for which we design custom primal dual solvers for generalized packing and covering SDP's under Ky-Fan norms, which may be of independent interest.

cs.DS↗

arXiv2020

Local Statistics, Semidefinite Programming, and Community Detection

We propose a new hierarchy of semidefinite programming relaxations for inference problems. As test cases, we consider the problem of community detection in block models. The vertices are partitioned into $k$ communities, and a graph is sampled conditional on a prescribed number of inter- and intra-community edges. The problem of detection, where we are to decide with high probability whether a graph was drawn from this model or the uniform distribution on regular graphs, is conjectured to undergo a computational phase transition at a point called the Kesten-Stigum (KS) threshold. In this work, we consider two models of random graphs namely the well-studied (irregular) stochastic block model and a distribution over random regular graphs we'll call the Degree Regular Block Model. For both these models, we show that sufficiently high constant levels of our hierarchy can perform detection arbitrarily close to the KS threshold and that our algorithm is robust to up to a linear number of adversarial edge perturbations. Furthermore, in the case of Degree Regular Block Model (DRBM), we show that below the Kesten-Stigum threshold no constant level can do so. In the case of the (irregular) Stochastic Block Model, it is known that efficient algorithms exist all the way down to this threshold, although none are robust to a linear number of adversarial perturbations of the graph when the average degree is small. More importantly, there is little complexity-theoretic evidence that detection is hard below the threshold. In the DRBM with more than two groups, it has not to our knowledge been proven that any algorithm succeeds down to the KS threshold, let alone that one can do so robustly, and there is a similar dearth of evidence for hardness below this point.

cs.DS↗

arXiv2019

Pseudo-deterministic Streaming

A pseudo-deterministic algorithm is a (randomized) algorithm which, when run multiple times on the same input, with high probability outputs the same result on all executions. Classic streaming algorithms, such as those for finding heavy hitters, approximate counting, $\ell_2$ approximation, finding a nonzero entry in a vector (for turnstile algorithms) are not pseudo-deterministic. For example, in the instance of finding a nonzero entry in a vector, for any known low-space algorithm $A$, there exists a stream $x$ so that running $A$ twice on $x$ (using different randomness) would with high probability result in two different entries as the output. In this work, we study whether it is inherent that these algorithms output different values on different executions. That is, we ask whether these problems have low-memory pseudo-deterministic algorithms. For instance, we show that there is no low-memory pseudo-deterministic algorithm for finding a nonzero entry in a vector (given in a turnstile fashion), and also that there is no low-dimensional pseudo-deterministic sketching algorithm for $\ell_2$ norm estimation. We also exhibit problems which do have low memory pseudo-deterministic algorithms but no low memory deterministic algorithm, such as outputting a nonzero row of a matrix, or outputting a basis for the row-span of a matrix. We also investigate multi-pseudo-deterministic algorithms: algorithms which with high probability output one of a few options. We show the first lower bounds for such algorithms. This implies that there are streaming problems such that every low space algorithm for the problem must have inputs where there are many valid outputs, all with a significant probability of being outputted.

cs.CC↗

arXiv2019

High-Dimensional Expanders from Expanders

We present an elementary way to transform an expander graph into a simplicial complex where all high order random walks have a constant spectral gap, i.e., they converge rapidly to the stationary distribution. As an upshot, we obtain new constructions, as well as a natural probabilistic model to sample constant degree high-dimensional expanders. In particular, we show that given an expander graph $G$, adding self loops to $G$ and taking the tensor product of the modified graph with a high-dimensional expander produces a new high-dimensional expander. Our proof of rapid mixing of high order random walks is based on the decomposable Markov chains framework introduced by Jerrum et al.

cs.DM↗

arXiv2019

Lifting Sum-of-Squares Lower Bounds: Degree-$2$ to Degree-$4$

The degree-$4$ Sum-of-Squares (SoS) SDP relaxation is a powerful algorithm that captures the best known polynomial time algorithms for a broad range of problems including MaxCut, Sparsest Cut, all MaxCSPs and tensor PCA. Despite being an explicit algorithm with relatively low computational complexity, the limits of degree-$4$ SoS SDP are not well understood. For example, existing integrality gaps do not rule out a $(2-\varepsilon)$-algorithm for Vertex Cover or a $(0.878+\varepsilon)$-algorithm for MaxCut via degree-$4$ SoS SDPs, each of which would refute the notorious Unique Games Conjecture. We exhibit an explicit mapping from solutions for degree-$2$ Sum-of-Squares SDP (Goemans-Williamson SDP) to solutions for the degree-$4$ Sum-of-Squares SDP relaxation on boolean variables. By virtue of this mapping, one can lift lower bounds for degree-$2$ SoS SDP relaxation to corresponding lower bounds for degree-$4$ SoS SDPs. We use this approach to obtain degree-$4$ SoS SDP lower bounds for MaxCut on random $d$-regular graphs, Sherington-Kirkpatrick model from statistical physics and PSD Grothendieck problem. Our constructions use the idea of pseudocalibration towards candidate SDP vectors, while it was previously only used to produce the candidate matrix which one would show is PSD using much technical work. In addition, we develop a different technique to bound the spectral norms of _graphical matrices_ that arise in the context of SoS SDPs. The technique is much simpler and yields better bounds in many cases than the _trace method_ -- which was the sole technique for this purpose.

cs.CC↗

arXiv2019

The SDP value for random two-eigenvalue CSPs

We precisely determine the SDP value (equivalently, quantum value) of large random instances of certain kinds of constraint satisfaction problems, ``two-eigenvalue 2CSPs''. We show this SDP value coincides with the spectral relaxation value, possibly indicating a computational threshold. Our analysis extends the previously resolved cases of random regular $\mathsf{2XOR}$ and $\textsf{NAE-3SAT}$, and includes new cases such as random $\mathsf{Sort}_4$ (equivalently, $\mathsf{CHSH}$) and $\mathsf{Forrelation}$ CSPs. Our techniques include new generalizations of the nonbacktracking operator, the Ihara--Bass Formula, and the Friedman/Bordenave proof of Alon's Conjecture.

cs.DS↗

arXiv2019

$X$-Ramanujan Graphs

Let $X$ be an infinite graph of bounded degree; e.g., the Cayley graph of a free product of finite groups. If $G$ is a finite graph covered by $X$, it is said to be $X$-Ramanujan if its second-largest eigenvalue $λ_2(G)$ is at most the spectral radius $ρ(X)$ of $X$, and more generally $k$-quasi-$X$-Ramanujan if $λ_k(G)$ is at most $ρ(X)$. In case $X$ is the infinite $Δ$-regular tree, this reduces to the well known notion of a finite $Δ$-regular graph being Ramanujan. Inspired by the Interlacing Polynomials method of Marcus, Spielman, and Srivastava, we show the existence of infinitely many $k$-quasi-$X$-Ramanujan graphs for a variety of infinite $X$. In particular, $X$ need not be a tree; our analysis is applicable whenever $X$ is what we call an additive product graph. This additive product is a new construction of an infinite graph $\mathsf{AddProd}(A_1, \dots, A_c)$ from finite 'atom' graphs $A_1, \dots, A_c$ over a common vertex set. It generalizes the notion of the free product graph $A_1 * \cdots * A_c$ when the atoms $A_j$ are vertex-transitive, and it generalizes the notion of the universal covering tree when the atoms $A_j$ are single-edge graphs. Key to our analysis is a new graph polynomial $α(A_1, \dots, A_c;x)$ that we call the additive characteristic polynomial. It generalizes the well known matching polynomial $μ(G;x)$ in case the atoms $A_j$ are the single edges of $G$, and it generalizes the $r$-characteristic polynomial introduced in [Ravichandran'16, Leake-Ravichandran'18]. We show that $α(A_1, \dots, A_c;x)$ is real-rooted, and all of its roots have magnitude at most $ρ(\mathsf{AddProd}(A_1, \dots, A_c))$. This last fact is proven by generalizing Godsil's notion of treelike walks on a graph $G$ to a notion of freelike walks on a collection of atoms $A_1, \dots, A_c$.

math.CO↗

arXiv2018

Algorithms for Noisy Broadcast under Erasures

The noisy broadcast model was first studied in [Gallager, TranInf'88] where an $n$-character input is distributed among $n$ processors, so that each processor receives one input bit. Computation proceeds in rounds, where in each round each processor broadcasts a single character, and each reception is corrupted independently at random with some probability $p$. [Gallager, TranInf'88] gave an algorithm for all processors to learn the input in $O(\log\log n)$ rounds with high probability. Later, a matching lower bound of $Ω(\log\log n)$ was given in [Goyal, Kindler, Saks; SICOMP'08]. We study a relaxed version of this model where each reception is erased and replaced with a `?' independently with probability $p$. In this relaxed model, we break past the lower bound of [Goyal, Kindler, Saks; SICOMP'08] and obtain an $O(\log^* n)$-round algorithm for all processors to learn the input with high probability. We also show an $O(1)$-round algorithm for the same problem when the alphabet size is $Ω(\mathrm{poly}(n))$.

cs.DS↗

arXiv2018

On Sketching the $q$ to $p$ norms

We initiate the study of data dimensionality reduction, or sketching, for the $q\to p$ norms. Given an $n \times d$ matrix $A$, the $q\to p$ norm, denoted $\|A\|_{q \to p} = \sup_{x \in \mathbb{R}^d \backslash \vec{0}} \frac{\|Ax\|_p}{\|x\|_q}$, is a natural generalization of several matrix and vector norms studied in the data stream and sketching models, with applications to datamining, hardness of approximation, and oblivious routing. We say a distribution $S$ on random matrices $L \in \mathbb{R}^{nd} \rightarrow \mathbb{R}^k$ is a $(k,α)$-sketching family if from $L(A)$, one can approximate $\|A\|_{q \to p}$ up to a factor $α$ with constant probability. We provide upper and lower bounds on the sketching dimension $k$ for every $p, q \in [1, \infty]$, and in a number of cases our bounds are tight. While we mostly focus on constant $α$, we also consider large approximation factors $α$, as well as other variants of the problem such as when $A$ has low rank.

cs.DS↗