arXiv subjects

Matthew Brennan

Publications and source records attributed to Matthew Brennan.

14 recordsLinked to original sources

arXiv2022

Threshold for Detecting High Dimensional Geometry in Anisotropic Random Geometric Graphs

In the anisotropic random geometric graph model, vertices correspond to points drawn from a high-dimensional Gaussian distribution and two vertices are connected if their distance is smaller than a specified threshold. We study when it is possible to hypothesis test between such a graph and an Erdős-Rényi graph with the same edge probability. If $n$ is the number of vertices and $α$ is the vector of eigenvalues, Eldan and Mikulincer show that detection is possible when $n^3 \gg (\|α\|_2/\|α\|_3)^6$ and impossible when $n^3 \ll (\|α\|_2/\|α\|_4)^4$. We show detection is impossible when $n^3 \ll (\|α\|_2/\|α\|_3)^6$, closing this gap and affirmatively resolving the conjecture of Eldan and Mikulincer.

math.ST↗

arXiv2021

The staircase property: How hierarchical structure can guide deep learning

This paper identifies a structural property of data distributions that enables deep neural networks to learn hierarchically. We define the "staircase" property for functions over the Boolean hypercube, which posits that high-order Fourier coefficients are reachable from lower-order Fourier coefficients along increasing chains. We prove that functions satisfying this property can be learned in polynomial time using layerwise stochastic coordinate descent on regular neural networks -- a class of network architectures and initializations that have homogeneity properties. Our analysis shows that for such staircase functions and neural networks, the gradient-based algorithm learns high-level features by greedily combining lower-level features along the depth of the network. We further back our theoretical results with experiments showing that staircase functions are also learnable by more standard ResNet architectures with stochastic gradient descent. Both the theoretical and experimental results support the fact that staircase properties have a role to play in understanding the capabilities of gradient-based learning on regular networks, in contrast to general polynomial-size networks that can emulate any SQ or PAC algorithms as recently shown.

cs.LG↗

arXiv2021

The Average-Case Complexity of Counting Cliques in Erdos-Renyi Hypergraphs

We consider the problem of counting $k$-cliques in $s$-uniform Erdos-Renyi hypergraphs $G(n,c,s)$ with edge density $c$, and show that its fine-grained average-case complexity can be based on its worst-case complexity. We prove the following: 1. Dense Erdos-Renyi graphs and hypergraphs: Counting $k$-cliques on $G(n,c,s)$ with $k$ and $c$ constant matches its worst-case time complexity up to a $\mathrm{polylog}(n)$ factor. Assuming randomized ETH, it takes $n^{Ω(k)}$ time to count $k$-cliques in $G(n,c,s)$ if $k$ and $c$ are constant. 2. Sparse Erdos-Renyi graphs and hypergraphs: When $c = Θ(n^{-α})$, we give several algorithms exploiting the sparsity of $G(n, c, s)$ that are faster than the best known worst-case algorithms. Complementing this, based on a fine-grained worst-case assumption, our results imply a different average-case phase diagram for each fixed $α$ depicting a tradeoff between a runtime lower bound and $k$. Surprisingly, in the hypergraph case ($s \ge 3$), these lower bounds are tight against our algorithms exactly when $c$ is above the Erdős-Rényi $k$-clique percolation threshold. This is the first worst-case-to-average-case hardness reduction for a problem on Erdős-Rényi hypergraphs that we are aware of. We also give a variant of our result for computing the parity of the $k$-clique count that tolerates higher error probability.

cs.CC↗

arXiv2021

Statistical Query Algorithms and Low-Degree Tests Are Almost Equivalent

Researchers currently use a number of approaches to predict and substantiate information-computation gaps in high-dimensional statistical estimation problems. A prominent approach is to characterize the limits of restricted models of computation, which on the one hand yields strong computational lower bounds for powerful classes of algorithms and on the other hand helps guide the development of efficient algorithms. In this paper, we study two of the most popular restricted computational models, the statistical query framework and low-degree polynomials, in the context of high-dimensional hypothesis testing. Our main result is that under mild conditions on the testing problem, the two classes of algorithms are essentially equivalent in power. As corollaries, we obtain new statistical query lower bounds for sparse PCA, tensor PCA and several variants of the planted clique problem.

cs.CC↗

arXiv2021

De Finetti-Style Results for Wishart Matrices: Combinatorial Structure and Phase Transitions

A recent line of work has studied the relationship between the Wishart matrix $X^\top X$, where $X\in \mathbb{R}^{d\times n}$ has i.i.d. standard Gaussian entries, and the corresponding Gaussian matrix with independent entries above the diagonal. Jiang and Li (2015) and Bubeck et al. (2016) showed that these two matrix ensembles converge in total variation whenever $d/n^3\to \infty$, and Bubeck et al. (2016) showed this to be sharp. In this paper we aim to identify the precise threshold for $d$ in terms of $n$ for subsets of Wishart matrices to converge in total variation to independent Gaussians. It turns out that the combinatorial structure of the revealed entries, viewed as the adjacency matrix of a graph $G$, characterizes the distance from fully independent. Specifically, we show that the threshold for $d$ depends on the number of various small subgraphs in $G$. So, even when the number of revealed entries is fixed, the threshold can vary wildly depending on their configuration. Convergence of masked Wishart to independent Gaussians thus inherently involves an interplay between both probabilistic and combinatorial phenomena. Our results determine the sharp threshold for a large family of $G$, including Erdős-Rényi $G\sim \mathcal{G}(n,p)$ at all values $p\gtrsim n^{-2}\mathrm{polylog}(n)$. Our proof techniques are both combinatorial and information theoretic, which together allow us to carefully unravel the dependencies in the masked Wishart ensemble.

math.PR↗

arXiv2020

Phase Transitions for Detecting Latent Geometry in Random Graphs

Random graphs with latent geometric structure are popular models of social and biological networks, with applications ranging from network user profiling to circuit design. These graphs are also of purely theoretical interest within computer science, probability and statistics. A fundamental initial question regarding these models is: when are these random graphs affected by their latent geometry and when are they indistinguishable from simpler models without latent structure, such as the Erdős-Rényi graph $\mathcal{G}(n, p)$? We address this question for two of the most well-studied models of random graphs with latent geometry -- the random intersection and random geometric graph. Our results are as follows: (1) we prove that the random intersection graph converges in total variation to $\mathcal{G}(n, p)$ when $d = \tildeω(n^3)$, and does not if $d = o(n^3)$, resolving an open problem in Fill et al. (2000), Rybarczyk (2011) and Kim et al. (2018); (2) we provide conditions under which the matrix of intersection sizes of random family of sets converges in total variation to a symmetric matrix with independent Poisson entries, yielding the first total variation convergence result for $τ$-random intersection graphs to $\mathcal{G}(n, p)$; and (3) we show that the random geometric graph on $\mathbb{S}^{d - 1}$ with edge density $p$ converges in total variation to $\mathcal{G}(n, p)$ when $d = \tildeω\left(\min\{ pn^3, p^2 n^{7/2} \} \right)$, yielding the first progress towards a conjecture of Bubeck et al. (2016). The first of these three results was obtained simultaneously and independently by Bubeck, Racz and Richey.

math.PR↗

arXiv2020

Reducibility and Statistical-Computational Gaps from Secret Leakage

Inference problems with conjectured statistical-computational gaps are ubiquitous throughout modern statistics, computer science and statistical physics. While there has been success evidencing these gaps from the failure of restricted classes of algorithms, progress towards a more traditional reduction-based approach to computational complexity in statistical inference has been limited. Existing reductions have largely been limited to inference problems with similar structure -- primarily mapping among problems representable as a sparse submatrix signal plus a noise matrix, which are similar to the common hardness assumption of planted clique. The insight in this work is that a slight generalization of the planted clique conjecture -- secret leakage planted clique -- gives rise to a variety of new average-case reduction techniques, yielding a web of reductions among problems with very different structure. Using variants of the planted clique conjecture for specific forms of secret leakage planted clique, we deduce tight statistical-computational tradeoffs for a diverse range of problems including robust sparse mean estimation, mixtures of sparse linear regressions, robust sparse linear regression, tensor PCA, variants of dense $k$-block stochastic block models, negatively correlated sparse PCA, semirandom planted dense subgraph, detection in hidden partition models and a universality principle for learning sparse mixtures. In particular, a $k$-partite hypergraph variant of the planted clique conjecture is sufficient to establish all of our computational lower bounds. Our techniques also reveal novel connections to combinatorial designs and to random matrix theory. This work gives the first evidence that an expanded set of hardness assumptions, such as for secret leakage planted clique, may be a key first step towards a more complete theory of reductions among statistical problems.

cs.CC↗

arXiv2020

Average-Case Lower Bounds for Learning Sparse Mixtures, Robust Estimation and Semirandom Adversaries

This paper develops several average-case reduction techniques to show new hardness results for three central high-dimensional statistics problems, implying a statistical-computational gap induced by robustness, a detection-recovery gap and a universality principle for these gaps. A main feature of our approach is to map to these problems via a common intermediate problem that we introduce, which we call Imbalanced Sparse Gaussian Mixtures. We assume the planted clique conjecture for a version of the planted clique problem where the position of the planted clique is mildly constrained, and from this obtain the following computational lower bounds: (1) a $k$-to-$k^2$ statistical-computational gap for robust sparse mean estimation, providing the first average-case evidence for a conjecture of Li (2017) and Balakrishnan et al. (2017); (2) a tight lower bound for semirandom planted dense subgraph, which shows that a semirandom adversary shifts the detection threshold in planted dense subgraph to the conjectured recovery threshold; and (3) a universality principle for $k$-to-$k^2$ gaps in a broad class of sparse mixture problems that includes many natural formulations such as the spiked covariance model. Our main approach is to introduce several average-case techniques to produce structured and Gaussianized versions of an input graph problem, and then to rotate these high-dimensional Gaussians by matrices carefully constructed from hyperplanes in $\mathbb{F}_r^t$. For our universality result, we introduce a new method to perform an algorithmic change of measure tailored to sparse mixtures. We also provide evidence that the mild promise in our variant of planted clique does not change the complexity of the problem.

cs.CC↗

arXiv2019

Ramsey numbers of trees and unicyclic graphs versus fans

The generalized Ramsey number $R(H, K)$ is the smallest positive integer $n$ such that for any graph $G$ with $n$ vertices either $G$ contains $H$ as a subgraph or its complement $\overline{G}$ contains $K$ as a subgraph. Let $T_n$ be a tree with $n$ vertices and $F_m$ be a fan with $2m + 1$ vertices consisting of $m$ triangles sharing a common vertex. We prove a conjecture of Zhang, Broersma and Chen for $m \ge 9$ that $R(T_n, F_m) = 2n - 1$ for all $n \ge m^2 - m + 1$. Zhang, Broersma and Chen showed that $R(S_n, F_m) \ge 2n$ for $n \le m^2 -m$ where $S_n$ is a star on $n$ vertices, implying that the lower bound we show is in some sense tight. We also extend this result to unicyclic graphs $UC_n$, which are connected graphs with $n$ vertices and a single cycle. We prove that $R(UC_n, F_m) = 2n - 1$ for all $n \ge m^2 - m + 1$ where $m \ge 18$. In proving this conjecture and extension, we present several methods for embedding trees in graphs, which may be of independent interest.

math.CO↗

arXiv2019

Ramsey numbers of trees versus odd cycles

Burr, Erdős, Faudree, Rousseau and Schelp initiated the study of Ramsey numbers of trees versus odd cycles, proving that $R(T_n, C_m) = 2n - 1$ for all odd $m \ge 3$ and $n \ge 756m^{10}$, where $T_n$ is a tree with $n$ vertices and $C_m$ is an odd cycle of length $m$. They proposed to study the minimum positive integer $n_0(m)$ such that this result holds for all $n \ge n_0(m)$, as a function of $m$. In this paper, we show that $n_0(m)$ is at most linear. In particular, we prove that $R(T_n, C_m) = 2n - 1$ for all odd $m \ge 3$ and $n \ge 50m$. Combining this with a result of Faudree, Lawrence, Parsons and Schelp yields $n_0(m)$ is bounded between two linear functions, thus identifying $n_0(m)$ up to a constant factor.

math.CO↗

arXiv2019

Reducibility and Computational Lower Bounds for Problems with Planted Sparse Structure

The prototypical high-dimensional statistics problem entails finding a structured signal in noise. Many of these problems exhibit an intriguing phenomenon: the amount of data needed by all known computationally efficient algorithms far exceeds what is needed for inefficient algorithms that search over all possible structures. A line of work initiated by Berthet and Rigollet in 2013 has aimed to explain these statistical-computational gaps by reducing from conjecturally hard average-case problems in computer science. However, the delicate nature of average-case reductions has limited the applicability of this approach. In this work we introduce several new techniques to give a web of average-case reductions showing strong computational lower bounds based on the planted clique conjecture using natural problems as intermediates. These include tight lower bounds for Planted Independent Set, Planted Dense Subgraph, Sparse Spiked Wigner, Sparse PCA, a subgraph variant of the Stochastic Block Model and a biased variant of Sparse PCA. We also give algorithms matching our lower bounds and identify the information-theoretic limits of the models we consider.

cs.CC↗

arXiv2019

Universality of Computational Lower Bounds for Submatrix Detection

In the general submatrix detection problem, the task is to detect the presence of a small $k \times k$ submatrix with entries sampled from a distribution $\mathcal{P}$ in an $n \times n$ matrix of samples from $\mathcal{Q}$. This formulation includes a number of well-studied problems, such as biclustering when $\mathcal{P}$ and $\mathcal{Q}$ are Gaussians and the planted dense subgraph formulation of community detection when the submatrix is a principal minor and $\mathcal{P}$ and $\mathcal{Q}$ are Bernoulli random variables. These problems all seem to exhibit a universal phenomenon: there is a statistical-computational gap depending on $\mathcal{P}$ and $\mathcal{Q}$ between the minimum $k$ at which this task can be solved and the minimum $k$ at which it can be solved in polynomial time. Our main result is to tightly characterize this computational barrier as a tradeoff between $k$ and the KL divergences between $\mathcal{P}$ and $\mathcal{Q}$ through average-case reductions from the planted clique conjecture. These computational lower bounds hold given mild assumptions on $\mathcal{P}$ and $\mathcal{Q}$ arising naturally from classical binary hypothesis testing. Our results recover and generalize the planted clique lower bounds for Gaussian biclustering in Ma-Wu (2015) and Brennan et al. (2018) and for the sparse and general regimes of planted dense subgraph in Hajek et al. (2015) and Brennan et al. (2018). This yields the first universality principle for computational lower bounds obtained through average-case reductions.

math.ST↗

arXiv2019

Optimal Average-Case Reductions to Sparse PCA: From Weak Assumptions to Strong Hardness

In the past decade, sparse principal component analysis has emerged as an archetypal problem for illustrating statistical-computational tradeoffs. This trend has largely been driven by a line of research aiming to characterize the average-case complexity of sparse PCA through reductions from the planted clique (PC) conjecture - which conjectures that there is no polynomial-time algorithm to detect a planted clique of size $K = o(N^{1/2})$ in $\mathcal{G}(N, \frac{1}{2})$. All previous reductions to sparse PCA either fail to show tight computational lower bounds matching existing algorithms or show lower bounds for formulations of sparse PCA other than its canonical generative model, the spiked covariance model. Also, these lower bounds all quickly degrade with the exponent in the PC conjecture. Specifically, when only given the PC conjecture up to $K = o(N^α)$ where $α< 1/2$, there is no sparsity level $k$ at which these lower bounds remain tight. If $α\le 1/3$ these reductions fail to even show the existence of a statistical-computational tradeoff at any sparsity $k$. We give a reduction from PC that yields the first full characterization of the computational barrier in the spiked covariance model, providing tight lower bounds at all sparsities $k$. We also show the surprising result that weaker forms of the PC conjecture up to clique size $K = o(N^α)$ for any given $α\in (0, 1/2]$ imply tight computational lower bounds for sparse PCA at sparsities $k = o(n^{α/3})$. This shows that even a mild improvement in the signal strength needed by the best known polynomial-time sparse PCA algorithms would imply that the hardness threshold for PC is subpolynomial. This is the first instance of a suboptimal hardness assumption implying optimal lower bounds for another problem in unsupervised learning.

cs.CC↗

arXiv2015

Cell-cell communication enhances the capacity of cell ensembles to sense shallow gradients during morphogenesis

Collective cell responses to exogenous cues depend on cell-cell interactions. In principle, these can result in enhanced sensitivity to weak and noisy stimuli. However, this has not yet been shown experimentally, and, little is known about how multicellular signal processing modulates single cell sensitivity to extracellular signaling inputs, including those guiding complex changes in the tissue form and function. Here we explored if cell-cell communication can enhance the ability of cell ensembles to sense and respond to weak gradients of chemotactic cues. Using a combination of experiments with mammary epithelial cells and mathematical modeling, we find that multicellular sensing enables detection of and response to shallow Epidermal Growth Factor (EGF) gradients that are undetectable by single cells. However, the advantage of this type of gradient sensing is limited by the noisiness of the signaling relay, necessary to integrate spatially distributed ligand concentration information. We calculate the fundamental sensory limits imposed by this communication noise and combine them with the experimental data to estimate the effective size of multicellular sensory groups involved in gradient sensing. Functional experiments strongly implicated intercellular communication through gap junctions and calcium release from intracellular stores as mediators of collective gradient sensing. The resulting integrative analysis provides a framework for understanding the advantages and limitations of sensory information processing by relays of chemically coupled cells.

q-bio.CB↗