arXiv subjects

Houssam El Cheairi

Publications and source records attributed to Houssam El Cheairi.

3 recordsLinked to original sources

arXiv2025

Algorithmic Universality, Low-Degree Polynomials, and Max-Cut in Sparse Random Graphs

Universality, namely distributional invariance, is a well-known property for many random structures. For example, it is known to hold for a broad range of variational problems with random input. Much less is known about the algorithmic universality of specific methods for solving such variational problems. Namely, whether algorithms tuned to specific variational tasks produce the same asymptotic behavior across different input distributions with matching moments. In this paper, we establish algorithmic universality for a class of models, which includes spin glass models and constraint satisfaction problems on sparse graphs, provided that an algorithm can be coded as a low-degree polynomial (LDP). We illustrate this specifically for the case of the Max-Cut problem in sparse Erdös-Rényi graph $\mathbb{G}(n,d/n)$. We use the fact that the Approximate Message Passing (AMP) algorithm, which is an effective algorithm for finding near-ground states of the Sherrington-Kirkpatrick (SK) model, is well approximated by an LDP. We then establish our main universality result: the performance of the LDP based algorithms exhibiting a certain connectivity property, is the same in the mean-field (SK) and in the random graph $\mathbb{G}(n,d/n)$ setting, up to an appropriate rescaling. The main technical challenge we address in this paper is showing that the output of an LDP algorithm on $\mathbb{G}(n,d/n)$ is truly discrete, namely, that it is close to the set of points in the binary cube. This is achieved by establishing universality of coordinate-wise statistics of the LDP output across disorder ensembles, which implies that proximity to the cube transfers from the Gaussian to the sparse graph setting.

cs.DS↗

arXiv2025

Theoretical Compression Bounds for Wide Multilayer Perceptrons

Pruning and quantization techniques have been broadly successful in reducing the number of parameters needed for large neural networks, yet theoretical justification for their empirical success falls short. We consider a randomized greedy compression algorithm for pruning and quantization post-training and use it to rigorously show the existence of pruned/quantized subnetworks of multilayer perceptrons (MLPs) with competitive performance. We further extend our results to structured pruning of MLPs and convolutional neural networks (CNNs), thus providing a unified analysis of pruning in wide networks. Our results are free of data assumptions, and showcase a tradeoff between compressibility and network width. The algorithm we consider bears some similarities with Optimal Brain Damage (OBD) and can be viewed as a post-training randomized version of it. The theoretical results we derive bridge the gap between theory and application for pruning/quantization, and provide a justification for the empirical success of compression in wide multilayer perceptrons.

cs.LG↗

arXiv2022

Densest Subgraphs of a Dense Erdös-Rényi Graph. Asymptotics, Landscape and Universality

We consider the problem of estimating the edge density of densest $K$-node subgraphs of an Erdös-Rényi graph $\mathbb{G}(n,1/2)$. The problem is well-understood in the regime $K=Θ(\log n)$ and in the regime $K=Θ(n)$. In the former case it can be reduced to the problem of estimating the size of largest cliques, and its extensions. In the latter case the full answer is known up to the order $n^{3\over 2}$ using sophisticated methods from the theory of spin glasses. The intermediate case $K=n^α, α\in (0,1)$ however is not well studied and this is our focus. We establish that that in this regime the density (that is the maximum number of edges supported by any $K$-node subgraph) is ${1\over 4}K^2+{1+o(1)\over 2}K^{3\over 2}\sqrt{\log (n/K)}$, w.h.p. as $n\to\infty$, and provide more refined asymptotics under the $o(\cdot)$, for various ranges of $α$. This extends earlier similar results where this asymptotics was confirmed only when $α$ is a small constant. We extend our results to the case of ''weighted'' graphs, when the weights have either Gaussian or arbitrary sub-Gaussian distributions. The proofs are based on the second moment method combined with concentration bounds, the Borell-TIS inequality for the Gaussian case and the Talagrand's inequality for the case of distributions with bounded support (including the $\mathbb{G}(n,1/2)$ case). The case of general distribution is treated using a novel symmetrized version of the Lindeberg argument, which reduces the general case to the Gaussian case. Finally, using the results above we conduct the landscape analysis of the related Hidden Clique Problem, and establish that it exhibits an overlap gap property when the size of the clique is $O(n^{2\over 3})$, confirming a hypothesis stated in a previous related work.

math.PR↗