Searcharxiv⌕ Search

arXiv subjects

Satyaki Mukherjee

Publications and source records attributed to Satyaki Mukherjee.

14 recordsLinked to original sources
arXiv2026

Beyond Worst-Case Coreset Bounds for $k$-Clustering via Determinantal Sampling

Massive datasets in modern machine learning have made data reduction a central challenge, particularly for clustering tasks where memory and computational constraints demand compact yet faithful summaries. A standard approach is to construct an \textit{$ε$-coreset}: a small weighted subset that approximately preserves the clustering cost for every plausible choice of centers. For the \textit{$(k,z)$-clustering problem}, existing worst-case bounds on coreset size are essentially tight, ruling out substantially smaller coresets in general. However, such worst-case instances are often unrepresentative of real-world data. In this work, we show that significantly smaller coresets are possible under mild and natural assumptions on the underlying data distribution. We introduce a new correlated sampling framework, called \textit{determinantal sampling}, based on a novel application of determinantal point processes. Using this framework, we obtain an efficiently constructible $\varepsilon$-coreset for $(k,z)$-clustering in $\mathbb R^d$ whose dependence on $1/\varepsilon$ has exponent strictly smaller than $2$ when $d$ is fixed. This improves over the worst-case $\varepsilon^{-2}$ barrier under our beyond-worst-case assumptions. To the best of our knowledge, this is the first result that provably surpasses these lower bounds through beyond-worst-case assumptions. Finally, we validate our approach on synthetic and real-world benchmark datasets, where it consistently achieves smaller coresets than existing state-of-the-art methods, even without explicitly enforcing the assumptions used in the analysis.

stat.ML↗
arXiv2026

Gaussian mixtures and non-parametric likelihoods through the lens of statistical mechanics

In this work, we investigate Gaussian Mixture Models ({\it abbrv} GMM) and the related problem of non parametric maximum likelihood estimation ({\it abbrv} NPMLE) from the perspective of statistical mechanics. In particular, we establish stability guarantees for the NPMLE procedure that extend well beyond the state of the art. Crucially, we obtain guarantees on the Kullback-Leibler divergence between NPMLE estimators and the ground truth, a type of result which has been known to be challenging in the literature on this problem. In particular, we provide high probability upper bounds on the KL divergence between the NPMLE and the true density that are of the order of $\min\big\{\frac{(\log n)^{d+2}}{n} , \frac{\log n}{\sqrt n}\big\}$, which cover a wide range of scenarios for the comparative sizes of $n$ and $d$. We obtain similar guarantees for approximate solutions to the NPMLE problem, addressing realistic situations wherein optimization algorithms need to be stopped in finite time, allowing access only to approximations to the true NPMLE. A cornerstone of our approach is an analysis of the function class complexity of logarithms of gaussian mixture densities, which is able to handle their unboundedness, and could be of wider interest. Our methods lead to novel confidence-interval guarantees for entropy estimation in Gaussian mixtures, demonstrating their wider impact. We also establish correspondences between stability phenomena in the NPMLE problem and concepts such as chaos and multiple valleys in random energy landscapes of statistical mechanics models. While these correspondences are largely of a conceptual nature at this point, we believe that these connections, especially those with concentration phenomena and Langevin dynamics, may be developed into a toolbox for studying a wide variety of random optimization problems in statistics and machine learning.

math.ST↗
arXiv2026

Beyond Linear and Overcomplete Regimes: A Mean-Field Analysis of Bottleneck Autoencoders

Autoencoders (AEs) learn low-dimensional representations by mapping data into a latent space while minimizing reconstruction error. Despite their empirical success, theoretical understanding remains limited and largely restricted to linear models or settings without a bottleneck. In this work, we study nonlinear AEs with a fixed finite-dimensional bottleneck in the mean-field (MF) regime. We derive explicit MF learning dynamics for both encoder and decoder, providing a tractable characterization of training in the nonlinear setting. We show that, over finite time horizons, the empirical risk of finite-width networks trained with stochastic gradient descent closely tracks the MF risk trajectory with high probability. At optimality, we further establish that the finite-width risk converges to the MF optimum, demonstrating that finite networks are sufficiently expressive to approximate the infinite-width solution.

cs.LG↗
arXiv2026

Approximate FKG inequalities for phase-bound spin systems, with applications to central limit theorems for exponential random graphs

The Fortuin-Kasteleyn-Ginibre (FKG) inequality is an invaluable tool in monotone spin systems satisfying the FKG lattice condition, which provides positive correlations for all coordinate-wise increasing functions of spins. This inequality has numerous applications and plays an integral role in the proof of various central limit theorems (CLTs), including recent work on ferromagnetic exponential random graph models (ERGMs) wherein a Hamiltonian tilt promotes the presence of small subgraphs like triangles. However, the FKG lattice condition fails to hold when confining a spin system to a particular phase in the low-temperature regime of parameters. Thus it is not a priori clear if each phase internally has positive correlations for increasing functions, or if the positive correlations in the overall model (which is a mixture of phases) arise primarily from the global choice of phase. In this article, we show that the individual phases in ERGMs do indeed satisfy an approximate form of the FKG inequality internally. We use this to finish the proof of various CLTs within each individual phase in the phase-coexistence regime, answering a question posed by Bianchi, Collet, and Magnanini. We present the FKG inequality for ERGMs as a consequence of a more general result which holds under certain inputs related to metastable mixing; we expect this general result to be widely applicable, and we devote a section to spelling out the details of its application to a class of generalized higher-order ferromagnetic Curie-Weiss models where the necessary inputs are relatively transparent.

math.PR↗
arXiv2025

Recovering Imbalanced Clusters via Gradient-Based Projection Pursuit

Projection Pursuit is a classic exploratory technique for finding interesting projections of a dataset. We propose a method for recovering projections containing either Imbalanced Clusters or a Bernoulli-Rademacher distribution using a gradient-based technique to optimize the projection index. As sample complexity is a major limiting factor in Projection Pursuit, we analyze our algorithm's sample complexity within a Planted Vector setting where we can observe that Imbalanced Clusters can be recovered more easily than balanced ones. Additionally, we give a generalized result that works for a variety of data distributions and projection indices. We compare these results to computational lower bounds in the Low-Degree-Polynomial Framework. Finally, we experimentally evaluate our method's applicability to real-world data using FashionMNIST and the Human Activity Recognition Dataset, where our algorithm outperforms others when only a few samples are available.

cs.LG↗
arXiv2023

On a generalisation of the coupon collector problem

We consider a generalisation of the classical coupon collector problem. We define a super-coupon to be any $s$-subset of a universe of $n$ coupons. In each round, a random $r$-subset from the universe is drawn and all its $s$-subsets are marked as collected. We show that the time to collect all super-coupons is $\binom{r}{s}^{-1}\binom{n}{s} \log \binom{n}{s}(1 + o(1))$ on average and has a Gumbel limit after a suitable normalisation. In a similar vein, we show that for any $α\in (0, 1)$, the expected time to collect $(1 - α)$ proportion of all super-coupons is $\binom{r}{s}^{-1}\binom{n}{s} \log \big(\frac{1}α\big)(1 + o(1))$. The $r = s$ case of this model is equivalent to the classical coupon collector model. We also consider a temporally dependent model where the $r$-subsets are drawn according to the following Markovian dynamics: the $r$-subset at round $k + 1$ is formed by replacing a random coupon from the $r$-subset drawn at round $k$ with another random coupon from outside this $r$-subset. We link the time it takes to collect all super-coupons in the $r = s$ case of this model to the cover time of random walk on a certain finite regular graph and conjecture that in general, it takes $\frac{r}{s} \binom{r}{s}^{-1}\binom{n}{s}\log\binom{n}{s}(1 + o(1))$ time on average to collect all super-coupons.

math.PR↗
arXiv2023

Representation Learning Dynamics of Self-Supervised Models

Self-Supervised Learning (SSL) is an important paradigm for learning representations from unlabelled data, and SSL with neural networks has been highly successful in practice. However current theoretical analysis of SSL is mostly restricted to generalisation error bounds. In contrast, learning dynamics often provide a precise characterisation of the behaviour of neural networks based models but, so far, are mainly known in supervised settings. In this paper, we study the learning dynamics of SSL models, specifically representations obtained by minimising contrastive and non-contrastive losses. We show that a naive extension of the dymanics of multivariate regression to SSL leads to learning trivial scalar representations that demonstrates dimension collapse in SSL. Consequently, we formulate SSL objectives with orthogonality constraints on the weights, and derive the exact (network width independent) learning dynamics of the SSL models trained using gradient descent on the Grassmannian manifold. We also argue that the infinite width approximation of SSL models significantly deviate from the neural tangent kernel approximations of supervised models. We numerically illustrate the validity of our theoretical findings, and discuss how the presented results provide a framework for further theoretical analysis of contrastive and non-contrastive SSL.

cs.LG↗
arXiv2023

Wasserstein Projection Pursuit of Non-Gaussian Signals

We consider the general dimensionality reduction problem of locating in a high-dimensional data cloud, a $k$-dimensional non-Gaussian subspace of interesting features. We use a projection pursuit approach -- we search for mutually orthogonal unit directions which maximise the 2-Wasserstein distance of the empirical distribution of data-projections along these directions from a standard Gaussian. Under a generative model, where there is a underlying (unknown) low-dimensional non-Gaussian subspace, we prove rigorous statistical guarantees on the accuracy of approximating this unknown subspace by the directions found by our projection pursuit approach. Our results operate in the regime where the data dimensionality is comparable to the sample size, and thus supplement the recent literature on the non-feasibility of locating interesting directions via projection pursuit in the complementary regime where the data dimensionality is much larger than the sample size.

cs.LG↗
arXiv2022

Improved Representation Learning Through Tensorized Autoencoders

The central question in representation learning is what constitutes a good or meaningful representation. In this work we argue that if we consider data with inherent cluster structures, where clusters can be characterized through different means and covariances, those data structures should be represented in the embedding as well. While Autoencoders (AE) are widely used in practice for unsupervised representation learning, they do not fulfil the above condition on the embedding as they obtain a single representation of the data. To overcome this we propose a meta-algorithm that can be used to extend an arbitrary AE architecture to a tensorized version (TAE) that allows for learning cluster-specific embeddings while simultaneously learning the cluster assignment. For the linear setting we prove that TAE can recover the principle components of the different clusters in contrast to principle component of the entire data recovered by a standard AE. We validated this on planted models and for general, non-linear and convolutional AEs we empirically illustrate that tensorizing the AE is beneficial in clustering and de-noising tasks.

cs.LG↗
arXiv2020

Point Spectrum of Periodic Operators on Universal Covering Trees

For any multi-graph $G$ with edge weights and vertex potential, and its universal covering tree $\mathcal{T}$, we completely characterize the point spectrum of operators $A_{\mathcal{T}}$ on $\mathcal{T}$ arising as pull-backs of local, self-adjoint operators $A_{G}$ on $G$. This builds on work of Aomoto, and includes an alternative proof of the necessary condition for point spectrum he derived in (Aomoto, 1991). Our result gives a finite time algorithm to compute the point spectrum of $A_{\mathcal{T}}$ from the graph $G$, and additionally allows us to show that this point spectrum is contained in the spectrum of $A_{G}$. Finally, we prove that typical pull-back operators have a spectral delocalization property: the set of edge weight and vertex potential parameters of $A_{G}$ giving rise to $A_{\mathcal{T}}$ with purely absolutely continuous spectrum is open and its complement has large codimension.

math.SP↗
arXiv2020

Gaussian Regularization of the Pseudospectrum and Davies' Conjecture

A matrix $A\in\mathbb{C}^{n\times n}$ is diagonalizable if it has a basis of linearly independent eigenvectors. Since the set of nondiagonalizable matrices has measure zero, every $A\in \mathbb{C}^{n\times n}$ is the limit of diagonalizable matrices. We prove a quantitative version of this fact conjectured by E.B. Davies: for each $δ\in (0,1)$, every matrix $A\in \mathbb{C}^{n\times n}$ is at least $δ\|A\|$-close to one whose eigenvectors have condition number at worst $c_n/δ$, for some constants $c_n$ dependent only on $n$. Our proof uses tools from random matrix theory to show that the pseudospectrum of $A$ can be regularized with the addition of a complex Gaussian perturbation. Along the way, we explain how a variant of a theorem of Śniady implies a conjecture of Sankar, Spielman and Teng on the optimal constant for smoothed analysis of condition numbers.

math.FA↗
arXiv2020

On Concentration Inequalities for Random Matrix Products

Consider $n$ complex random matrices $X_1,\ldots,X_n$ of size $d\times d$ sampled i.i.d. from a distribution with mean $E[X]=μ$. While the concentration of averages of these matrices is well-studied, the concentration of other functions of such matrices is less clear. One function which arises in the context of stochastic iterative algorithms, like Oja's algorithm for Principal Component Analysis, is the normalized matrix product defined as $\prod\limits_{i=1}^{n}\left(I + \frac{X_i}{n}\right).$ Concentration properties of this normalized matrix product were recently studied by \cite{HW19}. However, their result is suboptimal in terms of the dependence on the dimension of the matrices as well as the number of samples. In this paper, we present a stronger concentration result for such matrix products which is optimal in $n$ and $d$ up to constant factors. Our proof is based on considering a matrix Doob martingale, controlling the quadratic variation of that martingale, and applying the Matrix Freedman inequality of Tropp \cite{TroppIntro15}.

math.PR↗
arXiv2015

A note on the transversal size of a series of families constructed over Cycle Graph

Paul Erdős and László Lovász established by means of an example that there exists a maximal intersecting family of $k-$sets with approximately $(e-1)k!$ blocks. László Lovász conjectured that their example is best known example which has the maximum number of blocks. Later it was disproved. But the quest for such examples remain valid till this date. In this short note, by computing transversal size of a certain series of uniform intersecting families constructed over the cycle graph, we provide an example which has more than $(\frac{k}{2})^{k-1}$ (approximately) blocks.

math.CO↗