Iterative Exploration-Driven Sparse SDP Clustering via Thompson Sampling

Abstract

High-dimensional sparse clustering is a combinatorial NP-hard problem that arises from the coupling between cluster assignment and variable selection. We demonstrate that semidefinite programming (SDP) relaxation of K-means is robust to variable over-selection by establishing minimax separation bounds. Leveraging this robustness, we propose a block-coordinate ascent framework that alternates between SDP-based clustering and conservative variable selection. To address the tendency of deterministic greedy methods to become trapped in local optima, we formulate the variable selection step as a bandit problem. Crucially, to reliably evaluate feature utility and generate stochastic rewards for Thompson sampling even under imperfect intermediate cluster assignments, we employ a robust maximum mean discrepancy (MMD) permutation test. This approach introduces adaptive memory by aggregating historical variable-selection outcomes into posterior distributions, and selects features via posterior sampling, enabling stochastic exploration that promotes the inclusion of underexplored features and facilitates escape from local maxima. We establish conditions for consistent variable selection and exact cluster recovery, and extend the method to settings with unknown covariance through a scalable estimation procedure. Synthetic experiments and a real-data application in document clustering demonstrate that the proposed memory-driven randomized approach consistently outperforms state-of-the-art sparse clustering methods.