Spectral criteria for generalization in unsupervised Hebbian nets

Abstract

We consider an unsupervised Hebbian network where the pairwise interactions among neurons are built on noisy realizations of hidden ground-truth vectors. Unlike classical Hopfield models, designed as memory devices, this class of networks can be employed to extract latent structure and generalize beyond the "training" set. By combining random matrix theory and replica methods, we derive the asymptotic spectrum of the corresponding interaction matrix and show that the onset of generalization is controlled by a sharp spectral transition. Depending on the quality and the size of the accessible dataset, the spectrum displays either two separated bulks, encoding informative and noisy directions, or a merged single-bulk phase where such distinction is lost. We show that, when coupled with regularization, the emergence of such a spectral split predicts the network's capability to reconstruct the ground-truth vectors from corrupted samples.