Pierluigi Bianco | Searcharxiv

arXiv2026

The infinite volume limit of the Hopfield neural networks in the replica-symmetric regime

Despite extensive research over the past five decades, a rigorous determination of the infinite-volume limit of the free energy of the Hopfield model with an extensive number of patterns has remained an outstanding open problem as standard methods from spin-glass theory have so far proven inadequate for this purpose. While a complete proof for finite, non-vanishing, loads is still out of reach, here we develop a novel interpolation strategy that allows us to express the free energy of the Hopfield model in terms of a linear combination of the free energies of hard- and soft-spin glass models along with a remainder contribution. The payoff of this interpolation scheme is twofold: (i) As the remainder involves positive-definite fluctuations of the order parameters (and it vanishes under a concentration assumption on their means where we consistently recover the replica-symmetric picture) such an expression finally constitutes a rigorous {\em bound} on the exact free energy, rather than just an {\em approximation,} as it was the case for earlier interpolation procedures à la Guerra. \newline (ii) As for the spin-glass models the existence of the infinite-volume free energy has been previously determined by the Guerra--Toninelli scheme, this novel interpolation allows us to extend its validity to the Hopfield model too. Yet, as the equivalence between the Hopfield model and mixtures of spin-glasses is currently well established under replica symmetry and up to the first step of replica symmetry breaking, at present our approach can be rigorously implemented only within these regimes. \newline In this work, by inspecting how the order parameters have to self-average around their means, we restrict our attention to the replica-symmetric regime.

cond-mat.dis-nn↗

arXiv2024

Hebbian Learning from First Principles

Recently, the original storage prescription for the Hopfield model of neural networks -- as well as for its dense generalizations -- has been turned into a genuine Hebbian learning rule by postulating the expression of its Hamiltonian for both the supervised and unsupervised protocols. In these notes, first, we obtain these explicit expressions by relying upon maximum entropy extremization à la Jaynes. Beyond providing a formal derivation of these recipes for Hebbian learning, this construction also highlights how Lagrangian constraints within entropy extremization force network's outcomes on neural correlations: these try to mimic the empirical counterparts hidden in the datasets provided to the network for its training and, the denser the network, the longer the correlations that it is able to capture. Next, we prove that, in the big data limit, whatever the presence of a teacher (or its lacking), not only these Hebbian learning rules converge to the original storage prescription of the Hopfield model but also their related free energies (and, thus, the statistical mechanical picture provided by Amit, Gutfreund and Sompolinsky is fully recovered). As a sideline, we show mathematical equivalence among standard Cost functions (Hamiltonian), preferred in Statistical Mechanical jargon, and quadratic Loss Functions, preferred in Machine Learning terminology. Remarks on the exponential Hopfield model (as the limit of dense networks with diverging density) and semi-supervised protocols are also provided.

cond-mat.dis-nn↗