arXiv subjects

Andrea Lepre

Publications and source records attributed to Andrea Lepre.

7 recordsLinked to original sources

arXiv2026

Thermodynamics of Forgetting in a Bounded Hopfield Network

A memory that keeps learning must also forget. The Hopfield network does not: beyond a critical number of stored patterns it falls into a state of total confusion and retrieves none of them. Parisi showed in 1986 that bounding the synaptic strength cures this pathology, since the clipping of the couplings gradually erases the oldest memories and leaves room for the new ones. In this paper we provide a thermodynamic description of the resulting network, known as the Hopfield-Parisi model. Starting from the learning rule, we show that the network is equivalent to a Hopfield model whose patterns are weighted according to their age, superimposed on a Sherrington-Kirkpatrick spin glass, both governed by the synaptic bound: a larger bound makes the memory longer, but also noisier. The theory of nonlinear networks developed in the eighties already provides, through the heuristic replica method, the equations that govern this model at any temperature, but to the best of our knowledge they have been solved only at zero temperature. We rederive them at the replica symmetric level through Guerra's interpolation and solve them at finite temperature, obtaining the phase diagram of the model together with its memory horizon, namely the largest age at which a pattern can still be retrieved. The solution shows that forgetting acts on the signal alone, the noise being the same for patterns of every age, and that the horizon is maximal at an intermediate bound which is essentially insensitive to temperature. Monte Carlo simulations of networks trained with the original learning rule support these predictions.

cond-mat.dis-nn↗

arXiv2026

Classifications in modular restricted Boltzmann machines

We consider a modular associative neural network made of $L$ Hopfield models (HMs), coupled so that intra-module interactions are Hebbian and inter-module interactions are anti-Hebbian; this competitive coupling is known to endow the network with pattern-disentanglement capabilities. The integral representation of this system coincides with an assembly of $L$ restricted Boltzmann machines (RBMs) whose hidden layers are coupled, thereby extending the HM-RBM duality to the modular setting. We then train this modular RBM, via one-step contrastive divergence, to perform a classification task in which a query encoded on the visible layers is mapped onto an $L$-tuple of labels read off the hidden layers. When the query is composed of $L$ patterns that are mutually orthogonal on average, we prove that the RBM weights obtained as empirical means over the training dataset, as suggested by the HM-RBM equivalence, constitute a fixed point of the learning dynamics, and we derive an explicit, non-asymptotic bound on the residual drift. We then turn to a harder classification task in which each module is queried with a mixture of $L$ patterns and we show numerically that the same setting for the RBM weights still provides an effective set-up, letting the network jointly classify and disentangle the mixture.

cond-mat.dis-nn↗

arXiv2026

Exponential Capacity in Multilayer Hetero-Associative Neural Networks

Exponential Hopfield networks store a number of patterns that grows exponentially with the number of neurons, and in their classical formulation they are auto-associative: they complete a corrupted copy of a memory into the memory itself. Many of the tasks one wants such a network to perform are instead hetero-associative, mapping a cue to a different target. We introduce and analyse an exponential neural network of $L$ layers of $N$ binary neurons, each layer carrying its own dataset, whose energy is an exponential of the product of the per-layer Mattis overlaps, so that it is minimised precisely when every layer retrieves the pattern of the same index; the stored association must be a surjective function of the cue, and we show why nothing else can be stored at all. A cavity/signal-to-noise analysis, made exact at leading order by a large-deviation evaluation of the noise, shows that the aligned hetero-associative state is a fixed point of the zero-temperature dynamics up to a number of stored patterns $P_c\sim e^{Nρ_L}$, exponential in the layer size, with an explicit rate $ρ_L$ that grows like $L\log 2$; enlarging the basins of attraction lowers the rate but never destroys its exponential character. Comparing the theory with structured data we find that the exponential capacity and the predicted basins survive correlated, many-to-one patterns: the network is a near-perfect content-addressable memory. The same closed forms describe, without refitting, a synthetic manifold, real T-cell-receptor/epitope triples and natural-language intent data, so the mechanism is domain-universal. Generalisation to unseen cues, though significantly above chance, stays below memorisation, and it is the geometry of the encoding, rather than the data domain, that sets how far above chance it reaches. In this family, exponential storage and strong generalisation are distinct capabilities.

cond-mat.dis-nn↗

arXiv2026

Semi-supervised Hopfield model: Theoretical and Numerical results

In the daily practice of Machine Learning, fully labeled datasets are a luxury: labels demand expensive and time-consuming human annotation, whereas raw, unlabeled data can be harvested automatically and in bulk. Semi-supervised learning, where the network jointly exploits the few labeled and the many unlabeled examples at its disposal, is the standard answer to this asymmetry, yet a statistical mechanical theory of semi-supervised Hebbian learning is still lacking. In this paper we fill this gap for the Hopfield network: we prescribe a synaptic coupling given by the convex combination, weighted by a mixing parameter λin [0,1], of the supervised and unsupervised Hebbian kernels built from the same archetypes, and we solve for the emergent computational capabilities of the resulting network. A signal-to-noise analysis yields the one-step Mattis magnetization and the learning threshold, i.e. the minimum dataset size for stable retrieval. Using Guerra's interpolation, we then derive the Replica Symmetric quenched pressure in the high-storage regime, treating the correlated disorder generated by the supervised and unsupervised channels through a particular eigen-channel decomposition. The resulting phase diagram shows that a mixed strategy outperforms both pure protocols. Finally, we prove that the quenched pressure is convex in λ, so thermodynamics cannot select an interior mixture: λis therefore a learning hyperparameter. All the analytical findings are successfully checked against extensive Monte Carlo simulations.

cond-mat.dis-nn↗

arXiv2026

A Federated Many-to-One Hopfield model for associative Neural Networks

Federated learning enables collaborative training without sharing raw data, but struggles under client heterogeneity and streaming distribution shifts, where drift and novel data can impair convergence and cause forgetting. We propose a federated associative-memory framework that learns shared archetypes in heterogeneous, continual settings, where client data are independent but not necessarily balanced. Each client encodes its experience as a low-rank Hebbian operator, sent to a central server for aggregation and factorization into global archetypes. This approach preserves privacy, avoids centralized replay buffers, and is robust to small, noisy, or evolving datasets. We cast aggregation as a low-rank-plus-noise spectral inference problem, deriving theoretical thresholds for detectability and retrieval robustness. An entropy-based controller balances stability and plasticity in streaming regimes. Experiments with heterogeneous clients, drift, and novelty show improved global archetype reconstruction and associative retrieval, supporting the spectral view of federated consolidation.

cond-mat.dis-nn↗

arXiv2025

Supervised and Unsupervised protocols for hetero-associative neural networks

This paper introduces a learning framework for Three-Directional Associative Memory (TAM) models, extending the classical Hebbian paradigm to both supervised and unsupervised protocols within an hetero-associative setting. These neural networks consist of three interconnected layers of binary neurons interacting via generalized Hebbian synaptic couplings that allow learning, storage and retrieval of structured triplets of patterns. By relying upon glassy statistical mechanical techniques (mainly replica theory and Guerra interpolation), we analyze the emergent computational properties of these networks, at work with random (Rademacher) datasets and at the replica-symmetric level of description: we obtain a set of self-consistency equations for the order parameters that quantify the critical dataset sizes (i.e. their thresholds for learning) and describe the retrieval performance of these networks, highlighting the differences between supervised and unsupervised protocols. Numerical simulations validate our theoretical findings and demonstrate the robustness of the captured picture about TAMs also at work with structured datasets. In particular, this study provides insights into the cooperative interplay of layers, beyond that of the neurons within the layers, with potential implications for optimal design of artificial neural network architectures.

cond-mat.dis-nn↗

arXiv2025

Beyond Disorder: Unveiling Cooperativeness in Multidirectional Associative Memories

By leveraging tools from the statistical mechanics of complex systems, in these short notes we extend the architecture of a neural network for hetero-associative memory (called three-directional associative memories, TAM) to explore supervised and unsupervised learning protocols. In particular, by providing entropic-heterogeneous datasets to its various layers, we predict and quantify a new emergent phenomenon -- that we term {\em layer's cooperativeness} -- where the interplay of dataset entropies across network's layers enhances their retrieval capabilities Beyond those they would have without reciprocal influence. Naively we would expect layers trained with less informative datasets to develop smaller retrieval regions compared to those pertaining to layers that experienced more information: this does not happen and all the retrieval regions settle to the same amplitude, allowing for optimal retrieval performance globally. This cooperative dynamics marks a significant advancement in understanding emergent computational capabilities within disordered systems.

cond-mat.dis-nn↗