arXiv subjects

Andrea Alessandrelli

Publications and source records attributed to Andrea Alessandrelli.

18 recordsLinked to original sources

arXiv2026

Partial annealing and pattern decorrelation in associative neural networks

Using the Hopfield model as a benchmark case, the present work focuses on the investigation of partially annealed associative neural networks, wherein neural dynamics is coupled to slowly evolving patterns within the two-temperature-two-timescale framework. This setting inherently introduces a real parameter n, reminiscent of the number of replicas in the celebrated replica trick, that tunes the separation of timescales and the effective interaction between fast (i.e. the neurons) and slow (i.e. the synapses) degrees of freedom. By adapting Guerra's interpolation to the case, we derive the free energy without relying on analytical continuation. The obtained results demonstrate that negative values of n induce a progressive decorrelation of the stored patterns, thereby effectively reducing interference, promoting orthogonal configurations and ultimately conferring to the network the maximal storage alphac=1. Numerical simulations based on a mean field Monte Carlo dynamics have been employed to confirm this scenario and prove that partial annealing restores retrieval in challenging regimes, such as in the presence of biased patterns, outperforming standard decorrelation methods. These findings underscore the notion of partial annealing as an adaptive mechanism for enhancing memory organisation and retrieval in complex systems.

cond-mat.dis-nn↗

arXiv2026

Dense Associative Memory with biased patterns: a Replica Symmetric analysis

We investigate dense higher-order associative memories in the high storage regime when the stored patterns are biased, namely when the entries of the patterns are not symmetrically distributed around zero. In this setting, the standard Hebbian prescription must be modified by recentering and rescaling the pattern entries, and an additional term must be introduced in the Hamiltonian to enforce consistency between the average activity of the network and that of the stored patterns. As a first step, we perform a signal-to-noise analysis in the zero-temperature limit and show that the bias reduces the effective storage capacity through a multiplicative correction factor (1-b^2)^P, while preserving the superlinear scaling with the system size. We then derive the quenched statistical pressure within the Replica Symmetric framework by means of Guerra's interpolation method and obtain the corresponding self consistency equations for the relevant order parameters. The analytical treatment confirms the heuristic prediction of the signal-to-noise argument, showing that the same bias dependent renormalization naturally emerges in the variance of the cross-talk noise. Finally, we discuss the resulting phase behavior of the model and its implications for retrieval performance in the model.

cond-mat.dis-nn↗

arXiv2026

A Federated Many-to-One Hopfield model for associative Neural Networks

Federated learning enables collaborative training without sharing raw data, but struggles under client heterogeneity and streaming distribution shifts, where drift and novel data can impair convergence and cause forgetting. We propose a federated associative-memory framework that learns shared archetypes in heterogeneous, continual settings, where client data are independent but not necessarily balanced. Each client encodes its experience as a low-rank Hebbian operator, sent to a central server for aggregation and factorization into global archetypes. This approach preserves privacy, avoids centralized replay buffers, and is robust to small, noisy, or evolving datasets. We cast aggregation as a low-rank-plus-noise spectral inference problem, deriving theoretical thresholds for detectability and retrieval robustness. An entropy-based controller balances stability and plasticity in streaming regimes. Experiments with heterogeneous clients, drift, and novelty show improved global archetype reconstruction and associative retrieval, supporting the spectral view of federated consolidation.

cond-mat.dis-nn↗

arXiv2026

Serial vs parallel recall in the Blume-Every-Griffiths neural networks

Fully connected Blume-Emery-Griffiths neural networks performing pattern recognition and associative memory have been heuristically studied in the past (mainly via the replica trick and under the replica symmetric assumption) as generalization of the standard Hopfield reference. In these notes, at first, by relying upon Guerra interpolation, we re-obtain the existing picture rigorously. Next we show that, due to dilution in the patterns, these networks are able to switch from serial recall (where one pattern is retrieved per time) to parallel recall (where several patterns are retrieved at once) and the larger the dilution, the stronger this emerging multi-tasking capability. In particular, we inspect the regimes of mild dilution (where solely a low storage of pattern can be enabled) and extreme dilution (where a medium storage of patterns can be sustained) separately as they give rise to different outcomes: the former displays hierarchical recall (distributing the amplitudes of the retrieved signals with different amplitudes), the latter -- instead -- performs a equal-strength recall (where a O(1) fraction of all the patterns is simultaneously retrieved with the same amplitude per pattern). Finally, in order to implement graded responses in the neurons, variations on theme obtained by enlarging the possible values of neural activity these neurons may sustain are also discussed generalizing the Ghatak-Sherrington model for inverse freezing in Hebbian terms.

cond-mat.dis-nn↗

arXiv2025

Networks of neural networks: more is different

The common thread behind the recent Nobel Prize in Physics to John Hopfield and those conferred to Giorgio Parisi in 2021 and Philip Anderson in 1977 is disorder. Quoting Philip Anderson: "more is different". This principle has been extensively demonstrated in magnetic systems and spin glasses, and, in this work, we test its validity on Hopfield neural networks to show how an assembly of these models displays emergent capabilities that are not present at a single network level. Such an assembly is designed as a layered associative Hebbian network that, beyond accomplishing standard pattern recognition, spontaneously performs also pattern disentanglement. Namely, when inputted with a composite signal -- e.g., a musical chord -- it can return the single constituting elements -- e.g., the notes making up the chord. Here, restricting to notes coded as Rademacher vectors and chords that are their mixtures (i.e., spurious states), we use tools borrowed from statistical mechanics of disordered systems to investigate this task, obtaining the conditions over the model control-parameters such that pattern disentanglement is successfully executed.

cond-mat.dis-nn↗

arXiv2025

Yet another exponential Hopfield model

We propose and analyze a new variation of the so-called {\em exponential Hopfield model}, a recently introduced family of associative neural networks with unprecedented storage capacity. Our construction is based on a cost function defined through exponentials of standard quadratic loss functions, which naturally favors configurations corresponding to perfect recall. Despite not being a mean-field system, the model admits a tractable mathematical analysis of its dynamics and retrieval properties that agree with those for the original exponential model introduced by Ramsauer and coworkers. By means of a signal-to-noise approach, we demonstrate that stored patterns remain stable fixed points of the zero-temperature dynamics up to an exponentially large number of patterns in the system size. We further quantify the basins of attraction of the retrieved memories, showing that while enlarging their radius reduces the overall load, the storage capacity nonetheless retains its exponential scaling. An independent derivation within the perfect recall regime confirms these results and provides an estimate of the relevant prefactors. Our findings thus complement and extend previous studies on exponential Hopfield networks, establishing that even under robustness constraints these models preserve their exceptional storage capabilities. Beyond their theoretical interest, such networks point towards principled mechanisms for massively scalable associative memory, with potential implications for both neuroscience-inspired computation and high-capacity machine learning architectures.

cond-mat.dis-nn↗

arXiv2025

Supervised and Unsupervised protocols for hetero-associative neural networks

This paper introduces a learning framework for Three-Directional Associative Memory (TAM) models, extending the classical Hebbian paradigm to both supervised and unsupervised protocols within an hetero-associative setting. These neural networks consist of three interconnected layers of binary neurons interacting via generalized Hebbian synaptic couplings that allow learning, storage and retrieval of structured triplets of patterns. By relying upon glassy statistical mechanical techniques (mainly replica theory and Guerra interpolation), we analyze the emergent computational properties of these networks, at work with random (Rademacher) datasets and at the replica-symmetric level of description: we obtain a set of self-consistency equations for the order parameters that quantify the critical dataset sizes (i.e. their thresholds for learning) and describe the retrieval performance of these networks, highlighting the differences between supervised and unsupervised protocols. Numerical simulations validate our theoretical findings and demonstrate the robustness of the captured picture about TAMs also at work with structured datasets. In particular, this study provides insights into the cooperative interplay of layers, beyond that of the neurons within the layers, with potential implications for optimal design of artificial neural network architectures.

cond-mat.dis-nn↗

arXiv2025

Multi-channel pattern reconstruction through $L$-directional associative memories

We consider $L$-directional associative memories, composed of $L$ Hopfield networks, displaying imitative Hebbian intra-network interactions and anti-imitative Hebbian inter-network interactions, where couplings are built over a set of hidden binary patterns. We evaluate the model's performance in reconstructing the whole set of hidden binary patterns when provided with mixtures of noisy versions of these patterns. Our numerical results demonstrate the model's high effectiveness in the reconstruction task for structureless and structured datasets.

cond-mat.dis-nn↗

arXiv2025

Beyond Disorder: Unveiling Cooperativeness in Multidirectional Associative Memories

By leveraging tools from the statistical mechanics of complex systems, in these short notes we extend the architecture of a neural network for hetero-associative memory (called three-directional associative memories, TAM) to explore supervised and unsupervised learning protocols. In particular, by providing entropic-heterogeneous datasets to its various layers, we predict and quantify a new emergent phenomenon -- that we term {\em layer's cooperativeness} -- where the interplay of dataset entropies across network's layers enhances their retrieval capabilities Beyond those they would have without reciprocal influence. Naively we would expect layers trained with less informative datasets to develop smaller retrieval regions compared to those pertaining to layers that experienced more information: this does not happen and all the retrieval regions settle to the same amplitude, allowing for optimal retrieval performance globally. This cooperative dynamics marks a significant advancement in understanding emergent computational capabilities within disordered systems.

cond-mat.dis-nn↗

arXiv2024

Boolean SK Model

For over half a century, statistical mechanics of spin glasses played as a paradigm to model and interpret disparate phenomena, ranging from quantitative biology to computer science. However, despite the extensive body of research in this area, there is still a notable lack of studies addressing the replacement of Ising spins with Boolean spins: as the latter play as bits in Machine Learning, this gap to fill is now mandatory. Purpose of this paper is to address this study by focusing on the mean field assumption, providing a comprehensive description of the results pertaining to these networks, referred to as the Boolean SK model due to their close relationship with the SK one. We provide a comprehensive framework for this model by employing Guerra interpolation: the thermodynamic limit, the replica symmetric and the broken replica free energy expressions are derived. Further, we inspect the onset of the replica symmetry breaking -- i.e., the de Almeida-Thouless line -- and derive Ghirlanda-Guerra fluctuations. All theoretical findings are corroborated by numerical inspections and both highlight crucial differences in the network's behavior if compared with the Ising SK model: as the temperature is lowered, no phase transitions are evidenced and the model continuously moves from a random (ergodic) behavior to a disordered (glassy) phase.

cond-mat.dis-nn↗

arXiv2024

Generalized hetero-associative neural networks

Auto-associative neural networks (e.g., the Hopfield model implementing the standard Hebbian prescription) serve as a foundational framework for pattern recognition and associative memory in statistical mechanics. However, their hetero-associative counterparts, though less explored, exhibit even richer computational capabilities. In this work, we examine a straightforward extension of Kosko's Bidirectional Associative Memory (BAM), introducing a Three-directional Associative Memory (TAM), that is a tripartite neural network equipped with generalized Hebbian weights. Through both analytical approaches (using replica-symmetric statistical mechanics) and computational methods (via Monte Carlo simulations), we derive phase diagrams within the space of control parameters, revealing a region where the network can successfully perform pattern recognition as well as other tasks tasks. In particular, it can achieve pattern disentanglement, namely, when presented with a mixture of patterns, the network can recover the original patterns. Furthermore, the system is capable of retrieving Markovian sequences of patterns and performing generalized frequency modulation.

cond-mat.dis-nn↗

arXiv2023

Unsupervised and Supervised learning by Dense Associative Memory under replica symmetry breaking

Statistical mechanics of spin glasses is one of the main strands toward a comprehension of information processing by neural networks and learning machines. Tackling this approach, at the fairly standard replica symmetric level of description, recently Hebbian attractor networks with multi-node interactions (often called Dense Associative Memories) have been shown to outperform their classical pairwise counterparts in a number of tasks, from their robustness against adversarial attacks and their capability to work with prohibitively weak signals to their supra-linear storage capacities. Focusing on mathematical techniques more than computational aspects, in this paper we relax the replica symmetric assumption and we derive the one-step broken-replica-symmetry picture of supervised and unsupervised learning protocols for these Dense Associative Memories: a phase diagram in the space of the control parameters is achieved, independently, both via the Parisi's hierarchy within then replica trick as well as via the Guerra's telescope within the broken-replica interpolation. Further, an explicit analytical investigation is provided to deepen both the big-data and ground state limits of these networks as well as a proof that replica symmetry breaking does not alter the thresholds for learning and slightly increases the maximal storage capacity. Finally the De Almeida and Thouless line, depicting the onset of instability of a replica symmetric description, is also analytically derived highlighting how, crossed this boundary, the broken replica description should be preferred.

cond-mat.dis-nn↗

arXiv2023

About the de Almeida-Thouless line in neural networks

In this work we present a rigorous and straightforward method to detect the onset of the instability of replica-symmetric theories in information processing systems, which does not require a full replica analysis as in the method originally proposed by de Almeida and Thouless for spin glasses. The method is based on an expansion of the free-energy obtained within one-step of replica symmetry breaking (RSB) around the RS value. As such, it requires solely continuity and differentiability of the free-energy and it is robust to be applied broadly to systems with quenched disorder. We apply the method to the Hopfield model and to neural networks with multi-node Hebbian interactions, as case studies. In the appendices we test the method on the Sherrington-Kirkpatrick and the Ising P-spin models, recovering the AT lines known in the literature for these models, as a special limit, which corresponds to assuming that the transition from the RS to the RSB phase can be obtained by varying continuously the order parameters. Our method provides a generalization of the AT approach, which does not rely on this limit and can be applied to systems with discontinuous phase transitions, as we show explicitly for the spherical P-spin model, recovering the known RS instability line.

cond-mat.dis-nn↗

arXiv2023

Parallel Learning by Multitasking Neural Networks

A modern challenge of Artificial Intelligence is learning multiple patterns at once (i.e.parallel learning). While this can not be accomplished by standard Hebbian associative neural networks, in this paper we show how the Multitasking Hebbian Network (a variation on theme of the Hopfield model working on sparse data-sets) is naturally able to perform this complex task. We focus on systems processing in parallel a finite (up to logarithmic growth in the size of the network) amount of patterns, mirroring the low-storage level of standard associative neural networks at work with pattern recognition. For mild dilution in the patterns, the network handles them hierarchically, distributing the amplitudes of their signals as power-laws w.r.t. their information content (hierarchical regime), while, for strong dilution, all the signals pertaining to all the patterns are raised with the same strength (parallel regime). Further, confined to the low-storage setting (i.e., far from the spin glass limit), the presence of a teacher neither alters the multitasking performances nor changes the thresholds for learning: the latter are the same whatever the training protocol is supervised or unsupervised. Results obtained through statistical mechanics, signal-to-noise technique and Monte Carlo simulations are overall in perfect agreement and carry interesting insights on multiple learning at once: for instance, whenever the cost-function of the model is minimized in parallel on several patterns (in its description via Statistical Mechanics), the same happens to the standard sum-squared error Loss function (typically used in Machine Learning).

cond-mat.dis-nn↗

arXiv2023

Dense Hebbian neural networks: a replica symmetric picture of unsupervised learning

We consider dense, associative neural-networks trained with no supervision and we investigate their computational capabilities analytically, via a statistical-mechanics approach, and numerically, via Monte Carlo simulations. In particular, we obtain a phase diagram summarizing their performance as a function of the control parameters such as the quality and quantity of the training dataset and the network storage, valid in the limit of large network size and structureless datasets. Moreover, we establish a bridge between macroscopic observables standardly used in statistical mechanics and loss functions typically used in the machine learning. As technical remarks, from the analytic side, we implement large deviations and stability analysis within Guerra's interpolation to tackle the not-Gaussian distributions involved in the post-synaptic potentials while, from the computational counterpart, we insert Plefka approximation in the Monte Carlo scheme, to speed up the evaluation of the synaptic tensors, overall obtaining a novel and broad approach to investigate neural networks in general.

cond-mat.dis-nn↗

arXiv2023

Dense Hebbian neural networks: a replica symmetric picture of supervised learning

We consider dense, associative neural-networks trained by a teacher (i.e., with supervision) and we investigate their computational capabilities analytically, via statistical-mechanics of spin glasses, and numerically, via Monte Carlo simulations. In particular, we obtain a phase diagram summarizing their performance as a function of the control parameters such as quality and quantity of the training dataset, network storage and noise, that is valid in the limit of large network size and structureless datasets: these networks may work in a ultra-storage regime (where they can handle a huge amount of patterns, if compared with shallow neural networks) or in a ultra-detection regime (where they can perform pattern recognition at prohibitive signal-to-noise ratios, if compared with shallow neural networks). Guided by the random theory as a reference framework, we also test numerically learning, storing and retrieval capabilities shown by these networks on structured datasets as MNist and Fashion MNist. As technical remarks, from the analytic side, we implement large deviations and stability analysis within Guerra's interpolation to tackle the not-Gaussian distributions involved in the post-synaptic potentials while, from the computational counterpart, we insert Plefka approximation in the Monte Carlo scheme, to speed up the evaluation of the synaptic tensors, overall obtaining a novel and broad approach to investigate supervised learning in neural networks, beyond the shallow limit, in general.

cond-mat.dis-nn↗

arXiv2021

Replica symmetry breaking in dense neural networks

Understanding the glassy nature of neural networks is pivotal both for theoretical and computational advances in Machine Learning and Theoretical Artificial Intelligence. Keeping the focus on dense associative Hebbian neural networks, the purpose of this paper is two-fold: at first we develop rigorous mathematical approaches to address properly a statistical mechanical picture of the phenomenon of {\em replica symmetry breaking} (RSB) in these networks, then -- deepening results stemmed via these routes -- we aim to inspect the {\em glassiness} that they hide. In particular, regarding the methodology, we provide two techniques: the former is an adaptation of the transport PDE to the case, while the latter is an extension of Guerra's interpolation breakthrough. Beyond coherence among the results, either in replica symmetric and in the one-step replica symmetry breaking level of description, we prove the Gardner's picture and we identify the maximal storage capacity by a ground-state analysis in the Baldi-Venkatesh high-storage regime. In the second part of the paper we investigate the glassy structure of these networks: in contrast with the replica symmetric scenario (RS), RSB actually stabilizes the spin-glass phase. We report huge differences w.r.t. the standard pairwise Hopfield limit: in particular, it is known that it is possible to express the free energy of the Hopfield neural network as a linear combination of the free energies of an hard spin glass (i.e. the Sherrington-Kirkpatrick model) and a soft spin glass (the Gaussian or "spherical" model). This is no longer true when interactions are more than pairwise (whatever the level of description, RS or RSB): for dense networks solely the free energy of the hard spin glass survives, proving a huge diversity in the underlying glassiness of associative neural networks.

cond-mat.dis-nn↗

arXiv2021

Rigorous approaches for spin glass and Gaussian spin glass with P-wise interactions

Purpose of this paper is to face up to P-spin glass and Gaussian P-spin model, i.e. spin glasses with polynomial interactions of degree P > 2. We consider the replica symmetry and first step of replica simmetry breaking assumptions and we solve the models via transport equation and Guerra's interpolating technique, showing that we reach the same results. \\ Thus, using rigorous approaches, we recover the same expression for quenched statistical pressure and self-consistency equation in both assumption found with other techniques, including the well-known \textit{replica trick} technique. \\ At the end, we show that for $P=2$ the Gaussian P-spin glass model is intrinsecally RS.

cond-mat.dis-nn↗