arXiv subjects

Elena Agliari

Publications and source records attributed to Elena Agliari.

At least 19 recordsLinked to original sources

arXiv2026

Exponential Capacity in Multilayer Hetero-Associative Neural Networks

Exponential Hopfield networks store a number of patterns that grows exponentially with the number of neurons, and in their classical formulation they are auto-associative: they complete a corrupted copy of a memory into the memory itself. Many of the tasks one wants such a network to perform are instead hetero-associative, mapping a cue to a different target. We introduce and analyse an exponential neural network of $L$ layers of $N$ binary neurons, each layer carrying its own dataset, whose energy is an exponential of the product of the per-layer Mattis overlaps, so that it is minimised precisely when every layer retrieves the pattern of the same index; the stored association must be a surjective function of the cue, and we show why nothing else can be stored at all. A cavity/signal-to-noise analysis, made exact at leading order by a large-deviation evaluation of the noise, shows that the aligned hetero-associative state is a fixed point of the zero-temperature dynamics up to a number of stored patterns $P_c\sim e^{Nρ_L}$, exponential in the layer size, with an explicit rate $ρ_L$ that grows like $L\log 2$; enlarging the basins of attraction lowers the rate but never destroys its exponential character. Comparing the theory with structured data we find that the exponential capacity and the predicted basins survive correlated, many-to-one patterns: the network is a near-perfect content-addressable memory. The same closed forms describe, without refitting, a synthetic manifold, real T-cell-receptor/epitope triples and natural-language intent data, so the mechanism is domain-universal. Generalisation to unseen cues, though significantly above chance, stays below memorisation, and it is the geometry of the encoding, rather than the data domain, that sets how far above chance it reaches. In this family, exponential storage and strong generalisation are distinct capabilities.

cond-mat.dis-nn↗

arXiv2026

Spectral criteria for generalization in unsupervised Hebbian nets

We consider an unsupervised Hebbian network where the pairwise interactions among neurons are built on noisy realizations of hidden ground-truth vectors. Unlike classical Hopfield models, designed as memory devices, this class of networks can be employed to extract latent structure and generalize beyond the "training" set. By combining random matrix theory and replica methods, we derive the asymptotic spectrum of the corresponding interaction matrix and show that the onset of generalization is controlled by a sharp spectral transition. Depending on the quality and the size of the accessible dataset, the spectrum displays either two separated bulks, encoding informative and noisy directions, or a merged single-bulk phase where such distinction is lost. We show that, when coupled with regularization, the emergence of such a spectral split predicts the network's capability to reconstruct the ground-truth vectors from corrupted samples.

cond-mat.dis-nn↗

arXiv2025

Networks of neural networks: more is different

The common thread behind the recent Nobel Prize in Physics to John Hopfield and those conferred to Giorgio Parisi in 2021 and Philip Anderson in 1977 is disorder. Quoting Philip Anderson: "more is different". This principle has been extensively demonstrated in magnetic systems and spin glasses, and, in this work, we test its validity on Hopfield neural networks to show how an assembly of these models displays emergent capabilities that are not present at a single network level. Such an assembly is designed as a layered associative Hebbian network that, beyond accomplishing standard pattern recognition, spontaneously performs also pattern disentanglement. Namely, when inputted with a composite signal -- e.g., a musical chord -- it can return the single constituting elements -- e.g., the notes making up the chord. Here, restricting to notes coded as Rademacher vectors and chords that are their mixtures (i.e., spurious states), we use tools borrowed from statistical mechanics of disordered systems to investigate this task, obtaining the conditions over the model control-parameters such that pattern disentanglement is successfully executed.

cond-mat.dis-nn↗

arXiv2025

The importance of being empty: a spectral approach to Hopfield neural networks with diluted examples

We consider Hopfield networks, where neurons interact pair-wise by Hebbian couplings built over $i$. a set of definite patterns (ground truths), $ii$. a sample of labeled examples (supervised setting), $iii$. a sample of unlabeled examples (unsupervised setting). We focus on the case where ground-truths are Rademacher vectors and examples are noisy versions of these ground-truths, possibly displaying some blank entries (e.g., mimicking missing or dropped data), and we determine the spectral distribution of the coupling matrices in the three scenarios, by exploiting and extending the Marchenko-Pastur theorem. By levering this knowledge, we are able to analytically inspect the stability and attractiveness of the ground truths, as well as the generalization capabilities of the networks. In particular, as corroborated by long-running Monte Carlo simulations, the presence of black entries can have benefits in some specific conditions, suggesting strategies based on data sparsification; the robustness of these results in structured datasets is confirmed numerically. Finally, we demonstrate that the Hebbian matrix, built on sparse examples, can be recovered as the fixed point of a gradient descent algorithm with dropout, over a suitable loss function.

cond-mat.dis-nn↗

arXiv2025

Multi-channel pattern reconstruction through $L$-directional associative memories

We consider $L$-directional associative memories, composed of $L$ Hopfield networks, displaying imitative Hebbian intra-network interactions and anti-imitative Hebbian inter-network interactions, where couplings are built over a set of hidden binary patterns. We evaluate the model's performance in reconstructing the whole set of hidden binary patterns when provided with mixtures of noisy versions of these patterns. Our numerical results demonstrate the model's high effectiveness in the reconstruction task for structureless and structured datasets.

cond-mat.dis-nn↗

arXiv2024

Generalized hetero-associative neural networks

Auto-associative neural networks (e.g., the Hopfield model implementing the standard Hebbian prescription) serve as a foundational framework for pattern recognition and associative memory in statistical mechanics. However, their hetero-associative counterparts, though less explored, exhibit even richer computational capabilities. In this work, we examine a straightforward extension of Kosko's Bidirectional Associative Memory (BAM), introducing a Three-directional Associative Memory (TAM), that is a tripartite neural network equipped with generalized Hebbian weights. Through both analytical approaches (using replica-symmetric statistical mechanics) and computational methods (via Monte Carlo simulations), we derive phase diagrams within the space of control parameters, revealing a region where the network can successfully perform pattern recognition as well as other tasks tasks. In particular, it can achieve pattern disentanglement, namely, when presented with a mixture of patterns, the network can recover the original patterns. Furthermore, the system is capable of retrieving Markovian sequences of patterns and performing generalized frequency modulation.

cond-mat.dis-nn↗

arXiv2024

The thermodynamic limit in mean field neural networks

In the last five decades, mean-field neural-networks have played a crucial role in modelling associative memories and, in particular, the Hopfield model has been extensively studied using tools borrowed from the statistical mechanics of spin glasses. However, achieving mathematical control of the infinite-volume limit of the model's free-energy has remained elusive, as the standard treatments developed for spin-glasses have proven unfeasible. Here we address this long-standing problem by proving that a measure-concentration assumption for the order parameters of the theory is sufficient for the existence of the asymptotic limit of the model's free energy. The proof leverages the equivalence between the free energy of the Hopfield model and a linear combination of the free energies of a hard and a soft spin-glass, whose thermodynamic limits are rigorously known. Our work focuses on the replica-symmetry level of description (for which we recover the explicit expression of the free-energy found in the eighties via heuristic methods), yet, our scheme is expected to work also under (at least) the first step of replica symmetry breaking.

cond-mat.dis-nn↗

arXiv2024

Guerra interpolation for place cells

Pyramidal cells that emit spikes when the animal is at specific locations of the environment are known as "place cells": these neurons are thought to provide an internal representation of space via "cognitive maps". Here, we consider the Battaglia-Treves neural network model for cognitive map storage and reconstruction, instantiated with McCulloch & Pitts binary neurons. To quantify the information processing capabilities of these networks, we exploit spin-glass techniques based on Guerra's interpolation: in the low-storage regime (i.e., when the number of stored maps scales sub-linearly with the network size and the order parameters self-average around their means) we obtain an exact phase diagram in the noise vs inhibition strength plane (in agreement with previous findings) by adapting the Hamilton-Jacobi PDE-approach. Conversely, in the high-storage regime, we find that -- for mild inhibition and not too high noise -- memorization and retrieval of an extensive number of spatial maps is indeed possible, since the maximal storage capacity is shown to be strictly positive. These results, holding under the replica-symmetry assumption, are obtained by adapting the standard interpolation based on stochastic stability and are further corroborated by Monte Carlo simulations (and replica-trick outcomes for the sake of completeness). Finally, by relying upon an interpretation in terms of hidden units, in the last part of the work, we adapt the Battaglia-Treves model to cope with more general frameworks, such as bats flying in long tunnels.

cond-mat.dis-nn↗

arXiv2024

Inverse modeling of time-delayed interactions via the dynamic-entropy formalism

Although instantaneous interactions are unphysical, a large variety of maximum entropy statistical inference methods match the model-inferred and the empirically-measured equal-time correlation functions. Focusing on collective motion of active units, this constraint is reasonable when the interaction timescale is much faster than that of the interacting units, as in starling flocks, yet it fails in a number of counter examples, as in leukocyte coordination (where signalling proteins diffuse among two cells). Here, we relax this assumption and develop a path integral approach to maximum-entropy framework, which includes delay in signalling. Our method is able to infer the strength of couplings and fields, but also the time required by the couplings to completely transfer information among the units. We demonstrate the validity of our approach providing excellent results on synthetic datasets of non-Markovian trajectories generated by the Heisenberg-Kuramoto and Vicsek models equipped with delayed interactions. As a proof of concept, we also apply the method to experiments on dendritic migration, where matching equal-time correlations results in a significant information loss.

physics.bio-ph↗

arXiv2024

Fundamental operating regimes, hyper-parameter fine-tuning and glassiness: towards an interpretable replica-theory for trained restricted Boltzmann machines

We consider restricted Boltzmann machines with a binary visible layer and a Gaussian hidden layer trained by an unlabelled dataset composed of noisy realizations of a single ground pattern. We develop a statistical mechanics framework to describe the network generative capabilities, by exploiting the replica trick and assuming self-averaging of the underlying order parameters (i.e., replica symmetry). In particular, we outline the effective control parameters (e.g., the relative number of weights to be trained, the regularization parameter), whose tuning can yield qualitatively-different operative regimes. Further, we provide analytical and numerical evidence for the existence of a sub-region in the space of the hyperparameters where replica-symmetry breaking occurs.

cond-mat.dis-nn↗

arXiv2024

Regularization, early-stopping and dreaming: a Hopfield-like setup to address generalization and overfitting

In this work we approach attractor neural networks from a machine learning perspective: we look for optimal network parameters by applying a gradient descent over a regularized loss function. Within this framework, the optimal neuron-interaction matrices turn out to be a class of matrices which correspond to Hebbian kernels revised by a reiterated unlearning protocol. Remarkably, the extent of such unlearning is proved to be related to the regularization hyperparameter of the loss function and to the training time. Thus, we can design strategies to avoid overfitting that are formulated in terms of regularization and early-stopping tuning. The generalization capabilities of these attractor networks are also investigated: analytical results are obtained for random synthetic datasets, next, the emerging picture is corroborated by numerical experiments that highlight the existence of several regimes (i.e., overfitting, failure and success) as the dataset parameters are varied.

cs.LG↗

arXiv2024

A spectral approach to Hebbian-like neural networks

We consider the Hopfield neural network as a model of associative memory and we define its neuronal interaction matrix $\mathbf{J}$ as a function of a set of $K \times M$ binary vectors $\{\mathbfξ^{μ, A} \}_{μ=1,...,K}^{A=1,...,M}$ representing a sample of the reality that we want to retrieve. In particular, any item $\mathbfξ^{μ, A}$ is meant as a corrupted version of an unknown ground pattern $\mathbfζ^μ$, that is the target of our retrieval process. We consider and compare two definitions for $\mathbf{J}$, referred to as supervised and unsupervised, according to whether the class $μ$, each example belongs to, is unveiled or not, also, these definitions recover the paradigmatic Hebb's rule under suitable limits. The spectral properties of the resulting matrices are studied and used to inspect the retrieval capabilities of the related models as a function of their control parameters.

math-ph↗

arXiv2023

Parallel Learning by Multitasking Neural Networks

A modern challenge of Artificial Intelligence is learning multiple patterns at once (i.e.parallel learning). While this can not be accomplished by standard Hebbian associative neural networks, in this paper we show how the Multitasking Hebbian Network (a variation on theme of the Hopfield model working on sparse data-sets) is naturally able to perform this complex task. We focus on systems processing in parallel a finite (up to logarithmic growth in the size of the network) amount of patterns, mirroring the low-storage level of standard associative neural networks at work with pattern recognition. For mild dilution in the patterns, the network handles them hierarchically, distributing the amplitudes of their signals as power-laws w.r.t. their information content (hierarchical regime), while, for strong dilution, all the signals pertaining to all the patterns are raised with the same strength (parallel regime). Further, confined to the low-storage setting (i.e., far from the spin glass limit), the presence of a teacher neither alters the multitasking performances nor changes the thresholds for learning: the latter are the same whatever the training protocol is supervised or unsupervised. Results obtained through statistical mechanics, signal-to-noise technique and Monte Carlo simulations are overall in perfect agreement and carry interesting insights on multiple learning at once: for instance, whenever the cost-function of the model is minimized in parallel on several patterns (in its description via Statistical Mechanics), the same happens to the standard sum-squared error Loss function (typically used in Machine Learning).

cond-mat.dis-nn↗

arXiv2023

Dense Hebbian neural networks: a replica symmetric picture of unsupervised learning

We consider dense, associative neural-networks trained with no supervision and we investigate their computational capabilities analytically, via a statistical-mechanics approach, and numerically, via Monte Carlo simulations. In particular, we obtain a phase diagram summarizing their performance as a function of the control parameters such as the quality and quantity of the training dataset and the network storage, valid in the limit of large network size and structureless datasets. Moreover, we establish a bridge between macroscopic observables standardly used in statistical mechanics and loss functions typically used in the machine learning. As technical remarks, from the analytic side, we implement large deviations and stability analysis within Guerra's interpolation to tackle the not-Gaussian distributions involved in the post-synaptic potentials while, from the computational counterpart, we insert Plefka approximation in the Monte Carlo scheme, to speed up the evaluation of the synaptic tensors, overall obtaining a novel and broad approach to investigate neural networks in general.

cond-mat.dis-nn↗

arXiv2023

Dense Hebbian neural networks: a replica symmetric picture of supervised learning

We consider dense, associative neural-networks trained by a teacher (i.e., with supervision) and we investigate their computational capabilities analytically, via statistical-mechanics of spin glasses, and numerically, via Monte Carlo simulations. In particular, we obtain a phase diagram summarizing their performance as a function of the control parameters such as quality and quantity of the training dataset, network storage and noise, that is valid in the limit of large network size and structureless datasets: these networks may work in a ultra-storage regime (where they can handle a huge amount of patterns, if compared with shallow neural networks) or in a ultra-detection regime (where they can perform pattern recognition at prohibitive signal-to-noise ratios, if compared with shallow neural networks). Guided by the random theory as a reference framework, we also test numerically learning, storing and retrieval capabilities shown by these networks on structured datasets as MNist and Fashion MNist. As technical remarks, from the analytic side, we implement large deviations and stability analysis within Guerra's interpolation to tackle the not-Gaussian distributions involved in the post-synaptic potentials while, from the computational counterpart, we insert Plefka approximation in the Monte Carlo scheme, to speed up the evaluation of the synaptic tensors, overall obtaining a novel and broad approach to investigate supervised learning in neural networks, beyond the shallow limit, in general.

cond-mat.dis-nn↗

arXiv2023

Ultrametric identities in glassy models of Natural Evolution

Spin-glasses constitute a well-grounded framework for evolutionary models. Of particular interest for (some of) these models is the lack of self-averaging of their order parameters (e.g. the Hamming distance between the genomes of two individuals), even in asymptotic limits, much as like the behavior of the overlap between the configurations of two replica in mean-field spin-glasses. In the latter, this lack of self-averaging is related to peculiar fluctuations of the overlap, known as Ghirlanda-Guerra identities and Aizenman-Contucci polynomials, that cover a pivotal role in describing the ultrametric structure of the spin-glass landscape. As for evolutionary models, such identities may therefore be related to a taxonomic classification of individuals, yet a full investigation on their validity is missing. In this paper, we study ultrametric identities in simple cases where solely random mutations take place, while selective pressure is absent, namely in {\em flat landscape} models. In particular, we study three paradigmatic models in this setting: the {\em one parent model} (which, by construction, is ultrametric at the level of single individuals), the {\em homogeneous population model} (which is replica symmetric), and the {\em species formation model} (where a broken-replica scenario emerges at the level of species). We find analytical and numerical evidence that in the first and in the third model nor the Ghirlanda-Guerra neither the Aizenman-Contucci constraints hold, rather a new class of ultrametric identities is satisfied; in the second model all these constraints hold trivially. Very preliminary results on a real biological human genome derived by {\em The 1000 Genome Project Consortium} and on two artificial human genomes (generated by two different types neural networks) seem in better agreement with these new identities rather than the classic ones.

cond-mat.dis-nn↗

arXiv2022

Supervised Hebbian Learning

In neural network's Literature, Hebbian learning traditionally refers to the procedure by which the Hopfield model and its generalizations store archetypes (i.e., definite patterns that are experienced just once to form the synaptic matrix). However, the term "Learning" in Machine Learning refers to the ability of the machine to extract features from the supplied dataset (e.g., made of blurred examples of these archetypes), in order to make its own representation of the unavailable archetypes. Here, given a sample of examples, we define a supervised learning protocol by which the Hopfield network can infer the archetypes, and we detect the correct control parameters (including size and quality of the dataset) to depict a phase diagram for the system performance. We also prove that, for structureless datasets, the Hopfield model equipped with this supervised learning rule is equivalent to a restricted Boltzmann machine and this suggests an optimal and interpretable training routine. Finally, this approach is generalized to structured datasets: we highlight a quasi-ultrametric organization (reminiscent of replica-symmetry-breaking) in the analyzed datasets and, consequently, we introduce an additional "replica hidden layer" for its (partial) disentanglement, which is shown to improve MNIST classification from 75% to 95%, and to offer a new perspective on deep architectures.

cond-mat.dis-nn↗

arXiv2022

Pavlov Learning Machines

As well known, Hebb's learning traces its origin in Pavlov's Classical Conditioning, however, while the former has been extensively modelled in the past decades (e.g., by Hopfield model and countless variations on theme), as for the latter modelling has remained largely unaddressed so far; further, a bridge between these two pillars is totally lacking. The main difficulty towards this goal lays in the intrinsically different scales of the information involved: Pavlov's theory is about correlations among \emph{concepts} that are (dynamically) stored in the synaptic matrix as exemplified by the celebrated experiment starring a dog and a ring bell; conversely, Hebb's theory is about correlations among pairs of adjacent neurons as summarized by the famous statement {\em neurons that fire together wire together}. In this paper we rely on stochastic-process theory and model neural and synaptic dynamics via Langevin equations, to prove that -- as long as we keep neurons' and synapses' timescales largely split -- Pavlov mechanism spontaneously takes place and ultimately gives rise to synaptic weights that recover the Hebbian kernel.

cond-mat.dis-nn↗