On the equivalence of Hopfield Networks and Boltzmann Machines

Adriano Barra; Alberto Bernacchia; Enrica Santucci; Pierluigi Contucci

Home/Archive/arXiv

arXiv · 1105.2790

On the equivalence of Hopfield Networks and Boltzmann Machines

Adriano Barra·Alberto Bernacchia·Enrica Santucci·Pierluigi Contucci

View original source ↗Read source document ↗Export metadata ↓

Abstract

A specific type of neural network, the Restricted Boltzmann Machine (RBM), is implemented for classification and feature detection in machine learning. RBM is characterized by separate layers of visible and hidden units, which are able to learn efficiently a generative model of the observed data. We study a "hybrid" version of RBM's, in which hidden units are analog and visible units are binary, and we show that thermodynamics of visible units are equivalent to those of a Hopfield network, in which the N visible units are the neurons and the P hidden units are the learned patterns. We apply the method of stochastic stability to derive the thermodynamics of the model, by considering a formal extension of this technique to the case of multiple sets of stored patterns, which may act as a benchmark for the study of correlated sets. Our results imply that simulating the dynamics of a Hopfield network, requiring the update of N neurons and the storage of N(N-1)/2 synapses, can be accomplished by a hybrid Boltzmann Machine, requiring the update of N+P neurons but the storage of only NP synapses. In addition, the well known glass transition of the Hopfield network has a counterpart in the Boltzmann Machine: It corresponds to an optimum criterion for selecting the relative sizes of the hidden and visible layers, resolving the trade-off between flexibility and generality of the model. The low storage phase of the Hopfield model corresponds to few hidden units and hence a overly constrained RBM, while the spin-glass phase (too many hidden units) corresponds to unconstrained RBM prone to overfitting of the observed data.

Explore related subjects

cond-mat.dis-nn cs.AI

Keep this discovery

BibTeX RIS

Adriano Barra, Alberto Bernacchia, Enrica Santucci, Pierluigi Contucci. 2011-05-13. On the equivalence of Hopfield Networks and Boltzmann Machines. https://arxiv.org/abs/1105.2790

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Irrationality Measure Controls Long-Wavelength Charge Fluctuations in Quasiperiodic Systems

Quasiperiodic order is characterized by irrational frequencies whose rational approximability known as irrationality measure defines distinct arithmetic classes. We establish that this arithmetic classification has direct physical consequences for long-wavelength charge fluctuations. In translation-covariant quasiperiodic systems, the infrared scaling of charge fluctuation is governed by the interplay between the irrationality exponent of irrational frequency and the large-harmonic decay of the hull charge profile: the latter determines the available charge weight, while the former controls how efficiently that weight is transferred to the infrared. Consequently, all algebraic irrational frequencies share the same arithmetic scaling, whereas exceptionally well-approximable transcendental frequencies can exhibit strongly enhanced infrared fluctuation scaling. We further prove that occupied states separated from the Fermi level by a gap stable throughout the hull contribute only an analytic infrared background, leaving the nontrivial scaling to near-Fermi states. Our results extend to general translation-covariant multi-frequency quasiperiodic systems.

cond-mat.dis-nn↗

arXiv2026

Curvature-Induced Geometric Universality in Non-Hermitian Anderson Transitions

In Euclidean space, universality classes of Anderson transitions are primarily determined by symmetry and spatial dimensionality. Here, we present evidence for a geometry-controlled universality class of non-Hermitian Anderson transitions on hyperbolic-like lattices. In this setting, critical behavior is influenced by the large-scale hyperbolic geometry, characterized by negative curvature, exponential volume growth, and a non-Euclidean notion of spatial scaling. Finite-size scaling of participation ratios across several distinct $ \{p,q\} $ tilings reveals one-parameter scaling collapses with a common critical exponent $ \nu\simeq1 $ within numerical accuracy. A complementary phenomenological coarse-grained Landau-Ginzburg analysis shows how exponential correlation-volume growth suppresses critical fluctuations, offering a rationale for the observed mean-field-like scaling. Our results suggest that spatial curvature can act as an additional organizing principle for Anderson-transition universality beyond the conventional dimensionality- and symmetry-based classification.

cond-mat.dis-nn↗

arXiv2026

Finite-rank multiplicative perturbations of rotationally invariant non-Hermitian random matrices

We study finite-rank multiplicative deformations of rotationally invariant non-Hermitian random matrices. More precisely, we consider models of the form $\mathbf{A}(\mathbf{I}+\mathbf{T})$, where $\mathbf{A}$ is a large rotationally invariant non-Hermitian random matrix, $\mathbf{T}$ is a finite-rank normal perturbation, and $\mathbf{I}$ denotes the identity matrix. We characterize the emergence of outlier eigenvalues, their fluctuations, and the associated eigenvector overlaps. Our results provide a multiplicative non-Hermitian counterpart to the classical Baik--Ben Arous--P\'ech\'e framework.

cond-mat.dis-nn↗