Criticality of a Stochastic Dense Associative Memory Model with Exponential Interaction Function

Abstract

The Hopfield network (HN) is a classical model of associative memory with stored patterns encoded as minima of an energy function shaped by a Hebbian learning rule. Dense Associative Memory (DAM) models introduce n-body interactions among neurons with n greater than 2 and, more recently, also exponential interaction functions, which significantly improve the network's storing capacity. While the emergence of phase transitions in HN and DAM were extensively studied, the investigation of exponential DAM is still in its early stages. Further, an equilibrium thermodynamical condition is typically assumed, while out-of-equilibrium dynamics are not considered. Here, we study the temporal dynamics of a stochastic exponential DAM (SEDAM) with a multiplicative salt-and-pepper noise and trained on the MNIST dataset. While taking the noise probability p as control parameter, the time-averaged overlap Q and the diffusion scaling H are taken as order parameters, being H related to the network's time correlation features. The MNIST-based SEDAM is also compared with a SEDAM trained on standard Rademacher patterns and with a stochastic HN (SHN). We found the emergence of a phase transition in both Q and H, with the critical noise level pc decreasing as the load K increases. For each load K, Q highlights a transition between a sub-critical and a super-critical regime, both with short-time correlated dynamics. Conversely, in the critical regime of the MNIST-based SEDAM the network displays long-time correlated dynamics with highly persistent temporal memory marked by the high value H around 1.25. Similar behaviors are observed for both models trained with Rademacher patterns, but with a slightly higher temporal memory index H around 1.5.