Transient dynamics of associative memory models

Abstract

Associative memory models such as the Hopfield network and its dense generalizations with higher-order interactions exhibit a "blackout catastrophe" -- a discontinuous transition where stable memory states abruptly vanish when the number of stored patterns exceeds a critical capacity. This transition is often interpreted as rendering networks unusable beyond capacity limits. We argue that this interpretation is largely an artifact of the equilibrium perspective. We derive dynamical mean-field equations for graded-activity dense associative memory models, with the Hopfield model as a special case, using a bipartite cavity approach. We solve the resulting self-consistent equations using an iterative numerical scheme. We show that patterns can be transiently retrieved with high accuracy above capacity despite the absence of stable attractors. This occurs because slow regions persist in the above-capacity energy landscape near stored patterns as lingering traces of the stable basins that existed below capacity. The same transient-retrieval effect occurs in below-capacity networks initialized outside basins of attraction. "Transient-recovery curves" provide a concise visual summary of these effects, revealing graceful, non-catastrophic changes in retrieval behavior above capacity and allowing us to compare the behavior across interaction orders. This dynamical perspective reveals energy landscape structure obscured by equilibrium analysis, including slow regions near stored patterns that persist above capacity, and suggests biological neural circuits may exploit transient dynamics for memory retrieval. Furthermore, our approach suggests ways of understanding computational properties of neural circuits without reference to fixed points and yields new theoretical results on generalizations of the Hopfield model.