Searcharxiv⌕ Search

arXiv subjects

Carlo Orientale Caputo

Publications and source records attributed to Carlo Orientale Caputo.

2 recordsLinked to original sources

Dynamical mean-field theory of latching Potts networks

Spontaneous transitions between memories, as observed in internally generated cognition such as mind-wandering and free association, require neural dynamics capable of destabilizing one attractive memory state and moving towards another. We ask what minimal representational structure is required for such transitions to emerge in random associative networks. To address this question, we study Potts associative networks with $S$ active states per unit and slow adaptive thresholds. The $S=1$ case can model a standard Hopfield-like network of binary units. We employ dynamical mean-field theory in the extensive-load regime, where the number of stored patterns scales linearly with network size and connectivity, and we focus on transitions between two condensed memories, which are singled out in the connectivity matrix. In the finite-load regime, we show that transitions are governed by an effective drive term, whose sign and temporal evolution distinguish high-latching (structured), low-latching (random), and non-latching regimes. The resulting phase diagram shows that adaptation alone is not sufficient: transitions only occur within certain ranges of the overlap between the two memories. In the binary case $S=1$, in particular, no unit can be active in both memories but in different Potts states, thus strongly supressing the high-latching, structured transitions that exist for $S>1$. In the extensive-load regime, non-condensed memories generate colored fluctuations and retarded self-interactions through an effective memory kernel. We find that latching trajectories change the effective timescale of this kernel, a dynamical signature of transitions in the extensive-load regime. These results indicate that multi-state Potts-like representations are essential for structured memory-to-memory transitions, a hallmark of spontaneous thought.

q-bio.NC↗

Absolute abstraction: a renormalisation group approach

Abstraction is the process of extracting the essential features from raw data while ignoring irrelevant details. It is well known that abstraction emerges with depth in neural networks, where deep layers capture abstract characteristics of data by combining lower level features encoded in shallow layers (e.g. edges). Yet we argue that depth alone is not enough to develop truly abstract representations. We advocate that the level of abstraction crucially depends on how broad the training set is. We address the issue within a renormalisation group approach where a representation is expanded to encompass a broader set of data. We take the unique fixed point of this transformation -- the Hierarchical Feature Model -- as a candidate for a representation which is absolutely abstract. This theoretical picture is tested in numerical experiments based on Deep Belief Networks and auto-encoders trained on data of different breadth. These show that representations in neural networks approach the Hierarchical Feature Model as the data get broader and as depth increases, in agreement with theoretical predictions.

cs.LG↗