Information and Topology in Attractor Neural Network

Abstract

A wide range of networks, including small-world topology, can be modelled by the connectivity $γ$, and randomness $ω$ of the links. Both learning and attractor abilities of a neural network can be measured by the mutual information (MI), as a function of the load rate and overlap between patterns and retrieval states. We use MI to search for the optimal topology, for storage and attractor properties of the network. We find that, while the largest storage implies an optimal $MI(γ,ω)$ at $γ_{opt}(ω)\to 0$, the largest basin of attraction leads to an optimal topology at moderate levels of $γ_{opt}$, whenever $0\leqω<0.3$. This $γ_{opt}$ is related to the clustering and path-length of the network. We also build a diagram for the dynamical phases with random and local initial overlap, and show that very diluted networks lose their attractor ability.