High-rank connectivity scaffolds support precision and generalisation in recurrent neural networks

Abstract

A major challenge in neuroscience and machine learning is to connect single-neuron influence, population dynamics, and circuit connectivity in a causal account of computation. Much previous work has shown that low-rank connectivity can generate low-dimensional dynamics in trained artificial neural networks, but this leaves unclear the functional relevance of the higher-rank structure of biological neural circuits and many artificial neuronal networks. Here we analyse recurrent neural networks trained to locate rewards by integrating continuously varying speed inputs in one- and two-dimensional spatial tasks. We find that dominant low-dimensional dynamics encode task locations, and can be causally manipulated to instruct behavioural outcomes. However, after decomposing the underlying circuitry we found that while low-rank connectivity accounts for the low-dimensional dynamics, accurate performance and generalisation to novel speed distributions requires high-rank connectivity. We demonstrate that this is achieved through distributed signalling that corrects errors in low-dimensional location representations. Thus, combined perturbation-, representation-, and circuit-level analyses demonstrate a novel mechanism for robust spatial computation and show how high-rank connectivity in neural circuits can provide a scaffold that supports precision and generalisation.