Searcharxiv⌕ Search

arXiv · cond-mat/0301457

A replica free evaluation of the neuronal population information with mixed continuous and discrete stimuli: from the linear to the asymptotic regime

Abstract

Recent studies have explored theoretically the ability of populations of neurons to carry information about a set of stimuli, both in the case of purely discrete or purely continuous stimuli, and in the case of multidimensional continuous angular and discrete correlates, in presence of additional quenched disorder in the distribution. An analytical expression for the mutual information has been obtained in the limit of large noise by means of the replica trick. Here we show that the same results can actually be obtained in most cases without the use of replicas, by means of a much simpler expansion of the logarithm. Fitting the theoretical model to real neuronal data, we show that the introduction of correlations in the quenched disorder improves the fit, suggesting a possible role of signal correlations-actually detected in real data- in a redundant code. We show that even in the more difficult analysis of the asymptotic regime, an explicit expression for the mutual information can be obtained without resorting to the replica trick despite the presence of quenched disorder, both with a gaussian and with a more realistic thresholded-gaussian model. When the stimuli are mixed continuous and discrete, we find that with both models the information seem to grow logarithmically to infinity with the number of neurons and with the inverse of the noise, even though the exact general dependence cannot be derived explicitly for the thresholded gaussian model. In the large noise limit lower values of information were obtained with the thresholded-gaussian model, for a fixed value of the noise and of the population size. On the contrary, in the asymptotic regime, with very low values of the noise, a lower information value is obtained with the gaussian model.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Valeria Del Prete. 2003-01-23. A replica free evaluation of the neuronal population information with mixed continuous and discrete stimuli: from the linear to the asymptotic regime. https://doi.org/10.1103/physreve.68.031906

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

Embedding stochastic dynamics in latent manifolds of recurrent neural networks

Recurrent neural networks (RNNs) provide a theoretical framework for understanding computation in biological neural circuits through their ability to support rich time-dependent behaviour. Whilst high-dimensional by construction, they can be used to encode low-dimensional dynamical systems, for example by restricting to low-rank connectivity, hence mirroring the emergence of latent neural manifolds found in empirical neural data. Here, we exploit a relationship between RNNs and a class of universal approximators to train networks that can represent arbitrary, nonlinear stochastic differential equations (SDEs), with given drift and diffusion coefficients, within a low-dimensional latent subspace. We show that RNNs can faithfully embed the drift and diffusion of a given SDE, including nonlinear and nonequilibrium dynamics such as chaotic attractors. We apply this approach by constructing RNN realisations of stochastic systems that transiently explore various attractors through both input-driven switching and autonomous transitions driven by nonequilibrium currents, which we interpret as models of associative and sequential (episodic) memory. To elucidate how these dynamics are encoded in the network, we introduce decompositions of the RNN based on its asymmetric connectivity and its time-irreversibility. Our results show that neural populations can implement a broad class of stochastic dynamics within low-dimensional manifolds, unifying ideas from the dynamics of neural computation, the theory of neural manifolds, associative memory, and nonequilibrium statistical mechanics.

cond-mat.dis-nn↗

Deep-layered machines have a built-in Occam's razor

Many complex systems map vast numbers of microscopic descriptions onto far fewer macroscopic outcomes. Surprisingly, these input-output maps strongly favor simple outputs. In an exactly solvable deep-layered machine, we prove that simplicity bias strengthens with every layer. The probability of an output function $F$ approaches $\Pr(F)\propto 2^{-K(F)}$, where $K$ is the Kolmogorov complexity. Our results show that hierarchical composition alone can generate an intrinsic Occam's razor, suggesting a general mechanism for simplicity bias in learning and evolution.

cond-mat.dis-nn↗

Cup and Cap Topological Neural Network

Topological Deep Learning (TDL) is designed to learn features associated not only with nodes but also with edges, triangles and higher dimensional simplices. However, most TDL approaches, being based on boundary operators and Hodge Laplacians, have the limitation that features and signals cannot be lifted or lowered across more than one dimension per layer. To overcome this limitation, in this work, we propose the adoption of the cup and cap products. Specifically, we formulate the Cup and Cap Topological Neural Network (CCNN), a topological deep learning architecture designed to learn node-based variables (or 0-cochains) by taking into account their many-body interactions (e.g. triangles) present in the data. We validate CCNN by assessing its performance on the TopoBench datasets, demonstrating its competitive performance with respect to other simplicial complex neural networks.

cond-mat.dis-nn↗