When Response Variability Increases Neural Network Robustness to Synaptic Noise

Gleb Basalyga; Emilio Salinas

Home/Archive/arXiv

arXiv · q-bio/0510036

When Response Variability Increases Neural Network Robustness to Synaptic Noise

Gleb Basalyga·Emilio Salinas

View original source ↗Read source document ↗Export metadata ↓

Abstract

Cortical sensory neurons are known to be highly variable, in the sense that responses evoked by identical stimuli often change dramatically from trial to trial. The origin of this variability is uncertain, but it is usually interpreted as detrimental noise that reduces the computational accuracy of neural circuits. Here we investigate the possibility that such response variability might, in fact, be beneficial, because it may partially compensate for a decrease in accuracy due to stochastic changes in the synaptic strengths of a network. We study the interplay between two kinds of noise, response (or neuronal) noise and synaptic noise, by analyzing their joint influence on the accuracy of neural networks trained to perform various tasks. We find an interesting, generic interaction: when fluctuations in the synaptic connections are proportional to their strengths (multiplicative noise), a certain amount of response noise in the input neurons can significantly improve network performance, compared to the same network without response noise. Performance is enhanced because response noise and multiplicative synaptic noise are in some ways equivalent. These results are demonstrated analytically for the most basic network consisting of two input neurons and one output neuron performing a simple classification task, but computer simulations show that the phenomenon persists in a wide range of architectures, including recurrent (attractor) networks and sensory-motor networks that perform coordinate transformations. The results suggest that response variability could play an important dynamic role in networks that continuously learn.

Explore related subjects

q-bio.NC cond-mat.dis-nn

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Gleb Basalyga, Emilio Salinas. 2005-10-18. When Response Variability Increases Neural Network Robustness to Synaptic Noise. https://arxiv.org/abs/q-bio/0510036

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

BEAST3D: Animal behavioral analysis and neural encoding from multi-view video via Gaussian splatting

Multi-view video recordings are increasingly used to capture the 3D movements of animals in experimental settings, yet extracting rich 3D representations from these recordings remains challenging. Supervised pose estimation requires extensive manual annotation, while general-purpose 3D reconstruction models trained on generic scene datasets fail on the specialized imagery and sparse-view setting of laboratory experiments. We address these limitations with BEAST3D, a self-supervised pretraining framework that learns 3D visual representations from unlabeled, calibrated multi-view video. BEAST3D uses a vision transformer to predict 3D Gaussian splats that reconstruct held-out views through differentiable rendering, while simultaneously segmenting the animal from the background. BEAST3D reconstructs 3D structure with as few as four views by conditioning directly on known camera parameters--unlike general-purpose models, which must estimate camera geometry from dense overlapping viewpoints that are seldom available in lab settings. Through comprehensive evaluation across four species, we demonstrate that BEAST3D produces rich, viewpoint-invariant features that transfer effectively to three downstream tasks: novel view synthesis, which validates the quality of the learned 3D representations; multi-view pose estimation, which provides the sparse keypoint trajectories widely used in behavioral analysis; and neural encoding, which relates 3D behavioral features to simultaneously recorded neural activity. BEAST3D thus establishes a versatile framework for behavioral analysis that leverages 3D structure in modern multi-view laboratory recordings.

q-bio.NC↗

arXiv2026

Self-Supervised Learning of Local Predictive Directions as a Neural Computation

Natural sensory dynamics are globally nonlinear but often exhibit simple local geometry, with nearby trajectories evolving preferentially along a small number of predictive directions. We ask how neural circuits could discover this geometry directly from temporal experience. We introduce Local Predictive Directions (LPD), a self-supervised framework that learns local directions whose projections maximally predict near-future trajectory evolution. LPD applies past-future canonical correlation analysis to temporal differences, making the learned representation invariant to translations of the underlying state while avoiding estimation of equilibria or a global dynamical model. To represent systems with multiple predictive regimes, we introduce a non-negative similarity-matching objective, with offline and online algorithms, that partitions trajectory segments into predictive domains, each associated with a distinct local predictive direction. The online algorithm admits a natural neural-network interpretation: the resulting activities provide soft domain assignments, while synaptic weights encode the corresponding local predictive filters. On stochastic nonlinear dynamical systems, LPD recovers the local predictive geometry and partitions trajectories into distinct dynamical regimes. On naturalistic sensory streams, the online algorithm produces center-surround spatial receptive fields and ON/OFF sustained and transient temporal filters resembling those of early visual neurons. Our results suggest that predictive learning can provide a self-supervised mechanism by which neural circuits discover the local geometry of environmental dynamics and represent it in receptive fields.

q-bio.NC↗

arXiv2026

Effective pruning of task-trained recurrent neural networks using noisy fluctuations and connection rescaling

Pruning network connections is key to brain function but, despite its importance, there exist few biologically-plausible pruning rules with demonstrated good performance. We evaluate noise-prune, a recently introduced unsupervised local pruning rule for recurrent networks that uses noisy fluctuations to determine the importance of connections. Noise-prune has previously only been empirically tested on random networks without a specific computational function. We show that noise-prune preserves task-performance in task-trained recurrent neural networks, greatly outperforming a strategy that only uses connection magnitude and exceeding a non-local strategy that uses second-order information. Rather than deterministically removing connections that fall below a threshold, noise-prune samples connections to preserve based on their importance and strengthens retained connections to preserve average strength. We show that this sampling and rescaling is essential to good performance, and can augment other pruning strategies. We also find that the optimal empirical degree of rescaling is lower than predicted by the original theoretical argument, and thus develop a new version of noise-prune that outperforms the previous version. Our results demonstrate that a biologically-motivated pruning strategy based on local activity statistics can effectively preserve task performance in functional recurrent networks by combining activity-dependent connection selection with compensatory strengthening of surviving connections.

q-bio.NC↗