arXiv subjects

Robert Rosenbaum

Publications and source records attributed to Robert Rosenbaum.

15 recordsLinked to original sources

arXiv2026

Empirical scaling laws in balanced networks with conductance-based synapses

Strongly coupled, recurrent, balanced network models have been successful in describing and predicting many phenomena observed in cortical neural recordings. However, most balanced network models use current-based synapse models in place of more realistic, conductance-based models. Conductance-based synapse models predict unrealistically small membrane potential variability. On the other hand, introducing realistic levels of spike time correlations to models with current-based synapses predicts unrealistically large membrane potential variability. We use computer simulations to show that these two effects can cancel: Recurrent network models with conductance-based synapses and spike time correlations produce more realistic, moderate levels of membrane potential variability. Consistent with recent work on feedforward networks, our results show that including more realistic modeling assumptions produces more realistic dynamics, but only if when two modeling assumptions are included together.

q-bio.NC↗

arXiv2025

High-dimensional dynamics in low-dimensional networks

Many networks in nature and applications have an approximate low-rank structure in the sense that their connectivity structure is dominated by a few dimensions. It is natural to expect that dynamics on such networks would also be low-dimensional. Indeed, theoretical results show that low-rank networks produce low-dimensional dynamics whenever the network is isolated from external perturbations or input. However, networks in nature are rarely isolated. Here, we study the dimensionality of dynamics in recurrent networks with low-dimensional structure driven by high-dimensional inputs or perturbations. We find that dynamics in such networks can be high- or low-dimensional and we derive mathematical conditions on the network structure under which linearized dynamics are high-dimensional. In many low-rank networks, dynamics are suppressed in directions aligned with the network's low-rank structure, a phenomenon we term ``low-rank suppression.'' We show that several low-rank network structures arising in nature satisfy the conditions for generating high-dimensional dynamics and low-rank suppression. Our results clarify important, but counterintuitive relationships between a recurrent network's connectivity structure and the structure of its response to external input.

math.DS↗

arXiv2025

Oja's plasticity rule overcomes several challenges of training neural networks under biological constraints

Deep neural networks have achieved impressive performance through carefully engineered training strategies. Nonetheless, such methods lack parallels in biological neural circuits, relying heavily on non-local credit assignment, precise initialization, normalization layers, batch processing, and large datasets. Biologically plausible plasticity rules, such as random feedback alignment, often suffer from instability and unbounded weight growth without these engineered methods, while Hebbian-type schemes fail to provide goal-oriented credit. In this study, we demonstrate that incorporating Oja's plasticity rule into error-driven training yields stable, efficient learning in feedforward and recurrent architectures, obviating the need for carefully engineered tricks. Our results show that Oja's rule preserves richer activation subspaces, mitigates exploding or vanishing signals, and improves short-term memory in recurrent networks. Notably, meta-learned local plasticity rules incorporating Oja's principle not only match but surpass standard backpropagation in data-scarce regimes. These findings reveal a biologically grounded pathway bridging engineered deep networks and plausible synaptic mechanisms.

q-bio.NC↗

arXiv2024

On the relationship between predictive coding and backpropagation

Artificial neural networks are often interpreted as abstract models of biological neuronal networks, but they are typically trained using the biologically unrealistic backpropagation algorithm and its variants. Predictive coding has been proposed as a potentially more biologically realistic alternative to backpropagation for training neural networks. This manuscript reviews and extends recent work on the mathematical relationship between predictive coding and backpropagation for training feedforward artificial neural networks on supervised learning tasks. Implications of these results for the interpretation of predictive coding and deep neural networks as models of biological learning are discussed along with a repository of functions, Torch2PC, for performing predictive coding with PyTorch neural network models.

q-bio.NC↗

arXiv2023

Learning fixed points of recurrent neural networks by reparameterizing the network model

In computational neuroscience, fixed points of recurrent neural networks are commonly used to model neural responses to static or slowly changing stimuli. These applications raise the question of how to train the weights in a recurrent neural network to minimize a loss function evaluated on fixed points. A natural approach is to use gradient descent on the Euclidean space of synaptic weights. We show that this approach can lead to poor learning performance due, in part, to singularities that arise in the loss surface. We use a reparameterization of the recurrent network model to derive two alternative learning rules that produces more robust learning dynamics. We show that these learning rules can be interpreted as steepest descent and gradient descent, respectively, under a non-Euclidean metric on the space of recurrent weights. Our results question the common, implicit assumption that learning in the brain should be expected to follow the negative Euclidean gradient of synaptic weights.

q-bio.NC↗

arXiv2023

Meta-Learning Biologically Plausible Plasticity Rules with Random Feedback Pathways

Backpropagation is widely used to train artificial neural networks, but its relationship to synaptic plasticity in the brain is unknown. Some biological models of backpropagation rely on feedback projections that are symmetric with feedforward connections, but experiments do not corroborate the existence of such symmetric backward connectivity. Random feedback alignment offers an alternative model in which errors are propagated backward through fixed, random backward connections. This approach successfully trains shallow models, but learns slowly and does not perform well with deeper models or online learning. In this study, we develop a meta-learning approach to discover interpretable, biologically plausible plasticity rules that improve online learning performance with fixed random feedback connections. The resulting plasticity rules show improved online training of deep models in the low data regime. Our results highlight the potential of meta-learning to discover effective, interpretable learning rules satisfying biological constraints.

q-bio.NC↗

arXiv2022

Evaluating the extent to which homeostatic plasticity learns to compute prediction errors in unstructured neuronal networks

The brain is believed to operate in part by making predictions about sensory stimuli and encoding deviations from these predictions in the activity of "prediction error neurons." This principle defines the widely influential theory of predictive coding. The precise circuitry and plasticity mechanisms through which animals learn to compute and update their predictions are unknown. Homeostatic inhibitory synaptic plasticity is a promising mechanism for training neuronal networks to perform predictive coding. Homeostatic plasticity causes neurons to maintain a steady, baseline firing rate in response to inputs that closely match the inputs on which a network was trained, but firing rates can deviate away from this baseline in response to stimuli that are mismatched from training. We combine computer simulations and mathematical analysis systematically to test the extent to which randomly connected, unstructured networks compute prediction errors after training with homeostatic inhibitory synaptic plasticity. We find that homeostatic plasticity alone is sufficient for computing prediction errors for trivial time-constant stimuli, but not for more realistic time-varying stimuli. We use a mean-field theory of plastic networks to explain our findings and characterize the assumptions under which they apply.

q-bio.NC↗

arXiv2020

Synaptic Plasticity in Correlated Balanced Networks

The dynamics of local cortical networks are irregular, but correlated. Dynamic excitatory--inhibitory balance is a plausible mechanism that generates such irregular activity, but it remains unclear how balance is achieved and maintained in plastic neural networks. In particular, it is not fully understood how plasticity induced changes in the network affect balance, and in turn, how correlated, balanced activity impacts learning. How does the dynamics of balanced networks change under different plasticity rules? How does correlated spiking activity in recurrent networks change the evolution of weights, their eventual magnitude, and structure across the network? To address these questions, we develop a general theory of plasticity in balanced networks. We show that balance can be attained and maintained under plasticity induced weight changes. We find that correlations in the input mildly, but significantly affect the evolution of synaptic weights. Under certain plasticity rules, we find an emergence of correlations between firing rates and synaptic weights. Under these rules, synaptic weights converge to a stable manifold in weight space with their final configuration dependent on the initial state of the network. Lastly, we show that our framework can also describe the dynamics of plastic balanced networks when subsets of neurons receive targeted optogenetic input.

q-bio.NC↗

arXiv2019

A model of reward-modulated motor learning with parallelcortical and basal ganglia pathways

Many recent studies of the motor system are divided into two distinct approaches: Those that investigate how motor responses are encoded in cortical neurons' firing rate dynamics and those that study the learning rules by which mammals and songbirds develop reliable motor responses. Computationally, the first approach is encapsulated by reservoir computing models, which can learn intricate motor tasks and produce internal dynamics strikingly similar to those of motor cortical neurons, but rely on biologically unrealistic learning rules. The more realistic learning rules developed by the second approach are often derived for simplified, discrete tasks in contrast to the intricate dynamics that characterize real motor responses. We bridge these two approaches to develop a biologically realistic learning rule for reservoir computing. Our algorithm learns simulated motor tasks on which previous reservoir computing algorithms fail, and reproduces experimental findings including those that relate motor learning to Parkinson's disease and its treatment.

q-bio.NC↗

arXiv2017

From the statistics of connectivity to the statistics of spike times in neuronal networks

An essential step toward understanding neural circuits is linking their structure and their dynamics. In general, this relationship can be almost arbitrarily complex. Recent theoretical work has, however, begun to identify some broad principles underlying collective spiking activity in neural circuits. The first is that local features of network connectivity can be surprisingly effective in predicting global statistics of activity across a network. The second is that, for the important case of large networks with excitatory-inhibitory balance, correlated spiking persists or vanishes depending on the spatial scales of recurrent and feedforward connectivity. We close by showing how these ideas, together with plasticity rules, can help to close the loop between network structure and activity statistics.

q-bio.NC↗

arXiv2016

Spatiotemporal dynamics and reliable computations in recurrent spiking neural networks

Randomly connected networks of excitatory and inhibitory spiking neurons provide a parsimonious model of neural variability, but are notoriously unreliable for performing computations. We show that this difficulty is overcome by incorporating the well-documented dependence of connection probability on distance. Spatially extended spiking networks exhibit symmetry-breaking bifurcations and generate spatiotemporal patterns that can be trained to perform dynamical computations.

q-bio.NC↗

arXiv2016

Highly connected neurons spike less frequently in balanced networks

Many biological neuronal networks exhibit highly variable spiking activity. Balanced networks offer a parsimonious model of this variability. In balanced networks, strong excitatory synaptic inputs are canceled by strong inhibitory inputs on average and spiking activity is driven by transient breaks in this balance. Most previous studies of balanced networks assume a homogeneous or distance-dependent connectivity structure, but connectivity in biological cortical networks is more intricate. We use a heterogeneous mean-field theory of balanced networks to show that heterogeneous in-degrees can break balance, but balance can be restored by heterogeneous out-degrees that are correlated with in-degrees. In all examples considered, we find that highly connected neurons spike less frequently, consistent with recent experimental observations.

q-bio.NC↗

arXiv2013

Balanced networks of spiking neurons with spatially dependent recurrent connections

Networks of model neurons with balanced recurrent excitation and inhibition produce irregular and asynchronous spiking activity. We extend the analysis of balanced networks to include the known dependence of connection probability on the spatial separation between neurons. In the continuum limit we derive that stable, balanced firing rate solutions require that the spatial spread of external inputs be broader than that of recurrent excitation, which in turn must be broader than or equal to that of recurrent inhibition. For finite size networks we investigate the pattern forming dynamics arising when balanced conditions are not satisfied. The spatiotemporal dynamics of balanced networks offer new challenges in the statistical mechanics of complex systems.

q-bio.NC↗

arXiv2012

The impact of short term synaptic depression and stochastic vesicle dynamics on neural variability

Neural variability plays a central role in neural coding and neuronal network dynamics. Unreliability of synaptic transmission is a major source of neural variability: synaptic neurotransmitter vesicles are released probabilistically in response to presynaptic spikes and are recovered stochastically in time. The stochastic dynamics of this process interacts with variability in the arrival times of presynaptic spikes to shape the variability of the postsynaptic response. We use continuous time Markov chain methods to analyze a model of short term synaptic depression with stochastic vesicle dynamics coupled with three different models of presynaptic spiking: one model in which the timing of presynaptic spikes are modeled as a Poisson process, one in which spikes occur more regularly than a Poisson process and one in which spikes occur more irregularly. We use this analysis to investigate how variability in a presynaptic spike train is transformed by short term synaptic depression and stochastic vesicle dynamics to determine the variability of the postsynaptic response to a presynaptic spike train. We find that regular presynaptic spiking increases the average rate at which vesicles are released, that the number of vesicles released over a time window is more variable for smaller time windows than for larger time windows, and that fast presynaptic spiking gives rise to Poisson-like variability of the postsynaptic response even when presynaptic spike times are highly non-Poisson. Our results complement and extend previously reported theoretical results and provide possible explanations for some trends observed in recorded data.

q-bio.NC↗

arXiv2010

Finite volume and asymptotic methods for stochastic neuron models with correlated inputs

We consider a pair of stochastic integrate and fire neurons receiving correlated stochastic inputs. The evolution of this system can be described by the corresponding Fokker-Planck equation with non-trivial boundary conditions resulting from the refractory period and firing threshold. We propose a finite volume method that is orders of magnitude faster than the Monte Carlo methods traditionally used to model such systems. The resulting numerical approximations are proved to be accurate, nonnegative and integrate to 1. We also approximate the transient evolution of the system using an Ornstein--Uhlenbeck process, and use the result to examine the properties of the joint output of cell pairs. The results suggests that the joint output of a cell pair is most sensitive to changes in input variance, and less sensitive to changes in input mean and correlation.

q-bio.NC↗