arXiv subjects

Gilles Wainrib

Publications and source records attributed to Gilles Wainrib.

25 records · Page 2Linked to original sources

arXiv2014

Relative entropy minimizing noisy non-linear neural network to approximate stochastic processes

A method is provided for designing and training noise-driven recurrent neural networks as models of stochastic processes. The method unifies and generalizes two known separate modeling approaches, Echo State Networks (ESN) and Linear Inverse Modeling (LIM), under the common principle of relative entropy minimization. The power of the new method is demonstrated on a stochastic approximation of the El Nino phenomenon studied in climate research.

nlin.AO↗

arXiv2014

Index Distribution of the Ginibre Ensemble

Complex systems, and in particular random neural networks, are often described by randomly interacting dynamical systems with no specific symmetry. In that context, characterizing the number of relevant directions necessitates fine estimates on the Ginibre ensemble. In this Letter, we compute analytically the probability distribution of the number of eigenvalues $N_R$ with modulus greater than $R$ (the index) of a large $N\times N$ random matrix in the real or complex Ginibre ensemble. We show that the fraction $N_R/N=p$ has a distribution scaling as $\exp(-βN^2 ψ_R(p))$ with $β=1$ (respectively $β=1/2$) for the complex (resp. real) Ginibre ensemble. For any $p\in[0,1]$, the equilibrium spectral densities as well as the rate function $ψ_R(p)$ are explicitly derived. This function displays a third order phase transition at the critical (minimum) value $p^*_R=1-R^2$, associated to a phase transition of the Coulomb gas. We deduce that, in the central regime, the fluctuations of the index $N_R$ around its typical value $p^*_R N$ scale as $N^{1/3}$.

math.PR↗

arXiv2013

Synchronization in random balanced networks

Characterizing the in uence of network properties on the global emerging behavior of interacting elements constitutes a central question in many areas, from physical to social sciences. In this article we study a primary model of disordered neuronal networks with excitatory-inhibitory structure and balance constraints. We show how the interplay between structure and disorder in the connectivity leads to a universal transition from trivial to synchronized stationary or periodic states. This transition cannot be explained only through the analysis of the spectral density of the connectivity matrix. We provide a low dimensional approximation that shows the role of both the structure and disorder in the dynamics.

cond-mat.dis-nn↗

arXiv2013

A biological gradient descent for prediction through a combination of STDP and homeostatic plasticity

Identifying, formalizing and combining biological mechanisms which implement known brain functions, such as prediction, is a main aspect of current research in theoretical neuroscience. In this letter, the mechanisms of Spike Timing Dependent Plasticity (STDP) and homeostatic plasticity, combined in an original mathematical formalism, are shown to shape recurrent neural networks into predictors. Following a rigorous mathematical treatment, we prove that they implement the online gradient descent of a distance between the network activity and its stimuli. The convergence to an equilibrium, where the network can spontaneously reproduce or predict its stimuli, does not suffer from bifurcation issues usually encountered in learning in recurrent neural networks.

q-bio.NC↗

arXiv2013

Optimal system size for complex dynamics in random neural networks near criticality

In this Letter, we consider a model of dynamical agents coupled through a random connectivity matrix, as introduced in [Sompolinsky et. al, 1988] in the context of random neural networks. It is known that increasing the disorder parameter induces a phase transition leading to chaotic dynamics. We observe and investigate here a novel phenomenon in the subcritical regime : the probability of observing complex dynamics is maximal for an intermediate system size when the disorder is close enough to criticality. We give a more general explanation of this type of system size resonance in the framework of extreme values theory for eigenvalues of random matrices.

cond-mat.dis-nn↗

arXiv2013

Topological and Dynamical Complexity of Random Neural Networks

Random neural networks are dynamical descriptions of randomly interconnected neural units. These show a phase transition to chaos as a disorder parameter is increased. The microscopic mechanisms underlying this phase transition are unknown, and similarly to spin-glasses, shall be fundamentally related to the behavior of the system. In this Letter we investigate the explosion of complexity arising near that phase transition. We show that the mean number of equilibria undergoes a sharp transition from one equilibrium to a very large number scaling exponentially with the dimension on the system. Near criticality, we compute the exponential rate of divergence, called topological complexity. Strikingly, we show that it behaves exactly as the maximal Lyapunov exponent, a classical measure of dynamical complexity. This relationship unravels a microscopic mechanism leading to chaos which we further demonstrate on a simpler class of disordered systems, suggesting a deep and underexplored link between topological and dynamical complexity.

math-ph↗

arXiv2012

Limit theorems for infinite-dimensional piecewise deterministic Markov processes. Applications to stochastic excitable membrane models

We present limit theorems for a sequence of Piecewise Deterministic Markov Processes (PDMPs) taking values in a separable Hilbert space. This class of processes provides a rigorous framework for stochastic spatial models in which discrete random events are globally coupled with continuous space-dependent variables solving partial differential equations, e.g., stochastic hybrid models of excitable membranes. We derive a law of large numbers which establishes a connection to deterministic macroscopic models and a martingale central limit theorem which connects the stochastic fluctuations to diffusion processes. As a prerequisite we carry out a thorough discussion of Hilbert space valued martingales associated to the PDMPs. Furthermore, these limit theorems provide the basis for a general Langevin approximation to PDMPs, i.e., stochastic partial differential equations that are expected to be similar in their dynamics to PDMPs. We apply these results to compartmental-type models of spatially extended excitable membranes. Ultimately this yields a system of stochastic partial differential equations which models the internal noise of a biological excitable membrane based on a theoretical derivation from exact stochastic hybrid models.

math.PR↗