arXiv subjects

Ido Kanter

Publications and source records attributed to Ido Kanter.

At least 37 records · Page 2Linked to original sources

arXiv2022

Recurrent neural networks that generalize from examples and optimize by dreaming

The gap between the huge volumes of data needed to train artificial neural networks and the relatively small amount of data needed by their biological counterparts is a central puzzle in machine learning. Here, inspired by biological information-processing, we introduce a generalized Hopfield network where pairwise couplings between neurons are built according to Hebb's prescription for on-line learning and allow also for (suitably stylized) off-line sleeping mechanisms. Moreover, in order to retain a learning framework, here the patterns are not assumed to be available, instead, we let the network experience solely a dataset made of a sample of noisy examples for each pattern. We analyze the model by statistical-mechanics tools and we obtain a quantitative picture of its capabilities as functions of its control parameters: the resulting network is an associative memory for pattern recognition that learns from examples on-line, generalizes and optimizes its storage capacity by off-line sleeping. Remarkably, the sleeping mechanisms always significantly reduce (up to $\approx 90\%$) the dataset size required to correctly generalize, further, there are memory loads that are prohibitive to Hebbian networks without sleeping (no matter the size and quality of the provided examples), but that are easily handled by the present "rested" neural networks.

cond-mat.dis-nn↗

arXiv2022

Long anisotropic absolute refractory periods with rapid rise-times to reliable responsiveness

Refractoriness is a fundamental property of excitable elements, such as neurons, indicating the probability for re-excitation in a given time-lag, and is typically linked to the neuronal hyperpolarization following an evoked spike. Here we measured the refractory periods (RPs) in neuronal cultures and observed that an average anisotropic absolute RP could exceed 10 milliseconds and its tail 20 milliseconds, independent of a large stimulation frequency range. It is an order of magnitude longer than anticipated and comparable with the decaying membrane potential timescale. It is followed by a sharp rise-time (relative RP) of merely ~1 millisecond to complete responsiveness. Extracellular stimulations result in longer absolute RPs than solely intracellular ones, and a pair of extracellular stimulations from two different routes exhibits distinct absolute RPs, depending on their order. Our results indicate that a neuron is an accurate excitable element, where the diverse RPs cannot be attributed solely to the soma and imply fast mutual interactions between different stimulation routes and dendrites. Further elucidation of neuronal computational capabilities and their interplay with adaptation mechanisms is warranted.

q-bio.NC↗

arXiv2021

Significant anisotropic neuronal refractory period plasticity

Refractory periods are an unavoidable feature of excitable elements, resulting in necessary time-lags for re-excitation. Herein, we measure neuronal absolute refractory periods (ARPs) in synaptic blocked neuronal cultures. In so doing, we show that their duration can be significantly extended by dozens of milliseconds using preceding evoked spikes generated by extracellular stimulations. The ARP increases with the frequency of preceding stimulations, and saturates at the intermittent phase of the neuronal response latency, where a short relative refractory period might appear. Nevertheless, preceding stimulations via a different extracellular route does not affect the ARP. It is also found to be independent of preceding intracellular stimulations. All these features strongly suggest that the anisotropic ARPs originate in neuronal dendrites. The results demonstrate the fast and significant plasticity of the neuronal ARP, depending on the firing activity of its connecting neurons, which is expected to affect network dynamics.

q-bio.NC↗

arXiv2020

Brain experiments imply adaptation mechanisms which outperform common AI learning algorithms

Attempting to imitate the brain functionalities, researchers have bridged between neuroscience and artificial intelligence for decades; however, experimental neuroscience has not directly advanced the field of machine learning. Here, using neuronal cultures, we demonstrate that increased training frequency accelerates the neuronal adaptation processes. This mechanism was implemented on artificial neural networks, where a local learning step-size increases for coherent consecutive learning steps and tested on a simple dataset of handwritten digits, MNIST. Based on our online learning results with a few handwriting examples, success rates for brain-inspired algorithms substantially outperform the commonly used machine learning algorithms. We speculate this emerging bridge from slow brain function to machine learning will promote ultrafast decision making under limited examples, which is the reality in many aspects of human activity, robotic control, and network optimization.

q-bio.NC↗

arXiv2019

Embedding information in physically generated random bit sequences while maintaining certified randomness

Ultrafast physical random bit generation at hundreds of Gb/s rates, with verified randomness, is a crucial ingredient in secure communication and have recently emerged using optics based physical systems. Here we examine the inverse problem and measure the ratio of information bits that can be systematically embedded in a random bit sequence without degrading its certified randomness. These ratios exceed 0.01 in experimentally obtained long random bit sequences. Based on these findings we propose a high-capacity private-key cryptosystem with a finite key length, where the existence as well as the content of the communication is concealed in the random sequence. Our results call for a rethinking of the current quantitative definition of practical classical randomness as well as the measure of randomness generated by quantum methods, which have to include bounds using the proposed inverse information embedding method.

eess.SP↗

arXiv2017

Vitality of Neural Networks under Reoccurring Catastrophic Failures

Catastrophic failures are complete and sudden collapses in the activity of large networks such as economics, electrical power grids and computer networks, which typically require a manual recovery process. Here we experimentally show that excitatory neural networks are governed by a non-Poissonian reoccurrence of catastrophic failures, where their repetition time follows a multimodal distribution characterized by a few tenths of a second and tens of seconds timescales. The mechanism underlying the termination and reappearance of network activity is quantitatively shown here to be associated with nodal time-dependent features, neuronal plasticity, where hyperactive nodes damage the response capability of their neighbors. It presents a complementary mechanism for the emergence of Poissonian catastrophic failures from damage conductivity. The effect that hyperactive nodes degenerate their neighbors represents a type of local competition which is a common feature in the dynamics of real-world complex networks, whereas their spontaneous recoveries represent a vitality which enhances reliable functionality.

q-bio.NC↗

arXiv2017

Simultaneous multi-patch-clamp and extracellular-array recordings: Single neuron reflects network activity

The increasing number of recording electrodes enhances the capability of capturing the network's cooperative activity, however, using too many monitors might alter the properties of the measured neural network and induce noise. Using a technique that merges simultaneous multi-patch-clamp and multi-electrode array recordings of neural networks in-vitro, we show that the membrane potential of a single neuron is a reliable and super-sensitive probe for monitoring such cooperative activities and their detailed rhythms. Specifically, the membrane potential and the spiking activity of a single neuron are either highly correlated or highly anti-correlated with the time-dependent macroscopic activity of the entire network. This surprising observation also sheds light on the cooperative origin of neuronal burst in cultured networks. Our findings present an alternative flexible approach to the technique based on a massive tiling of networks by large-scale arrays of electrodes to monitor their activity.

q-bio.NC↗

arXiv2017

Fast Reversible Learning based on Neurons functioning as Anisotropic Multiplex Hubs

Neural networks are composed of neurons and synapses, which are responsible for learning in a slow adaptive dynamical process. Here we experimentally show that neurons act like independent anisotropic multiplex hubs, which relay and mute incoming signals following their input directions. Theoretically, the observed information routing enriches the computational capabilities of neurons by allowing, for instance, equalization among different information routes in the network, as well as high-frequency transmission of complex time-dependent signals constructed via several parallel routes. In addition, this kind of hubs adaptively eliminate very noisy neurons from the dynamics of the network, preventing masking of information transmission. The timescales for these features are several seconds at most, as opposed to the imprint of information by the synaptic plasticity, a process which exceeds minutes. Results open the horizon to the understanding of fast and adaptive learning realities in higher cognitive functionalities of the brain.

q-bio.NC↗

arXiv2017

Oscillations in networks of networks stem from adaptive nodes with memory

We present an analytical framework that allows the quantitative study of statistical dynamic properties of networks with adaptive nodes that have memory and is used to examine the emergence of oscillations in networks with response failures. The frequency of the oscillations was quantitatively found to increase with the excitability of the nodes and with the average degree of the network and to decrease with delays between nodes. For networks of networks, diverse cluster oscillation modes were found as a function of the topology. Analytical results are in agreement with large-scale simulations and open the horizon for understanding network dynamics composed of finite memory nodes as well as their different phases of activity.

q-bio.NC↗

arXiv2016

Mimicking Collective Firing Patterns of Hundreds of Connected Neurons using a Single-Neuron Experiment

The experimental study of neural networks requires simultaneous measurements of a massive number of neurons, while monitoring properties of the connectivity, synaptic strengths and delays. Current technological barriers make such a mission unachievable. In addition, as a result of the enormous number of required measurements, the estimated network parameters would differ from the original ones. Here we present a versatile experimental technique, which enables the study of recurrent neural networks activity while being capable of dictating the network connectivity and synaptic strengths. This method is based on the observation that the response of neurons depends solely on their recent stimulations, a short-term memory. It allows a long-term scheme of stimulation and recording of a single neuron, to mimic simultaneous activity measurements of neurons in a recurrent network. Utilization of this technique demonstrates the spontaneous emergence of cooperative synchronous oscillations, in particular the coexistence of fast Gamma and slow Delta oscillations, and opens the horizon for the experimental study of other cooperative phenomena within large-scale neural networks.

q-bio.NC↗

arXiv2015

Broadband Macroscopic Cortical Oscillations Emerge from Intrinsic Neuronal Response Failures

Broadband spontaneous macroscopic neural oscillations are rhythmic cortical firing which were extensively examined during the last century, however, their possible origination is still controversial. In this work we show how macroscopic oscillations emerge in solely excitatory random networks and without topological constraints. We experimentally and theoretically show that these oscillations stem from the counterintuitive underlying mechanism - the intrinsic stochastic neuronal response failures. These neuronal response failures, which are characterized by short-term memory, lead to cooperation among neurons, resulting in sub- or several- Hertz macroscopic oscillations which coexist with high frequency gamma oscillations. A quantitative interplay between the statistical network properties and the emerging oscillations is supported by simulations of large networks based on single-neuron in-vitro experiments and a Langevin equation describing the network dynamics. Results call for the examination of these oscillations in the presence of inhibition and external drives.

q-bio.NC↗

arXiv2015

Neuronal Response Impedance Mechanism Implementing Cooperative Networks with Low Firing Rates and Microseconds Precision

Realizations of low firing rates in neural networks usually require globally balanced distributions among excitatory and inhibitory links, while feasibility of temporal coding is limited by neuronal millisecond precision. We show that cooperation, governing global network features, emerges through nodal properties, as opposed to link distributions. Using in vitro and in vivo experiments we demonstrate microsecond precision of neuronal response timings under low stimulation frequencies, whereas moderate frequencies result in a chaotic neuronal phase characterized by degraded precision. Above a critical stimulation frequency, which varies among neurons, response failures were found to emerge stochastically such that the neuron functions as a low pass filter, saturating the average inter-spike-interval. This intrinsic neuronal response impedance mechanism leads to cooperation on a network level, such that firing rates are suppressed towards the lowest neuronal critical frequency simultaneously with neuronal microsecond precision. Our findings open up opportunities of controlling global features of network dynamics through few nodes with extreme properties.

q-bio.NC↗

arXiv2014

Chaotic and non-chaotic phases in experimental responses of a single neuron

Consistency and predictability of brain functionalities depend on reproducible activity of a single neuron. We identify a reproducible non-chaotic neuronal phase where deviations between concave response latency profiles of a single neuron do not increase with the number of stimulations. A chaotic neuronal phase emerges at a transition to convex latency profiles which diverge exponentially, indicating irreproducible response timings. Our findings are supported by a quantitative mathematical framework and found robust to periodic and random stimulation patterns. In addition, these results put a bound on the neuronal temporal resolution which can be enhanced below a millisecond using neuronal chains.

q-bio.NC↗

arXiv2014

Computational paradigm for dynamic logic-gates in neuronal activity

In 1943 McCulloch and Pitts suggested that the brain is composed of reliable logic-gates similar to the logic at the core of today's computers. This framework had a limited impact on neuroscience, since neurons exhibit far richer dynamics. Here we propose a new experimentally corroborated paradigm in which the truth tables of the brain's logic-gates are time dependent, i.e. dynamic logicgates (DLGs). The truth tables of the DLGs depend on the history of their activity and the stimulation frequencies of their input neurons. Our experimental results are based on a procedure where conditioned stimulations were enforced on circuits of neurons embedded within a large-scale network of cortical cells in-vitro. We demonstrate that the underlying biological mechanism is the unavoidable increase of neuronal response latencies to ongoing stimulations, which imposes a nonuniform gradual stretching of network delays. The limited experimental results are confirmed and extended by simulations and theoretical arguments based on identical neurons with a fixed increase of the neuronal response latency per evoked spike. We anticipate our results to lead to better understanding of the suitability of this computational paradigm to account for the brain's functionalities and will require the development of new systematic mathematical methods beyond the methods developed for traditional Boolean algebra.

q-bio.QM↗

arXiv2014

Error correction and fast detectors implemented by ultra-fast neuronal plasticity

We experimentally show that the neuron functions as a precise time-integrator, where the accumulated changes in neuronal response latencies, under complex and random stimulation patterns, are solely a function of a global quantity, the average time-lag between stimulations. In contrast, momentary leaps in the neuronal response latency follow trends of consecutive stimulations, indicating ultra-fast neuronal plasticity. On a circuit level, this ultra-fast neuronal plasticity phenomenon implements error-correction mechanisms and fast detectors for misplaced stimulations. Additionally, at moderate/high stimulation rates this phenomenon destabilizes/stabilizes a periodic neuronal activity disrupted by misplaced stimulations.

q-bio.NC↗

arXiv2013

Sudden synchrony leaps accompanied by frequency multiplications in neuronal activity

A classical view of neural coding relies on temporal firing synchrony among functional groups of neurons; however the underlying mechanism remains an enigma. Here we experimentally demonstrate a mechanism where time-lags among neuronal spiking leap from several tens of milliseconds to nearly zero-lag synchrony. It also allows sudden leaps out of synchrony, hence forming short epochs of synchrony. Our results are based on an experimental procedure where conditioned stimulations were enforced on circuits of neurons embedded within a large-scale network of cortical cells in vitro and are corroborated by simulations of neuronal populations. The underlying biological mechanisms are the unavoidable increase of the neuronal response latency to ongoing stimulations and temporal or spatial summation required to generate evoked spikes. These sudden leaps in and out of synchrony may be accompanied by multiplications of the neuronal firing frequency, hence offering reliable information-bearing indicators which may bridge between the two principal neuronal coding paradigms.

q-bio.NC↗

arXiv2013

An experimental evidence-based computational paradigm for new logic-gates in neuronal activity

We propose a new experimentally corroborated paradigm in which the functionality of the brain's logic-gates depends on the history of their activity, e.g. an OR-gate that turns into a XOR-gate over time. Our results are based on an experimental procedure where conditioned stimulations were enforced on circuits of neurons embedded within a large-scale network of cortical cells in-vitro. The underlying biological mechanism is the unavoidable increase of neuronal response latency to ongoing stimulations, which imposes a non-uniform gradual stretching of network delays.

q-bio.NC↗

arXiv2013

Chaos Pass Filter: Linear Response of Synchronized Chaotic Systems

The linear response of synchronized chaotic units with delayed couplings and feedback to small external perturbations is investigated in the context of communication with chaos synchronization. For iterated chaotic maps, the distribution of distances is calculated numerically and, for some special cases, analytically as well. Depending on model parameters, this distribution has power law tails leading to diverging moments of distances in the region of synchronization. The corresponding linear equations have multiplicative and additive noise due to perturbations and chaos. The response to small harmonic perturbations shows resonances related to coupling and feedback delay times. For perturbation from a binary message the bit error rate is calculated. The bit error rate is not related to the transverse Lyapunov exponents, and it can be reduced when additional noise is added to the transmitted signal. For some special cases, the bit error rate as a function of coupling strength has the structure of a devil's staircase, related to an iterated function system. Finally, the security of communication is discussed by comparing uni- and bi-directional couplings.

nlin.CD↗