arXiv subjects

Nadezhda Semenova

Publications and source records attributed to Nadezhda Semenova.

At least 19 recordsLinked to original sources

arXiv2025

Impact of internal noise on convolutional neural networks

In this paper, we investigate the impact of noise on a simplified trained convolutional network. The types of noise studied originate from a real optical implementation of a neural network, but we generalize these types to enhance the applicability of our findings on a broader scale. The noise types considered include additive and multiplicative noise, which relate to how noise affects individual neurons, as well as correlated and uncorrelated noise, which pertains to the influence of noise across one layers. We demonstrate that the propagation of uncorrelated noise primarily depends on the statistical properties of the connection matrices. Specifically, the mean value of the connection matrix following the layer impacted by noise governs the propagation of correlated additive noise, while the mean of its square contributes to the accumulation of uncorrelated noise. Additionally, we propose an analytical assessment of the noise level in the network's output signal, which shows a strong correlation with the results of numerical simulations.

nlin.AO↗

arXiv2025

Internal noise in hardware deep and recurrent neural networks helps with learning

Recently, the field of hardware neural networks has been actively developing, where neurons and their connections are not simulated on a computer but are implemented at the physical level, transforming the neural network into a tangible device. In this paper, we investigate how internal noise during the training of neural networks affects the final performance of recurrent and deep neural networks. We consider feedforward networks (FNN) and echo state networks (ESN) as examples. The types of noise examined originated from a real optical implementation of a neural network. However, these types were subsequently generalized to enhance the applicability of our findings on a broader scale. The noise types considered include additive and multiplicative noise, which depend on how noise influences each individual neuron, and correlated and uncorrelated noise, which pertains to the impact of noise on groups of neurons (such as the hidden layer of FNNs or the reservoir of ESNs). In this paper, we demonstrate that, in most cases, both deep and echo state networks benefit from internal noise during training, as it enhances their resilience to noise. Consequently, the testing performance at the same noise intensities is significantly higher for networks trained with noise than for those trained without it. Notably, only multiplicative correlated noise during training has minimal has almost no impact on both deep and recurrent networks.

cs.NE↗

arXiv2024

Impact of white noise in artificial neural networks trained for classification: performance and noise mitigation strategies

In recent years, the hardware implementation of neural networks, leveraging physical coupling and analog neurons has substantially increased in relevance. Such nonlinear and complex physical networks provide significant advantages in speed and energy efficiency, but are potentially susceptible to internal noise when compared to digital emulations of such networks. In this work, we consider how additive and multiplicative Gaussian white noise on the neuronal level can affect the accuracy of the network when applied for specific tasks and including a softmax function in the readout layer. We adapt several noise reduction techniques to the essential setting of classification tasks, which represent a large fraction of neural network computing. We find that these adjusted concepts are highly effective in mitigating the detrimental impact of noise.

cs.LG↗

arXiv2024

Impact of white Gaussian internal noise on analog echo-state neural networks

In recent years, more and more works have appeared devoted to the analog (hardware) implementation of artificial neural networks, in which neurons and the connection between them are based not on computer calculations, but on physical principles. Such networks offer improved energy efficiency and, in some cases, scalability, but may be susceptible to internal noise. This paper studies the influence of noise on the functioning of recurrent networks using the example of trained echo state networks (ESNs). The most common reservoir connection matrices were chosen as various topologies of ESNs: random uniform and band matrices with different connectivity. White Gaussian noise was chosen as the influence, and according to the way of its introducing it was additive or multiplicative, as well as correlated or uncorrelated. In the paper, we show that the propagation of noise in reservoir is mainly controlled by the statistical properties of the output connection matrix, namely the mean and the mean square. Depending on these values, more correlated or uncorrelated noise accumulates in the network. We also show that there are conditions under which even noise with an intensity of $10^{-20}$ is already enough to completely lose the useful signal. In the article we show which types of noise are most critical for networks with different activation functions (hyperbolic tangent, sigmoid and linear) and if the network is self-closed.

cs.NE↗

arXiv2024

Impact of pulse exposure on chimera state in ensemble of FitzHugh-Nagumo systems

In this article we consider the influence of a periodic sequence of Gaussian pulses on a chimera state in a ring of coupled FitzHugh-Nagumo systems. We found that on the way to complete spatial synchronization one can observe a number of variations of chimera states that are not typical for the parameter range under consideration. For example, the following modes were found: breathing chimera, chimera with intermittency in the incoherent part, traveling chimera with strong intermittency, and others. For comparison, here we also consider the impact of a harmonic influence on the same chimera, and to preserve the generality of the conclusions, we compare the regimes caused by both a purely positive harmonic influence and a positive-negative one.

nlin.AO↗

arXiv2023

Delay-induced self-oscillation excitation in the FitzHugh-Nagumo model: regular and chaotic dynamics

The stochastic FitzHugh-Nagumo model with time delayed-feedback is often studied in excitable regime to demonstrate the time-delayed control of coherence resonance. Here, we show that the impact of time-delayed feedback in the FitzHugh-Nagumo neuron is not limited by control of noise-induced oscillation regularity (coherence), but also results in excitation of the regular and chaotic self-oscillatory dynamics in the deterministic model. We demonstrate this numerically by means of simulations, linear stability analysis, the study of Lyapunov exponents and basins of attraction for both positive and negative delayed-feedback strengths. It has been established that one can implement a route to chaos in the explored model, where the intrinsic peculiarities of the Feigenbaum scenario are exhibited. For large time delay, we complement the study of temporal evolution by the interpretation of the dynamics as patterns in virtual space.

nlin.AO↗

arXiv2023

Symbiosis of an artificial neural network and models of biological neurons: training and testing

In this paper we show the possibility of creating and identifying the features of an artificial neural network (ANN) which consists of mathematical models of biological neurons. The FitzHugh--Nagumo (FHN) system is used as an example of model demonstrating simplified neuron activity. First, in order to reveal how biological neurons can be embedded within an ANN, we train the ANN with nonlinear neurons to solve a a basic image recognition problem with MNIST database; and next, we describe how FHN systems can be introduced into this trained ANN. After all, we show that an ANN with FHN systems inside can be successfully trained and its accuracy becomes larger. What has been done above opens up great opportunities in terms of the direction of analog neural networks, in which artificial neurons can be replaced by biological ones. \end{abstract}

cs.NE↗

arXiv2023

Simple method for detecting sleep episodes in rats ECoG using machine learning

In this paper we propose a new method for the automatic recognition of the state of behavioral sleep (BS) and waking state (WS) in freely moving rats using their electrocorticographic (ECoG) data. Three-channels ECoG signals were recorded from frontal left, frontal right and occipital right cortical areas. We employed a simple artificial neural network (ANN), in which the mean values and standard deviations of ECoG signals from two or three channels were used as inputs for the ANN. Results of wavelet-based recognition of BS/WS in the same data were used to train the ANN and evaluate correctness of our classifier. We tested different combinations of ECoG channels for detecting BS/WS. Our results showed that the accuracy of ANN classification did not depend on ECoG-channel. For any ECoG-channel, networks were trained on one rat and applied to another rat with an accuracy of at least 80~\%. Itis important that we used a very simple network topology to achieve a relatively high accuracy of classification. Our classifier was based on a simple linear combination of input signals with some weights, and these weights could be replaced by the averaged weights of all trained ANNs without decreases in classification accuracy. In all, we introduce a new sleep recognition method that does not require additional network training. It is enough to know the coefficients and the equations suggested in this paper. The proposed method showed very fast performance and simple computations, therefore it could be used in real time experiments. It might be of high demand in preclinical studies in rodents that require vigilance control or monitoring of sleep-wake patterns.

cs.NE↗

arXiv2022

Noise mitigation strategies in physical feedforward neural networks

Physical neural networks are promising candidates for next generation artificial intelligence hardware. In such architectures, neurons and connections are physically realized and do not leverage digital concepts with their practically infinite signal-to-noise ratio to encode, transduce and transform information. They therefore are prone to noise with a variety of statistical and architectural properties, and effective strategies leveraging network-inherent assets to mitigate noise in an hardware-efficient manner are important in the pursuit of next generation neural network hardware. Based on analytical derivations, we here introduce and analyse a variety of different noise-mitigation approaches. We analytically show that intra-layer connections in which the connection matrix's squared mean exceeds the mean of its square fully suppresses uncorrelated noise. We go beyond and develop two synergistic strategies for noise that is uncorrelated and correlated across populations of neurons. First, we introduce the concept of ghost neurons, where each group of neurons perturbed by correlated noise has a negative connection to a single neuron, yet without receiving any input information. Secondly, we show that pooling of neuron populations is an efficient approach to suppress uncorrelated noise. As such, we developed a general noise mitigation strategy leveraging the statistical properties of the different noise terms most relevant in analogue hardware. Finally, we demonstrate the effectiveness of this combined approach for trained neural network classifying the MNIST handwritten digits, for which we achieve a 4-fold improvement of the output signal-to-noise ratio and increase the classification accuracy almost to the level of the noise-free network.

cs.NE↗

arXiv2021

Understanding and mitigating noise in trained deep neural networks

Deep neural networks unlocked a vast range of new applications by solving tasks of which many were previously deemed as reserved to higher human intelligence. One of the developments enabling this success was a boost in computing power provided by special purpose hardware, such as graphic or tensor processing units. However, these do not leverage fundamental features of neural networks like parallelism and analog state variables. Instead, they emulate neural networks relying on binary computing, which results in unsustainable energy consumption and comparatively low speed. Fully parallel and analogue hardware promises to overcome these challenges, yet the impact of analogue neuron noise and its propagation, i.e. accumulation, threatens rendering such approaches inept. Here, we determine for the first time the propagation of noise in deep neural networks comprising noisy nonlinear neurons in trained fully connected layers. We study additive and multiplicative as well as correlated and uncorrelated noise, and develop analytical methods that predict the noise level in any layer of symmetric deep neural networks or deep neural networks trained with back propagation. We find that noise accumulation is generally bound, and adding additional network layers does not worsen the signal to noise ratio beyond a limit. Most importantly, noise accumulation can be suppressed entirely when neuron activation functions have a slope smaller than unity. We therefore developed the framework for noise in fully connected deep neural networks implemented in analog systems, and identify criteria allowing engineers to design noise-resilient novel neural network hardware.

cs.NE↗

arXiv2021

Unbalanced clustering and solitary states in coupled excitable systems

We demonstrate the mechanisms of emergence and the link between two types of symmetry-broken states, the unbalanced periodic two-cluster states and solitary states, in coupled excitable systems with prevalent repulsive interactions. Solitary states in non-locally coupled arrays inherit their dynamical features from unbalanced cluster states in globally coupled networks. Apart from self-organization based on phase-locked synchrony, interplay of excitability and local multiscale dynamics also gives rise to leap-frog states involving alternating order of spiking. Noise suppresses multistability of cluster states and induces pattern homogenization, transforming solitary states into patterns of patched

nlin.AO↗

arXiv2021

Non-invasive analysis of blood-brain barrier permeability based on wavelet and machine learning approaches

The blood-brain barrier plays a decisive role in protecting the brain from toxins and pathogens. The ability to analyze the BBB opening (OBBB) is crucial for the treatment of many brain diseases, but it is very difficult to noninvasively monitor OBBB. In this paper we analyse the EEG series of healthy rats in free behaviour and after music-induced OBBB. The research is performed using two completely different methods based on wavelet analysis and machine learning approach. Both methods enable us to recognize OBBB and are in a good agreement with each other. The comparative analysis was carried out using F-measures and ROC-curves.

nlin.AO↗

arXiv2019

Chimera states in ensembles of excitable FitzHugh-Nagumo systems

An ensemble of nonlocally coupled excitable FitzHugh-Nagumo systems is studied. In the presence of noise the explored system can exhibit a special kind of chimera states called coherence-resonance chimera. As previously thought, noise plays principal role in forming these structures. It is shown in the present paper that these regimes appear because of the specific coupling between the elements. The action of coupling involve a spatial wave regime, which occurs in ensemble of excitable nodes even if the noise is switched off. In addition, a new chimera state is obtained in an excitable regime. It is shown that the noise makes this chimera more stable near an Andronov-Hopf bifurcation.

nlin.AO↗

arXiv2019

Fundamental aspects of noise in analog-hardware neural networks

We study and analyze the fundamental aspects of noise propagation in recurrent as well as deep, multi-layer networks. The main focus of our study are neural networks in analogue hardware, yet the methodology provides insight for networks in general. The system under study consists of noisy linear nodes, and we investigate the signal-to-noise ratio at the network's outputs which is the upper limit to such a system's computing accuracy. We consider additive and multiplicative noise which can be purely local as well as correlated across populations of neurons. This covers the chief internal-perturbations of hardware networks and noise amplitudes were obtained from a physically implemented recurrent neural network and therefore correspond to a real-world system. Analytic solutions agree exceptionally well with numerical data, enabling clear identification of the most critical components and aspects for noise management. Focusing on linear nodes isolates the impact of network connections and allows us to derive strategies for mitigating noise. Our work is the starting point in addressing this aspect of analogue neural networks, and our results identify notoriously sensitive points while simultaneously highlighting the robustness of such computational systems.

cs.ET↗

arXiv2018

Mechanism of solitary state appearance in an ensemble of nonlocally coupled Lozi maps

We study the peculiarities of the solitary state appearance in the ensemble of nonlocally coupled chaotic maps. We show that nonlocal coupling and features of the partial elements lead to arising of multistability in the system. The existence of solitary state is caused by formation of two attractive sets with different basins of attraction. Their basins are analysed depending on coupling parameters.

nlin.AO↗

arXiv2017

New type of chimera and mutual synchronization of spatiotemporal structures in two coupled ensembles of nonlocally interacting chaotic maps

We study numerically the dynamics of a network made of two coupled one-dimensional ensembles of discrete-time systems. The first ensemble is represented by a ring of nonlocally coupled Henon maps, and the second one - by a ring of nonlocally coupled Lozi maps. We find that the network of coupled ensembles can realize all the spatio-temporal structures which are observed both in the Henon map ensemble and in the Lozi map ensemble when uncoupled. Moreover, we reveal a new type of spatiotemporal structure, a solitary state chimera, in the considered network. We also establish and describe the effect of mutual synchronization of various complex spatiotemporal patterns in the system of two coupled ensembles of Henon and Lozi maps.

nlin.AO↗

arXiv2017

Time-delayed feedback control of coherence resonance chimeras

Using the model of a FitzHugh-Nagumo system in the excitable regime we investigate the influence of time-delayed feedback on noise-induced chimera states in a network with nonlocal coupling, i.e., coherence resonance chimeras. It is shown that time-delayed feedback allows for control of the range of parameter values where these chimera states occur. Moreover, for the feedback delay close to the intrinsic period of the system we find a novel regime which we call period-two coherence resonance chimera.

nlin.AO↗

arXiv2016

Noise-induced chimera states in a neural network

We show that chimera patterns can be induced by noise in nonlocally coupled neural networks in the excitable regime. In contrast to classical chimeras, occurring in noise-free oscillatory networks, they have features of two phenomena: coherence resonance and chimera states. Therefore, we call them coherence-resonance chimeras. These patterns demonstrate the constructive role of noise and appear for intermediate values of noise intensity, which is a characteristic feature of coherence resonance. In the coherence-resonance chimera state a neural network of identical elements splits into two coexisting domains with different behavior: spatially coherent and spatially incoherent, a typical property of chimera states. Moreover, these noise-induced chimera states are characterized by alternating behavior: coherent and incoherent domains switch periodically their location. We show that this alternating switching can be explained by analyzing the coupling functions.

nlin.AO↗