arXiv subjects

D. Bolle

Publications and source records attributed to D. Bolle.

At least 19 recordsLinked to original sources

arXiv2007

Adaptive thresholds for neural networks with synaptic noise

The inclusion of a macroscopic adaptive threshold is studied for the retrieval dynamics of both layered feedforward and fully connected neural network models with synaptic noise. These two types of architectures require a different method to be solved numerically. In both cases it is shown that, if the threshold is chosen appropriately as a function of the cross-talk noise and of the activity of the stored patterns, adapting itself automatically in the course of the recall process, an autonomous functioning of the network is guaranteed. This self-control mechanism considerably improves the quality of retrieval, in particular the storage capacity, the basins of attraction and the mutual information content.

cond-mat.dis-nn↗

arXiv2006

Thermodynamics of spin systems on small-world hypergraphs

We study the thermodynamic properties of spin systems on small-world hypergraphs, obtained by superimposing sparse Poisson random graphs with p-spin interactions onto a one-dimensional Ising chain with nearest-neighbor interactions. We use replica-symmetric transfer-matrix techniques to derive a set of fixed-point equations describing the relevant order parameters and free energy, and solve them employing population dynamics. In the special case where the number of connections per site is of the order of the system size we are able to solve the model analytically. In the more general case where the number of connections is finite we determine the static and dynamic ferromagnetic-paramagnetic transitions using population dynamics. The results are tested against Monte-Carlo simulations.

cond-mat.dis-nn↗

arXiv2006

Adaptive thresholds for layered neural networks with synaptic noise

The inclusion of a macroscopic adaptive threshold is studied for the retrieval dynamics of layered feedforward neural network models with synaptic noise. It is shown that if the threshold is chosen appropriately as a function of the cross-talk noise and of the activity of the stored patterns, adapting itself automatically in the course of the recall process, an autonomous functioning of the network is guaranteed.This self-control mechanism considerably improves the quality of retrieval, in particular the storage capacity, the basins of attraction and the mutual information content.

cond-mat.dis-nn↗

arXiv2005

On metastable configurations of small-world networks

We calculate the number of metastable configurations of Ising small-world networks which are constructed upon superimposing sparse Poisson random graphs onto a one-dimensional chain. Our solution is based on replicated transfer-matrix techniques. We examine the denegeracy of the ground state and we find a jump in the entropy of metastable configurations exactly at the crossover between the small-world and the Poisson random graph structures. We also examine the difference in entropy between metastable and all possible configurations, for both ferromagnetic and bond-disordered long-range couplings.

cond-mat.dis-nn↗

arXiv2004

Multiplicative versus additive noise in multi-state neural networks

The effects of a variable amount of random dilution of the synaptic couplings in Q-Ising multi-state neural networks with Hebbian learning are examined. A fraction of the couplings is explicitly allowed to be anti-Hebbian. Random dilution represents the dying or pruning of synapses and, hence, a static disruption of the learning process which can be considered as a form of multiplicative noise in the learning rule. Both parallel and sequential updating of the neurons can be treated. Symmetric dilution in the statics of the network is studied using the mean-field theory approach of statistical mechanics. General dilution, including asymmetric pruning of the couplings, is examined using the generating functional (path integral) approach of disordered systems. It is shown that random dilution acts as additive gaussian noise in the Hebbian learning rule with a mean zero and a variance depending on the connectivity of the network and on the symmetry. Furthermore, a scaling factor appears that essentially measures the average amount of anti-Hebbian couplings.

cond-mat.dis-nn↗

arXiv2003

The signal-to-noise analysis of the Little-Hopfield model revisited

Using the generating functional analysis an exact recursion relation is derived for the time evolution of the effective local field of the fully connected Little-Hopfield model. It is shown that, by leaving out the feedback correlations arising from earlier times in this effective dynamics, one precisely finds the recursion relations usually employed in the signal-to-noise approach. The consequences of this approximation as well as the physics behind it are discussed. In particular, it is pointed out why it is hard to notice the effects, especially for model parameters corresponding to retrieval. Numerical simulations confirm these findings. The signal-to-noise analysis is then extended to include all correlations, making it a full theory for dynamics at the level of the generating functional analysis. The results are applied to the frequently employed extremely diluted (a)symmetric architectures and to sequence processing networks.

cond-mat.dis-nn↗

arXiv2003

Multi-state neural networks based upon spin-glasses: a biased overview

Recent results are reviewed on both the time evolution and retrieval properties of multi-state neural networks that are based upon spin-glass models. In particular, the properties of models with neuron states having Q-Ising symmetry are discussed for various architectures. The main common features and differences are highlighted.

cond-mat.dis-nn↗

arXiv2003

A layered neural network with three-state neurons optimizing the mutual information

The time evolution of an exactly solvable layered feedforward neural network with three-state neurons and optimizing the mutual information is studied for arbitrary synaptic noise (temperature). Detailed stationary temperature-capacity and capacity-activity phase diagrams are obtained. The model exhibits pattern retrieval, pattern-fluctuation retrieval and spin-glass phases. It is found that there is an improved performance in the form of both a larger critical capacity and information content compared with three-state Ising-type layered network models. Flow diagrams reveal that saddle-point solutions associated with fluctuation overlaps slow down considerably the flow of the network states towards the stable fixed-points.

cond-mat.dis-nn↗

arXiv2003

A spherical Hopfield model

We introduce a spherical Hopfield-type neural network involving neurons and patterns that are continuous variables. We study both the thermodynamics and dynamics of this model. In order to have a retrieval phase a quartic term is added to the Hamiltonian. The thermodynamics of the model is exactly solvable and the results are replica symmetric. A Langevin dynamics leads to a closed set of equations for the order parameters and effective correlation and response function typical for neural networks. The stationary limit corresponds to the thermodynamic results. Numerical calculations illustrate our findings.

cond-mat.dis-nn↗

arXiv2003

The Blume-Emery-Griffiths neural network: dynamics for arbitrary temperature

The parallel dynamics of the fully connected Blume-Emery-Griffiths neural network model is studied for arbitrary temperature. By employing a probabilistic signal-to-noise approach, a recursive scheme is found determining the time evolution of the distribution of the local fields and, hence, the evolution of the order parameters. A comparison of this approach is made with the generating functional method, allowing to calculate any physical relevant quantity as a function of time. Explicit analytic formula are given in both methods for the first few time steps of the dynamics. Up to the third time step the results are identical. Some arguments are presented why beyond the third time step the results differ for certain values of the model parameters. Furthermore, fixed-point equations are derived in the stationary limit. Numerical simulations confirm our theoretical findings.

cond-mat.dis-nn↗

arXiv2002

Optimal Capacity of the Blume-Emery-Griffiths perceptron

A Blume-Emery-Griffiths perceptron model is introduced and its optimal capacity is calculated within the replica-symmetric Gardner approach, as a function of the pattern activity and the imbedding stability parameter. The stability of the replica-symmetric approximation is studied via the analogue of the Almeida-Thouless line. A comparison is made with other three-state perceptrons.

cond-mat.dis-nn↗

arXiv2002

Thermodynamics of fully connected Blume-Emery-Griffiths neural networks

The thermodynamic and retrieval properties of fully connected Blume-Emery-Griffiths networks, storing ternary patterns, are studied using replica mean-field theory. Capacity-temperature phase diagrams are derived for several values of the pattern activity. It is found that the retrieval phase is the largest in comparison with other three-state neuron models. Furthermore, the meaning and stability of the so-called quadrupolar phase is discussed as a function of both the temperature and the pattern activity. Where appropriate, the results are compared with the diluted version of the model.

cond-mat.dis-nn↗

arXiv2002

Parallel dynamics of the fully connected Blume-Emery-Griffiths neural network

The parallel dynamics of the fully connected Blume-Emery-Griffiths neural network model is studied at zero temperature for arbitrary using a probabilistic approach. A recursive scheme is found determining the complete time evolution of the order parameters, taking into account all feedback correlations. It is based upon the evolution of the distribution of the local field, the structure of which is determined in detail. As an illustrative example, explicit analytic formula are given for the first few time steps of the dynamics. Furthermore, equilibrium fixed-point equations are derived and compared with the thermodynamic approach. The analytic results find excellent confirmation in extensive numerical simulations.

cond-mat.dis-nn↗

arXiv2001

An optimal Q-state neural network using mutual information

Starting from the mutual information we present a method in order to find a hamiltonian for a fully connected neural network model with an arbitrary, finite number of neuron states, Q. For small initial correlations between the neurons and the patterns it leads to optimal retrieval performance. For binary neurons, Q=2, and biased patterns we recover the Hopfield model. For three-state neurons, Q=3, we find back the recently introduced Blume-Emery-Griffiths network hamiltonian. We derive its phase diagram and compare it with those of related three-state models. We find that the retrieval region is the largest.

cond-mat.dis-nn↗

arXiv2000

The Ashkin-Teller neural network near saturation

The thermodynamic and retrieval properties of the Ashkin-Teller neural network model storing an infinite number of patterns are examined in the replica-symmetric mean-field approximation. In particular, for linked patterns temperature-capacity phase diagrams are derived for different values of the two-neuron and four-neuron coupling strengths. This model can be considered as a particular non-trivial generalisation of the Hopfield model and exhibits a number of interesting new features. Some aspects of replica-symmetry breaking are discussed.

cond-mat.dis-nn↗

arXiv1999

Q-Ising neural network dynamics: a comparative review of various architectures

This contribution reviews the parallel dynamics of Q-Ising neural networks for various architectures: extremely diluted asymmetric, layered feedforward, extremely diluted symmetric, and fully connected. Using a probabilistic signal-to-noise ratio analysis, taking into account all feedback correlations, which are strongly dependent upon these architectures the evolution of the distribution of the local field is found. This leads to a recursive scheme determining the complete time evolution of the order parameters of the network. Arbitrary Q and mainly zero temperature are considered. For the asymmetrically diluted and the layered feedforward network a closed-form solution is obtained while for the symmetrically diluted and fully connected architecture the feedback correlations prevent such a closed-form solution. For these symmetric networks equilibrium fixed-point equations can be derived under certain conditions on the noise in the system. They are the same as those obtained in a thermodynamic replica-symmetric mean-field theory approach.

cond-mat.dis-nn↗

arXiv1999

Correlated patterns in non-monotonic graded-response perceptrons

The optimal capacity of graded-response perceptrons storing biased and spatially correlated patterns with non-monotonic input-output relations is studied. It is shown that only the structure of the output patterns is important for the overall performance of the perceptrons.

cond-mat.dis-nn↗

arXiv1998

Parallel dynamics of extremely diluted symmetric Q-Ising neural networks

The parallel dynamics of extremely diluted symmetric Q-Ising neural networks is studied for arbitrary Q using a probabilistic approach. In spite of the extremely diluted architecture the feedback correlations arising from the symmetry prevent a closed-form solution in contrast with the extremely diluted asymmetric model. A recursive scheme is found determining the complete time evolution of the order parameters taking into account all feedback. It is based upon the evolution of the distribution of the local field, as in the fully connected model. As an illustrative example an explicit analysis is carried out for the Q=2 and Q=3 model. These results agree with and extend the partial results existing for Q=2. For Q>2 the analysis is entirely new. Finally, equilibrium fixed-point equations are derived and a capacity-gain function diagram is obtained.

cond-mat.dis-nn↗