arXiv subjects

A. C. C. Coolen

Publications and source records attributed to A. C. C. Coolen.

60 records · Page 4Linked to original sources

arXiv1997

Statistical Mechanical Analysis of the Dynamics of Learning in Perceptrons

We describe the application of tools from statistical mechanics to analyse the dynamics of various classes of supervised learning rules in perceptrons. The character of this paper is mostly that of a cross between a biased non-encyclopedic review and lecture notes: we try to present a coherent and self-contained picture of the basics of this field, to explain the ideas and tricks, to show how the predictions of the theory compare with (simulation) experiments, and to bring together scattered results. Technical details are given explicitly in an appendix. In order to avoid distraction we concentrate the references in a final section. In addition this paper contains some new results: (i) explicit solutions of the macroscopic equations that describe the error evolution for on-line and batch learning rules, (ii) an analysis of the dynamics of arbitrary macroscopic observables (for complete and incomplete trainingsets), leading to a general Fokker-Planck equation, and (iii) the macroscopic laws describing batch learning with complete training sets. We close the paper with a preliminary expose of ongoing research on the dynamics of learning for the case where the training set is incomplete (i.e. where the number of examples scales linearly with the network size).

cond-mat.dis-nn↗

arXiv1996

Feed-Forward Chains of Recurrent Attractor Neural Networks Near Saturation

We perform a stationary state replica analysis for a layered network of Ising spin neurons, with recurrent Hebbian interactions within each layer, in combination with strictly feed-forward Hebbian interactions between successive layers. This model interpolates between the fully recurrent and symmetric attractor network studied by Amit el al, and the strictly feed-forward attractor network studied by Domany et al. Due to the absence of detailed balance, it is as yet solvable only in the zero temperature limit. The built-in competition between two qualitatively different modes of operation, feed-forward (ergodic within layers) versus recurrent (non- ergodic within layers), is found to induce interesting phase transitions.

cond-mat↗

arXiv1996

Macrodynamics of Disordered and Frustrated Systems

It is shown how the macroscopic non-equilibrium dynamics of a class of systems whose microscopic stochastic dynamics involves disordered and frustrated but range-free interactions can be well described by closed deterministic flow equations; this requires an appropriate choice of order parameters/function and ansatze.

cond-mat↗

arXiv1995

Order-Parameter Flow in the SK Spin-Glass II: Inclusion of Microscopic Memory Effects

We develop further a recent dynamical replica theory to describe the dynamics of the Sherrington-Kirkpatrick spin-glass in terms of closed evolution equations for macroscopic order parameters. We show how microscopic memory effects can be included in the formalism through the introduction of a dynamic order parameter function: the joint spin-field distribution. The resulting formalism describes very accurately the relaxation phenomena observed in numerical simulations, including the typical overall slowing down of the flow that was missed by the previous simple two-parameter theory. The advanced dynamical replica theory is either exact or a very good approximation.

cond-mat↗

arXiv1995

Dynamical Replica Theory for Disordered Spin Systems

We present a new method to solve the dynamics of disordered spin systems on finite time-scales. It involves a closed driven diffusion equation for the joint spin-field distribution, with time-dependent coefficients described by a dynamical replica theory which, in the case of detailed balance, incorporates equilibrium replica theory as a stationary state. The theory is exact in various limits. We apply our theory to both the symmetric- and the non-symmetric Sherrington-Kirkpatrick spin-glass, and show that it describes the (numerical) experiments very well.

cond-mat↗

arXiv1994

Closure of Macroscopic Laws in Disordered Spin Systems: A Toy Model

We use a linear system of Langevin spins with disordered interactions as an exactly solvable toy model to investigate a procedure, recently proposed by Coolen and Sherrington, for closing the hierarchy of macroscopic order parameter equations in disordered spin systems. The closure procedure, based on the removal of microscopic memory effects, is shown to reproduce the correct equations for short times and in equilibrium. For intermediate time-scales the procedure does not lead to the exact equations, yet for homogeneous initial conditions succeeds at capturing the main characteristics of the flow in the order parameter plane. The procedure fails in terms of the long-term temporal dependence of the order parameters. For low energy inhomogeneous initial conditions and near criticality (where zero modes appear) deviations in temporal behaviour are most apparent. For homogeneous initial conditions the impact of microscopic memory effects on the evolution of macroscopic order parameters in disordered spin systems appears to be mainly an overall slowing down.

cond-mat↗