arXiv subjects

K. Y. Michael Wong

Publications and source records attributed to K. Y. Michael Wong.

46 records · Page 3Linked to original sources

arXiv2005

Effects of Diversity on Multi-agent Systems: Minority Games

We consider a version of large population games whose agents compete for resources using strategies with adaptable preferences. The games can be used to model economic markets, ecosystems or distributed control. Diversity of initial preferences of strategies is introduced by randomly assigning biases to the strategies of different agents. We find that diversity among the agents reduces their maladaptive behavior. We find interesting scaling relations with diversity for the variance and other parameters such as the convergence time, the fraction of fickle agents, and the variance of wealth, illustrating their dynamical origin. When diversity increases, the scaling dynamics is modified by kinetic sampling and waiting effects. Analyses yield excellent agreement with simulations.

cond-mat.stat-mech↗

arXiv2003

Dynamical Mechanisms in Multi-agent Systems: Minority Games

We consider a version of large population games whose agents compete for resources using strategies with adaptable preferences. Diversity among the agents reduces their maladpative behavior. We find interesting scaling relations with diversity for the variance of decisions. When diversity increases, the scaling dynamics is modified by kinetic sampling and waiting mechanisms.

cond-mat↗

arXiv2003

Diversity and Adaptation in Large Population Games

We consider a version of large population games whose players compete for resources using strategies with adaptable preferences. The system efficiency is measured by the variance of the decisions. In the regime where the system can be plagued by the maladaptive behavior of the players, we find that diversity among the players improves the system efficiency, though it slows the convergence to the steady state. Diversity causes a mild spread of resources at the transient state, but reduces the uneven distribution of resources in the steady state.

cond-mat↗

arXiv2002

Dynamical and Stationary Properties of On-line Learning from Finite Training Sets

The dynamical and stationary properties of on-line learning from finite training sets are analysed using the cavity method. For large input dimensions, we derive equations for the macroscopic parameters, namely, the student-teacher correlation, the student-student autocorrelation and the learning force uctuation. This enables us to provide analytical solutions to Adaline learning as a benchmark. Theoretical predictions of training errors in transient and stationary states are obtained by a Monte Carlo sampling procedure. Generalization and training errors are found to agree with simulations. The physical origin of the critical learning rate is presented. Comparison with batch learning is discussed throughout the paper.

cond-mat.dis-nn↗

arXiv2001

The Cavity Approach to Noisy Learning in Nonlinear Perceptrons

We analyze the learning of noisy teacher-generated examples by nonlinear and differentiable student perceptrons using the cavity method. The generic activation of an example is a function of the cavity activation of the example, which is its activation in the perceptron that learns without the example. Mean field equations for the macroscopic parameters and the stability condition yield results consistent with the replica method. When a single value of the cavity activation maps to multiple values of the generic activation, there is a competition in learning strategy between preferentially learning an example and sacrificing it in favor of the background adjustment. We find parameter regimes in which examples are learned preferentially or sacrificially, leading to a gap in the activation distribution. Full phase diagrams of this complex system are presented, and the theory predicts the existence of a phase transition from poor to good generalization states in the system. Simulation results confirm the theoretical predictions.

cond-mat.dis-nn↗

arXiv2000

Mean-field theory of learning: from dynamics to statics

Using the cavity method and diagrammatic methods, we model the dynamics of batch learning of restricted sets of examples. Simulations of the Green's function and the cavity activation distributions support the theory well. The learning dynamics approaches a steady state in agreement with the static version of the cavity method. The picture of the rough energy landscape is reviewed.

cond-mat.dis-nn↗

arXiv2000

Sacrificial Learning in Nonlinear Perceptrons

Using the cavity method we consider the learning of noisy teacher-generated examples by a nonlinear student perceptron. For insufficient examples and weak weight decay, the activation distribution of the training examples exhibits a gap for the more difficult examples. This illustrates that the outliers are sacrificed for the overall performance. Simulation shows that the picture of the smooth energy landscape cannot describe the gapped distributions well, implying that a rough energy landscape may complicate the learning process.

cond-mat.dis-nn↗

arXiv2000

Error-correcting codes and image restoration with multiple stages of dynamics

We consider the problems of error-correcting codes and image restoration with multiple stages of dynamics. Information extracted from the former stage can be used selectively to improve the performance of the latter one. Analytic results were derived for the mean-field systems using the cavity method. We find that it has the advantage of being tolerant to uncertainties in hyperparameter estimation, as confirmed by simulations.

cond-mat.dis-nn↗

arXiv1999

Many-body approach to the dynamics of batch learning

Using the cavity method and diagrammatic methods, we model the dynamics of batch learning of restricted sets of examples, widely applicable to general learning cost functions, and fully taking into account the temporal correlations introduced by the recycling of the examples.

cond-mat.dis-nn↗

arXiv1998

Phase Transition in a Noise Reduction Model: Shrinking or Percolation?

A model of noise reduction (NR) for signal processing is introduced. Each noise source puts a symmetric constraint on the space of the signal vector within a tolerable overlap. When the number of noise sources increases, sequences of transitions take place, causing the solution space to vanish. We found that the transition from an extended solution space to a shrunk space is retarded because of the symmetry of the constraints, in contrast to the analogous problem of pattern storage. For low tolerance, the solution space vanishes by volume reduction, whereas for high tolerance, the vanishing becomes more and more like percolation. The model is studied in the replica symmetric, first step and full replica symmetry breaking schemes.

cond-mat.dis-nn↗