Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · adap-org/9604001

Learning from Minimum Entropy Queries in a Large Committee Machine

Peter Sollich
View original source ↗Read source document ↗Export metadata ↓

Abstract

In supervised learning, the redundancy contained in random examples can be avoided by learning from queries. Using statistical mechanics, we study learning from minimum entropy queries in a large tree-committee machine. The generalization error decreases exponentially with the number of training examples, providing a significant improvement over the algebraic decay for random examples. The connection between entropy and generalization error in multi-layer networks is discussed, and a computationally cheap algorithm for constructing queries is suggested and analysed.

Explore related subjects

adap-orgcond-matnlin.AO

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Peter Sollich. 1996-04-11. Learning from Minimum Entropy Queries in a Large Committee Machine. https://doi.org/10.1103/physreve.53.r2060

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2012

Stable Adaptive Control Using New Critic Designs

Classical adaptive control proves total-system stability for control of linear plants, but only for plants meeting very restrictive assumptions. Approximate Dynamic Programming (ADP) has the potential, in principle, to ensure stability without such tight restrictions. It also offers nonlinear and neural extensions for optimal control, with empirically supported links to what is seen in the brain. However, the relevant ADP methods in use today -- TD, HDP, DHP, GDHP -- and the Galerkin-based versions of these all have serious limitations when used here as parallel distributed real-time learning systems; either they do not possess quadratic unconditional stability (to be defined) or they lead to incorrect results in the stochastic case. (ADAC or Q-learning designs do not help.) After explaining these conclusions, this paper describes new ADP designs which overcome these limitations. It also addresses the Generalized Moving Target problem, a common family of static optimization problems, and describes a way to stabilize large-scale economic equilibrium models, such as the old long-term energy model of DOE.

adap-org↗
arXiv2011

Flame front propagation II: Random Noise and Pole-Dynamics in Unstable Front Propagation

The problem of flame propagation is studied as an example of unstable fronts that wrinkle on many scales is studied. The analytic tool of pole expansion in the complex plane is emloyed to address the interaction of the unstable growth process with random initial conditions and perturbations. We argue that the effect of random noise is immense and that it can never be neglected in sufficiently large systems. We present simulations that lead to scaling laws for the velocity and acceleration of the front as a function of the system size and the level of noise, and analytic arguments that explain these results in terms of the noisy pole dynamics.

adap-org↗
arXiv2002

Amplifying Phenomenal Information: Toward a Fundamental Theory of Consciousness

Fundamental approaches bypass the problem of getting consciousness from non-conscious components by positing that consciousness is a universal primitive. For example, the double aspect theory of information holds that information has a phenomenal aspect. How then do you get from phenomenal information to human consciousness? This paper proposes that an entity is conscious to the extent it amplifies information, first by trapping and integrating it through closure, and second by maintaining dynamics at the edge of chaos through simultaneous processes of divergence and convergence. The origin of life through autocatalytic closure, and the origin of an interconnected worldview through conceptual closure, induced phase transitions in the degree to which information, and thus consciousness, is locally amplified. Divergence and convergence of cognitive information may involve phenomena observed in light e.g. focusing, interference, and resonance. By making information flow inward-biased, closure shields us from external consciousness; thus the paucity of consciousness may be an illusion.

adap-org↗