Self-shaping dynamical systems and learning

Abstract

We associate learning and adaptation in living systems with the shaping of the velocity vector field in the respective dynamical systems in response to external, generally random, stimuli. With this, a mathematical concept of self-shaping dynamical systems is proposed. Initially there is a zero vector field and an "empty" phase space with no attractors or other non-trivial objects. As the random stimulus begins, the vector field deforms and eventually becomes smooth and deterministic, despite the random nature of the applied force, while the phase space develops various geometrical objects. We consider gradient self-shaping systems, whose vector field is the gradient of some energy function, which under certain conditions develops into the multi-dimensional probability density distribution (PDD) of the input. Self-shaping systems are relevant to neural networks (NNs) of two types: Hopfield, and probabilistic. Firstly, we show that they can potentially perform pattern recognition tasks traditionally delegated to Hopfield NNs, but without supervision and on-line, and without developing spurious minima of the energy. Secondly, like probabilistic NNs, they can reconstruct the PDD of input signals, without the limitation that new training patterns have to enter as new hardware units. Thus, self-shaping systems can be regarded as a generalization of the NN concept, achieved by abandoning the "rigid units" - "flexible couplings" paradigm and making the vector field fully flexible and amenable to external force. The new concept presents an engineering challenge requiring new principles of hardware design. It might also become an alternative paradigm for modeling of living and learning systems.