Learning to crawl: Benefits and limits of centralized vs distributed control

Abstract

We present a model of a crawler consisting of several suction units distributed along a straight line and connected by springs. The suction units are rudimentary proprioceptors-actuators, which sense binary states of compression vs elongation of the springs, and can either adhere or remain idle. Muscular contraction is not controlled by the crawler, but follows an endogenous, stereotyped wave. The crawler is tasked to learn patterns of adhesion that generate thrust in response to the wave of contraction. Using tabular Q-learning we demonstrate that crawling can be learned by trial and error and we ask what are the benefits and limitations of distributed vs centralized learning architectures. We find that by centralizing proprioceptive feedback and control, the crawler leverages long range correlations in the dynamics and ride the endogenous wave smoothly. The ensuing benefits are measured in terms of both speed and robustness to failure, although they come at increased computational cost. At the opposite extreme, purely distributed feedback and control only leverages local information and yield a jerkier and slower crawling, although computationally cheap. Intermediate levels of centralization can negotiate fast and robust crawling while avoiding excessive computational burden, demonstrating the computational benefits of a hierarchical organization of crawling. Our model unveils the trade-offs between crawling speed, robustness to failure, computational cost and information exchange that may shape biological solutions for crawling and could inspire the design of robotic crawlers.