Learning continually with representational drift

Abstract

Deep artificial neural networks famously struggle to learn from non-stationary streams of data. Without dedicated mitigation strategies, continual learning is associated with continuous forgetting of previous tasks and a progressive loss of plasticity. Current approaches to continual learning have either focused on increasing the stability of representations of past tasks, or on promoting plasticity for future learning. Paradoxically, while animals including humans achieve a desirable stability-plasticity trade-off, the responses of biological neurons to external stimuli that are associated with stable behaviors gradually change over time. This suggests that, although unstable representations have historically been seen as undesirable in artificial systems, they could be a core property of biological neural networks learning continually. Here, we examine how linking representational drift to continual learning in biological neural networks could inform artificial systems. We highlight the existence of representational drift across numerous animal species and brain regions and propose that drift reflects a mixture of homeostatic turnover and learning-related synaptic plasticity. In particular, we evaluate how plasticity induced by learning new tasks could induce drift in the representation of previous tasks, and how such drift could accumulate across brain regions. In deep artificial neural networks, we propose that representational drift is only compatible with approaches that do not explicitly prevent parameter changes to mitigate forgetting. Remarkably, jointly promoting plasticity while mitigating forgetting could in principle induce representational drift in continual learning. While we argue that drift is a byproduct rather than a solution to incremental learning, its investigation could inform approaches to continual learning in artificial systems.