A Noise Optimum in Rehearsal-Free Continual Learning: Isolation, Mechanism, and Scope

Abstract

Injecting stochastic noise into a consolidation rule can improve a network's retention of earlier tasks up to an optimal level, then degrade it -- an inverted-U in retention vs. noise. This paper isolates what produces that optimum and maps where it holds, entirely in simulation. (1) Phenomenon: the retention inverted-U appears on several related-task continual-learning benchmarks (Split-MNIST, FashionMNIST, continual Yin-Yang). (2) Isolation: a magnitude-matched ladder shows the effect requires coherent restoring toward the consolidated weights -- a random-direction force of identical magnitude produces no optimum, and a coherent force toward the wrong target actively hurts. (3) Active ingredient: most of the optimum is recovered by coupling the anchor gain to the injected-noise variance sigma^2 -- a one-line rule that neither Ornstein-Uhlenbeck Adaptation (fixed gain) nor MESU (posterior-variance gain) implements. A forced Ornstein-Uhlenbeck calculation derives the rising flank and predicts that the optimal noise rises with per-task interference g -- confirmed out-of-sample in direction against pre-existing measurements (the exponent is unresolved at our grid). The barrier-conditioning of the originating Doob h-transform is a low-sigma safety net that bounds forgetting where the coupled gain is too weak. (4) Scope: the optimum requires shared task structure -- it is absent on permuted-MNIST, and a controlled rotated-vs-permuted comparison localizes the boundary to task structure; the precise governing quantity is left open. (5) Length: at matched severity the advantage persists but attenuates with task count, and we show no rotation family can attribute the trend (a compact-group identity). A single-seed BrainScaleS-2 demonstration of the originating rule is reported separately (Howe, arXiv:2607.06924); this paper makes no hardware claim.