arXiv · 2610.10857
Self-Supervised Keyframe Discovery for Horizon-Invariant Behavior Cloning
Abstract
Behavior cloning (BC) in non-Markovian environments is a challenging problem because policies have to reason over contextual information over long horizons. Existing policy architectures rely on recurrent or attention-based mechanisms to capture long-term dependencies. However, recurrent models suffer from hidden-state collapse and gradient instability under backpropagation through time while attention-based models are fundamentally limited by context length. To address these issues, we propose Keyframe Mnemonics, a novel self-supervised method that $\textit{discovers}$ a set of information-critical observations ($\textit{mnemonics}$) by learning an objective from randomly sampled past observations and using it as a reward for keyframe selection. We then train a BC policy that conditions on the discovered keyframes to model the action distribution. Under the simplicity bias assumption, our formulation provides context retention guarantees over an infinite horizon, while maintaining a small set of decision-relevant keyframes in the policy's working memory. We evaluate our method on synthetic memory domains, where mnemonic-conditioned BC policies achieve $100$% success rates (SR) and generalize to horizons orders of magnitude beyond training without performance degradation. Moreover, we evaluate on a memory-intensive robot manipulation benchmark, achieving a $13.9$% average absolute SR improvement over the strongest baseline across $23$ tasks. Finally, we demonstrate generalization with $80$% SR at $20\times$ longer time horizons on a real robot. Code and videos available at https://keyframe-mnemonics.github.io.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Prabin Kumar Rath, Omkar Patil, Nakul Gopalan. 2026-11-07. Self-Supervised Keyframe Discovery for Horizon-Invariant Behavior Cloning. https://searcharxiv.com/abs/2610.10857
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.