WARP-RM: A Warp-Augmented Relative Progress Reward Model for Data Curation

Abstract

Scaling imitation learning requires large datasets, yet human teleoperation inevitably produces mixed-quality demonstrations containing hesitations, retries, and pauses. Prior frame-level progress reward models supervise on absolute temporal progress proxies that suffer from label noise, or require costly human annotations to define subtask boundaries. We present WARP (Warp-Augmented Relative Progress), a novel fully self-supervised algorithm for learning dense, signed relative progress magnitudes directly from successful demonstrations. WARP generates per-frame progress targets via time-warp augmentations of demonstrations (variable playback speeds and reversals) and we train WARP-RM to predict normalized signed temporal displacement from the start of each sampled window. Aggregating these predictions across overlapping windows yields a dense frame-level progress signal. We then introduce WARP-BC, which uses these scalar reward estimates to filter and reweight action chunks during behavior cloning. We evaluate our approach on a physical bimanual robot system performing a long-horizon deformable object manipulation task: folding T-shirts from a random crumpled start. To evaluate policy robustness against suboptimal data, we construct training datasets of varying quality using episode length as a proxy for teleoperation sub-optimality. Across these training tiers, WARP-BC improves successful-folding throughput by up to ~18x over vanilla BC. Furthermore, we evaluate bottle-in-bin placement in the real world and in simulation. Across 512 paired simulated scenes, WARP-BC achieves 290 bottles/hr versus 237 for vanilla BC and 271 for DemInf, with all curation methods retaining 31.5% of the data. We release open simulation data, code, checkpoints, and evaluation artifacts for end-to-end reproduction of the WARP pipeline. Project page: https://uynitsuj.github.io/warp-rm/