WALDO: One-Shot Exemplar-Conditioned Object Detection in Cluttered Scenes with World-Model Features
Finding a particular object instance in a cluttered scene from a single reference image and saying when it is not there is typically given to large vision-language models. We ask whether the representations of a world model add something that the strongest self-supervised image features do not have. We introduce WALDO, a one-shot detector that finds a specific object instance in a cluttered real scene from a single reference image. Instance detection has usually needed per-object training. The alternative is a category-level open-vocabulary detector or vision-language model, which can find a mug but not this mug. We show that a lightweight conditioned head on frozen self-supervised features is enough to localize unseen instances without any per-object adaptation. We also find that image features and world-model features are complementary: DINOv3 and the video world model V-JEPA 2.1 succeed on different queries. There is one catch. Used the way video models are usually applied to still images, V-JEPA's dense features underperform. Reading them through the model's image pathway improves their localization. Combining the two backbones with a validation-tuned late fusion modestly improves DINOv3's localization of unseen instances without hurting its ability to tell whether the object is in the scene. A strong vision-language model and an image-guided open-vocabulary detector almost never localize the specific instance, and take far longer per query. Our fusion with world-model features strengthens localization from image features alone.