Searcharxiv⌕ Search

arXiv subjects

Song Ren

Publications and source records attributed to Song Ren.

2 recordsLinked to original sources

FuGIBench: Evaluating Reliable Preparation in Streaming Video

An assistant watching a worker can prepare for an emerging need before the worker is ready to receive help. As the need evolves, the assistant must maintain preparation until a response is due, or withdraw it if the need disappears. Video anticipation and interactive response benchmarks leave this interval between early evidence and response largely unmeasured. We introduce FuGIBench (Future-Grounded Interaction), a streaming-video benchmark in which a model chooses among waiting, reversible preparation, and response given a causal video prefix and a task contract. The benchmark contains 19,893 synthetic tool-assistance videos across ten tool tasks, in handover, ordinary-activity, and cancelled-need tracks. One primary score credits sustained preparation, incorporates response-interval alignment, and penalizes activity when the reference prescribes waiting. On a frozen set of 1,000 episodes, a reference system with a longer preparation lead has poorer response alignment, while similar handover leads accompany different levels of activity during ordinary work in matched scenarios. Together with cancellation analyses, these results show why evaluating preparation requires tracking decisions as needs develop, become actionable, or disappear.

cs.CV↗

EviBridge: Encoding Predictive Visual Evidence for Proactive Interaction

Tool-handover interaction calls for preparation before a response is warranted. The model waits until interaction cues appear, then prepares while assessing whether the accumulating evidence supports a response or calls for cancellation. We introduce EviBridge, a framework that integrates predictive visual representations into a vision-language interaction model for joint state and tool prediction. A trainable evidence interface maps features from a frozen V-JEPA 2 encoder into the language decoder alongside native visual inputs, preserving their temporal context and using only observations available at each decision time. The framework has two instantiations. EviBridge-E distinguishes recent from historical evidence and encodes the timing and spatial location of each observation. EviBridge-I preserves the temporal order of pooled V-JEPA 2 features and interaction turns and generates dedicated interaction tokens. We evaluate on FuGIBench, a streaming-video benchmark for preparation, response, and cancellation in tool-handover assistance. The best-performing variants of EviBridge-E and EviBridge-I achieve Overall scores of 0.6448 and 0.5892, compared with 0.2178 and 0.5327 for their respective task-adapted native baselines. These gains span target handovers, long waits, and cancelled requests, demonstrating the value of predictive visual evidence for interaction decisions.

cs.CV↗