arXiv subjects

Xichen Tan

Publications and source records attributed to Xichen Tan.

3 recordsLinked to original sources

arXiv2026

Video-HOCA: A Diagnostic Benchmark for Physical Anomaly Reasoning in Video-LLMs

We introduce Video-HOCA, a diagnostic benchmark for physical anomaly reasoning in videos. Video-HOCA uses an Ontological-Causal taxonomy to distinguish violations of an entity's own properties or capabilities from violations of physical relations among entities and the environment. It contains more than 1,400 generated and real-world videos and 3,470 question-answer pairs, with human verification of labels and reference answers. The benchmark evaluates four levels of reasoning: plausibility checking, anomaly attribution, fine-grained recognition, and open-ended physical explanation. Across 20 Instruct-mode Video-LLMs, we find that recognition outpaces explanation: Task I scores cluster at 75-88, while Task II macro-F1 stays mostly below 50. We also find that the Ontological-Causal gap depends on the task and model configuration, and that Thinking-mode gains are not explained by sampling or output budget alone. Annotation agreement, Task-IV human-judge and judge-judge checks, alternative metrics, and temporal/decoding controls validate the evaluation pipeline and bound the claims supported by the benchmark.

cs.CV↗

arXiv2025

RAG-Adapter: A Plug-and-Play RAG-enhanced Framework for Long Video Understanding

Multi-modal Large Language Models (MLLMs) capable of video understanding are advancing rapidly. To effectively assess their video comprehension capabilities, long video understanding benchmarks, such as Video-MME and MLVU, are proposed. However, these benchmarks directly use uniform frame sampling for testing, which results in significant information loss and affects the accuracy of the evaluations in reflecting the true abilities of MLLMs. To address this, we propose RAG-Adapter, a plug-and-play framework that reduces information loss during testing by sampling frames most relevant to the given question. Additionally, we introduce a Grouped-supervised Contrastive Learning (GCL) method to further enhance sampling effectiveness of RAG-Adapter through fine-tuning on our constructed MMAT dataset. Finally, we test numerous baseline MLLMs on various video understanding benchmarks, finding that RAG-Adapter sampling consistently outperforms uniform sampling (e.g., Accuracy of GPT-4o increases by 9.3 percent on Video-MME), providing a more accurate testing method for long video benchmarks.

cs.CV↗

arXiv2025

ALLVB: All-in-One Long Video Understanding Benchmark

From image to video understanding, the capabilities of Multi-modal LLMs (MLLMs) are increasingly powerful. However, most existing video understanding benchmarks are relatively short, which makes them inadequate for effectively evaluating the long-sequence modeling capabilities of MLLMs. This highlights the urgent need for a comprehensive and integrated long video understanding benchmark to assess the ability of MLLMs thoroughly. To this end, we propose ALLVB (ALL-in-One Long Video Understanding Benchmark). ALLVB's main contributions include: 1) It integrates 9 major video understanding tasks. These tasks are converted into video QA formats, allowing a single benchmark to evaluate 9 different video understanding capabilities of MLLMs, highlighting the versatility, comprehensiveness, and challenging nature of ALLVB. 2) A fully automated annotation pipeline using GPT-4o is designed, requiring only human quality control, which facilitates the maintenance and expansion of the benchmark. 3) It contains 1,376 videos across 16 categories, averaging nearly 2 hours each, with a total of 252k QAs. To the best of our knowledge, it is the largest long video understanding benchmark in terms of the number of videos, average duration, and number of QAs. We have tested various mainstream MLLMs on ALLVB, and the results indicate that even the most advanced commercial models have significant room for improvement. This reflects the benchmark's challenging nature and demonstrates the substantial potential for development in long video understanding.

cs.CV↗