Xueyao Wan | Searcharxiv

arXiv2026

EgoAERO: Learning Dexterous Manipulation from a Single Egocentric Video without Object Assets

Egocentric RGB-D videos offer a natural source of human dexterous manipulation demonstrations, but existing data is difficult to use for robot learning because object pose, geometry, and contact information are often missing or require pre-scanned object assets. We present EgoAERO, the first framework that learns dexterous manipulation from a single egocentric RGB-D human demonstration without object assets. EgoAERO reconstructs contact-consistent hand-object trajectories through asset-free object tracking and reconstruction, ego motion compensation, and adaptive contact optimization, then converts them into robot policies using two-stage residual learning. We further introduce an online quality assessment mechanism and construct EgoDex-R, a large-scale egocentric dataset with 4.3M RGB-D frames for dexterous policy learning. Simulation and real-world experiments show that EgoAERO enables single-demonstration dexterous manipulation and achieves downstream performance close to CAD-based reconstructions on HOI4D.

cs.RO↗

arXiv2025

MMGraphRAG: Bridging Vision and Language with Interpretable Multimodal Knowledge Graphs

Large Language Models (LLMs) suffer from hallucinations due to their static parametric knowledge. Retrieval-Augmented Generation (RAG) and GraphRAG mitigate this issue by incorporating external knowledge and structured reasoning over knowledge graphs (KGs). However, existing approaches remain largely text-centric, as constructing fine-grained multimodal knowledge graphs (MMKGs) with explicit cross-modal semantics remains challenging. In this paper, we propose MMGraphRAG, a framework for building interpretable MMKGs that unify textual and visual knowledge. Our approach represents visual content as structured scene graphs and integrates them with textual KGs through a novel cross-modal entity linking method, SpecLink, which leverages spectral clustering to jointly model semantic similarity and graph structure. This design preserves explicit entities, relations, and reasoning paths across modalities, enabling structure-aware retrieval and generation. To support evaluation, we introduce the CMEL dataset, a benchmark for fine-grained cross-modal entity alignment. Experimental results on CMEL demonstrate improved entity linking accuracy, while evaluations on DocBench and MMLongBench show that MMGraphRAG achieves superior performance and stronger robustness, particularly in complex multimodal reasoning scenarios.

cs.AI↗