arXiv subjects

Jiayi Guan

Publications and source records attributed to Jiayi Guan.

10 recordsLinked to original sources

arXiv2026

SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models

Vision-Language Models (VLMs) perform well on commonsense reasoning tasks but struggle with visual spatial reasoning. Most existing solutions introduce extra 3D prior inputs or external spatial encoders, which increase complexity and degrade the underlying VLMs' general-purpose capabilities after spatial fine-tuning. To this end, we propose a parameter-efficient \textit{\textbf{Spatio}-vision \textbf{L}anguage \textbf{M}odels (SpatioLM)}, that enhances spatial intelligence without extra 3D prior inputs or third-party spatial encoders. Concretely, we design a plug-and-play and non-invasive spatio-vision module that elicits the spatial knowledge inherent in VLMs. Furthermore, we innovatively leverage pseudo depth and camera information as supervision to guide the model in learning physically coherent representations. Extensive experiments show that SpatioLM achieves significant improvements in diverse tasks, including spatial perception and understanding while effectively limiting the degradation of general capabilities. Notably, the model achieves an impressive score of 71.6 on the VSI-Bench (the first model to surpass 70). In addition, it attains competitive performance when transferred to embodied manipulation tasks. Code is available at \href{https://github.com/xiaomi-research/spatio-lm}{\faGithub~spatio-lm}.

cs.CV↗

arXiv2026

CDCP: Conditional Diffusion Model with Contextual Prompts for Multi-task Offline Safe Reinforcement Learning

Multi-task offline safe reinforcement learning (RL) promises to learn a shared optimal safe policy from offline data across multiple tasks. This paradigm provides an effective means for the widespread application of RL in multi-task scenarios with high risk and interaction costs. However, the triple challenges of multi-tasking, safety constraints, and out-of-distribution (OOD) actions pose a significant hurdle for existing methods to ensure safety while maximizing reward returns. In this work, we propose a Conditional Diffusion model with Contextual Prompts (CDCP) to address these challenges. Concretely, we first rethink the requirements and challenges in current multi-task decision-making and control scenarios and establish the objectives of multi-task offline safe RL. Subsequently, we transform the multi-task constrained optimization problem into a conditional generation problem using the diffusion model. Based on this, we design a classifier-free guided cost-constraint strategy to provide flexible cost constraints and eliminate extrapolation errors from OOD actions via supervised learning. Additionally, we introduce a novel contextual prompting method to enhance multi-task representation accuracy and adaptability to unseen tasks. A gradient loss synchronization strategy is also introduced to eliminate gradient interference, improving training stability. Finally, extensive experiments demonstrate that the CDCP algorithm exhibits higher performance and safety in multi-task scenarios than the current state-of-the-art baseline methods. It meets different cost constraints without further training, providing a more flexible cost-constraint solution for the multi-task safe RL.

cs.LG↗

arXiv2026

UniviewVLA: A Unified Multiview Vision-Language-Action Model with World Modeling

Occluded tasks remain a bottleneck in robot manipulation. Existing solutions either deploy additional physical cameras requiring training-inference camera parity, or rely on explicit 3D reconstruction with high computational cost. Moreover, both approaches rely on standard agent-view and wrist-view observations, while failing to capture occlusion information and future scene evolution. To this end, we propose UniviewVLA, a unified multiview Vision-Language-Action model with world modeling, which infers multiview scene evolution for action prediction from only standard two-camera observations. We demonstrate that by leveraging generated multiview future views from the world model, UniviewVLA reveals occluded cues and models future scene evolution, improving action prediction and removing the need for extra hardware or explicit reconstruction. Besides, to accelerate inference while preserving prediction accuracy, UniviewVLA develops Motion-Informative Token Compression, which compresses each generated view from 625 to 16 tokens and reduces per-view latency from 6-7s to 0.2-0.3s. UniviewVLA also proposes training-free Action-Entropy View Selection, which dynamically identifies the most action-informative view at different inference stages. Extensive experiments show that UniviewVLA achieves 95.8% on LIBERO and 4.60 on CALVIN ABCD to D, both standard occlusion-free benchmarks. On customized occlusion-focused tasks, it improves success rate from 40.0% to 73.3%, and average real-robot success rate by 33.4 points, demonstrating stronger occlusion-focused performance without sacrificing standard occlusion-free benchmarks.

cs.RO↗

arXiv2026

ATHENA: Accelerated Multi-Task Heterogeneous Influence Functions for Robot Data Curation

In robot imitation learning, influence functions provide a principled approach to quantify each demonstration's effect on robot task outcomes, yet scaling them to billion-parameter Vision-Language-Action (VLA) models is limited by computational and multitask bottlenecks. To this end, we propose ATHENA, an influence function framework tailored for multitask VLA data curation at a billion-parameter scale. Concretely, it leverages the Kronecker structure of linear-layer gradients to reduce projection cost, and approximates dense Hessian inversion with a rank-r Random Truncated Approximation, achieving about a 313.4x speedup in influence computation. Furthermore, ATHENA formulates global and local interactive influence to balance data curation across 50 jointly trained tasks. Extensive evaluations on RoboTwin 2.0 and real-robot deployment, covering 9.34 and 6.90 hours of demonstrations, respectively, show that ATHENA matches or exceeds full-data joint fine-tuning using only 50% of demonstrations in simulation and 66.7% of data across six real-robot tasks. Overall, ATHENA demonstrates its effectiveness for data curation in billion-parameter multitask VLA fine-tuning.

cs.RO↗

arXiv2026

Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation

Chain-of-Thought (CoT) reasoning has become a powerful driver of trajectory prediction in VLA-based autonomous driving, yet its autoregressive nature imposes a latency cost that is prohibitive for real-time deployment. Latent CoT methods attempt to close this gap by compressing reasoning into continuous hidden states, but consistently fall short of their explicit counterparts. We suggest that this is due to purely linguistic latent representations compressing a symbolic abstraction of the world, rather than the causal dynamics that actually govern driving. Thus, we present OneVL (One-step latent reasoning and planning with Vision-Language explanations), a unified VLA and World Model framework that routes reasoning through compact latent tokens supervised by dual auxiliary decoders. Alongside a language decoder that reconstructs text CoT, we introduce a visual world model decoder that predicts future-frame tokens, forcing the latent space to internalize the causal dynamics of road geometry, agent motion, and environmental change. A three-stage training pipeline progressively aligns these latents with trajectory, language, and visual objectives, ensuring stable joint optimization. In inference, the auxiliary decoders are discarded, and all latent tokens are prefilled in a single parallel pass, matching the speed of answer-only prediction. Across four benchmarks, OneVL becomes the first latent CoT method to surpass explicit CoT, delivering superior accuracy at answer-only latency. These results show that with world model supervision, latent CoT produces more generalizable representations than verbose token-by-token reasoning. Code has been open-sourced to the community. Project Page: https://xiaomi-embodied-intelligence.github.io/OneVL

cs.CV↗

arXiv2026

MiMo-Embodied: X-Embodied Foundation Model Technical Report

We open-source MiMo-Embodied, the first cross-embodied foundation model to successfully integrate and achieve state-of-the-art performance in both Autonomous Driving and Embodied AI. MiMo-Embodied sets new records across 17 embodied AI benchmarks in Task Planning, Affordance Prediction and Spatial Understanding, while also excelling in 12 autonomous driving benchmarks across Environmental Perception, Status Prediction, and Driving Planning. Across these tasks, MiMo-Embodied significantly outperforms existing open-source, closed-source, and specialized baselines. Our results indicate that through multi-stage learning, curated data construction, and CoT/RL fine-tuning, these two domains exhibit strong positive transfer and mutually reinforce one another. We provide a detailed analysis of our model design and training methodologies to facilitate further research. Code and models are available at https://github.com/XiaomiMiMo/MiMo-Embodied.

cs.RO↗

arXiv2025

Orbital Signatures of Density Wave Transition in La3Ni2O7-delta and La2PrNi2O7-delta RP-Nickelates Probed via in-situ X-ray Absorption Near-edge Spectroscopy

The report of superconductivity (SC) with Tc~80 K in bilayer Ruddlesden-Popper (RP) nickelate La3Ni2O7-delta have sparked considerable investigations on its normal state properties and SC mechanism under pressure and at low temperature. It is believed that the density wave (DW) at ~150 K plays an important role in SC emergence, but its nature remains largely underexplored. Here, we utilized temperature-dependent in-situ Ni K-edge X-ray Absorption Near-edge Spectroscopy (XANES) to probe the Ni-3d/4p electronic states of La3Ni2O7-delta and La2PrNi2O7-delta samples down to 4.8 K, enabling us to witness the evolution of both in-plane d_(x^2-y^2)/p_x (p_y) and out-of-plane d_(3z^2-r^2)/p_z orbitals of NiO6 octahedron across the DW transition. Main edge energy associated with Ni 4p orbital shows an anomalous decline near DW transition, signifying the occurrence of lattice distortions as a hallmark of charge density wave. Below DW transition, the enlarged crystal field splitting (CFS) indicates an enhanced NiO6 octahedral distortion. Intriguingly, magnetic Pr substituents could activate the mutual interplay of d_(x^2-y^2) and d_(3z^2-r^2) orbitals. We discussed its relevance to the favored bulk SC in the pressurized polycrystalline La2PrNi2O7-delta than pristine.

cond-mat.supr-con↗

arXiv2024

Distinguishing Electronic Band Structure of Single-layer and Bilayer Ruddlesden-Popper Nickelates Probed by in-situ High Pressure X-ray Absorption Near-edge Spectroscopy

We report a comprehensive study of electronic band structure for single-layer (SL) and bilayer (BL) RP-nickelates probed by in-situ HP X-ray absorption near edge spectroscopy (XANES). At ambient pressure (AP), the energy splitting delta_E of d_3z^2-r^2 and d_x^2-y^2 bands are directly observed in La3Ni2O7 (BL-La327) but not in La2NiO4 (SL-La214) above E_F, underlining the critical role of inner apical O atoms. A combination of DFT-based electronic band structure and projected density of states (PDOS) calculations with simulated XANES enables us to explain the observed main XANES features labelled by a, A, B', B and C when considering the orbital hybridizations, crystal field splitting (CFS) and core-hole screening of different 3d configurations for SL-La214 and BL-La327 nickelates. At high pressure (HP), the delta_E values of pre-edge peak form a dome-like evolution above 7.7 GPa with the maximum locating at around 20 GPa for metallic BL-La327. Analysis of its integrated area and FWHM provides strong evidence that the bonding d_3z^2-r^2 band crosses E_F above about 7.7 GPa for the metallic BL-La327. Growth of integrated area of pre-edge peak and C peak further evidences pressure-induced hole doping effect. Meanwhile, the pressure dependent FWHM of pre-edge peak implies a nonmonotonic evolution of orbital-selective electronic correlation above 7.7 GPa with extrema emerging at about 20 GPa. Moreover, we estimate the relative hole doping level using the energy shift of pre-edge peak, yielding 0.074 hole per Ni site or equivalently 1.1*10^21 cm^-3 at 20 GPa for the metallic BL-La327, which is comparable to cuprates. Our results have timely examined the electronic band structures as obtained from theoretical calculations, emphasizing the essential role of both d_3z^2-r^2 and d_x^2-y^2 bands as well as the electronic correlation in superconducting pairing for pressurized La3Ni2O7.

cond-mat.supr-con↗

arXiv2024

Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey

Reinforcement Learning (RL) is a potent tool for sequential decision-making and has achieved performance surpassing human capabilities across many challenging real-world tasks. As the extension of RL in the multi-agent system domain, multi-agent RL (MARL) not only need to learn the control policy but also requires consideration regarding interactions with all other agents in the environment, mutual influences among different system components, and the distribution of computational resources. This augments the complexity of algorithmic design and poses higher requirements on computational resources. Simultaneously, simulators are crucial to obtain realistic data, which is the fundamentals of RL. In this paper, we first propose a series of metrics of simulators and summarize the features of existing benchmarks. Second, to ease comprehension, we recall the foundational knowledge and then synthesize the recently advanced studies of MARL-related autonomous driving and intelligent transportation systems. Specifically, we examine their environmental modeling, state representation, perception units, and algorithm design. Conclusively, we discuss open challenges as well as prospects and opportunities. We hope this paper can help the researchers integrate MARL technologies and trigger more insightful ideas toward the intelligent and autonomous driving.

cs.AI↗

arXiv2024

Preferred-Action-Optimized Diffusion Policies for Offline Reinforcement Learning

Offline reinforcement learning (RL) aims to learn optimal policies from previously collected datasets. Recently, due to their powerful representational capabilities, diffusion models have shown significant potential as policy models for offline RL issues. However, previous offline RL algorithms based on diffusion policies generally adopt weighted regression to improve the policy. This approach optimizes the policy only using the collected actions and is sensitive to Q-values, which limits the potential for further performance enhancement. To this end, we propose a novel preferred-action-optimized diffusion policy for offline RL. In particular, an expressive conditional diffusion model is utilized to represent the diverse distribution of a behavior policy. Meanwhile, based on the diffusion model, preferred actions within the same behavior distribution are automatically generated through the critic function. Moreover, an anti-noise preference optimization is designed to achieve policy improvement by using the preferred actions, which can adapt to noise-preferred actions for stable training. Extensive experiments demonstrate that the proposed method provides competitive or superior performance compared to previous state-of-the-art offline RL methods, particularly in sparse reward tasks such as Kitchen and AntMaze. Additionally, we empirically prove the effectiveness of anti-noise preference optimization.

cs.LG↗