arXiv subjects

Siyuan Qian

Publications and source records attributed to Siyuan Qian.

13 recordsLinked to original sources

arXiv2026

Do Robotic World Models Really Follow Actions? Diagnosing and Aligning Action-Conditioned Generation for Policy Learning

Action-conditioned world models are increasingly used as learned simulators for policy evaluation and improvement, yet their effectiveness rests on an unverified assumption: generated futures faithfully reflect arbitrary valid actions. Existing benchmarks are typically confined to expert demonstrations, leaving off-expert action following inadequately evaluated. To address this gap, we introduce WorldEcho, which probes action following over a broader action distribution using visual integrity and SE(3) trajectory alignment. Our diagnosis shows that current world models reasonably execute expert actions but struggle with diverse off-expert trajectories, either ignoring the commanded actions or producing visually invalid rollouts. We further propose WorldSync, which strengthens action following along three complementary axes: distributional coverage, representational grounding, and intervention-effect alignment. It broadens the training distribution over action consequences, grounds intermediate video representations in action-induced robot dynamics through an Action-Forcing Expert, and aligns predicted changes under action interventions with the corresponding changes in ground-truth futures. Experiments on RoboTwin benchmarks and real-robot tasks show that WorldSync improves WorldEcho metrics and serves as a more reliable simulator for iterative policy improvement, enabling policies to achieve higher success rates.

cs.RO↗

arXiv2026

Group-of-Latents: Perceptual Video Compression at Extreme Bitrates via Masked Latent Generative Modeling

Most existing video compression algorithms follow a paradigm of transformation and quantization, optimizing the trade-off between distortion and bitrate. However, extremely low-bitrate compression remains an underexplored frontier where perceptual quality optimization under severely constrained coding resources has not been adequately addressed. In this paper, we propose a unified generative framework that leverages pre-trained Diffusion Transformer (DiT) priors to achieve high perceptual quality at extremely low bitrates. We first introduce a flexible Group-of-Latents (GoL) strategy within the latent space of a causal tokenizer, explicitly partitioning the latent stream into intra $I$-latents and inter $P$-latents. The Deep Compression Module (I-DCM) then encodes key $I$-latents to preserve perceptual anchors with minimal overhead. Building upon these anchors, the DiT-based Unified Latent Denoising Module (U-LDM) refines intra-frame textures and synthesizes $P$-latents from noise, reconstructing temporal dynamics at zero additional bitrate cost. Extensive experiments demonstrate that our method uniquely operates in the extreme-low-bitrate regime (e.g., (<0.005) bpp), achieving state-of-the-art perceptual fidelity with rich spatial details and robust temporal consistency. The code will be made publicly available.

eess.IV↗

arXiv2026

Hierarchical Denoising For Multi-Step Visual Reasoning

Video models are evolving into vision foundation models, yet they still lack human-like multi-step reasoning. Streaming autoregressive diffusion models are efficient but limited in reasoning, while bidirectional diffusion enables global revision with high inference costs due to dense frame-level denoising. Both paradigms struggle to achieve logical consistency and low-latency streaming for complex reasoning tasks. We propose HDR (Hierarchical Denoising for Visual Reasoning), a unified framework that integrates hierarchical latents into causal video generation for multi-step reasoning. HDR organizes video latents into a tree-structured hierarchy, enabling coarse-to-fine reasoning before streaming output. Coarse denoising layers preserve uncertain hypotheses for global planning, while finer layers progressively refine them into concrete visual states. A sparse hierarchical attention pattern (SHAP) further reduces temporal attention costs. We introduce a level-stratified multi-step video reasoning benchmark with out-of-distribution cases, covering six tasks: maze navigation, Tower of Hanoi, one-line drawing, sliding puzzle, Sokoban, and water pouring. Compared with streaming autoregressive diffusion baselines, HDR improves success from 34.22 to 60.29 (76.2% relative gain) and increases average progress from 76.00 to 89.56, demonstrating more consistent reasoning trajectories. HDR maintains low-latency streaming at 0.70 seconds per latent, achieving 54.2 times faster inference than bidirectional diffusion. It also retains 82.9% of full-data performance with only 2% training data, compared with 52.0% for bidirectional diffusion. Real-world robot experiments further demonstrate HDR's potential for physical interaction and world modeling. Project demo: https://hierarchical-diffusion-reasoning.github.io/.

cs.CV↗

arXiv2026

LaST-HD: Learning Latent Physical Reasoning from Scalable Human Data for Robot Manipulation

Human-hand demonstrations provide a direct and scalable source of physical interaction data for robot learning. While manual retargeting is indispensable for establishing kinematic action correspondence across different morphologies, robust transfer requires going beyond geometry to address the underlying alignment of physical dynamics between human and robot manipulation. To address this, we introduce LaST-HD, a novel human-to-robot action learning paradigm that extends reasoning-before-acting VLA by aligning human-hand and robot demonstrations in a shared latent reasoning space. Rather than mimicking human kinematics, LaST-HD trains an auxiliary action-conditioned world model on unpaired human-hand and robot trajectories to synthesize unified latent targets. After aligning cross-embodiment representations in this shared forward-dynamics space, these targets supervise LaST-HD's latent reasoning process, enabling it to internalize shared physical dynamics and drive efficient human-hand action learning. Moreover, we develop Out-of-Lab (OOL) Glove, a low-cost motion-capture glove tailored to LaST-HD for human-hand data collection. The captured human data provide precise keypoints and serve as universal action supervision across grippers and dexterous hands. Armed with the aligned latent space and high-fidelity human-hand data, we develop a progressive mixed-to-human training recipe comprising mixed human-robot co-training and human-hand online correction post-training. Through mixed co-training, LaST-HD improves generalization to novel objects, scenes, and positions using only human-hand demonstrations. With online correction, LaST-HD further adapts to novel environments and achieves over 90\% accuracy using only 20 minutes of OOL glove data.

cs.RO↗

arXiv2026

Fokker--Planck framework for stochastic octupole moment dynamics in chiral antiferromagnet Mn3Sn

We develop a reduced stochastic framework for thermally assisted octupole moment dynamics in Mn3Sn by combining the reduced Landau--Lifshitz--Gilbert (LLG) equation with the Fokker--Planck formalism. The reduced model is benchmarked against the complete three-sublattice octupole dynamics and is shown to capture the essential switching behavior with good accuracy. We then derive the corresponding Fokker--Planck equation, which is implemented and solved via a CUDA-accelerated solver. The analysis shows that the octupole dynamics are highly sensitive to the out-of-plane grid resolution because ultrafast rotation of the octupole is controlled by its very small deviations from the basal plane. The solver is validated against Monte Carlo simulations through equilibrium distributions, relaxation trajectories, and switching times. Finally, we apply the method to thermally assisted field-driven switching and demonstrate efficient access to ultra-low error probabilities beyond the practical reach of direct Monte Carlo simulations.

cond-mat.mes-hall↗

arXiv2026

LaST-R1: Reinforcing Robotic Manipulation via Adaptive Physical Latent Reasoning

Robotic foundation models require reasoning over complex visual scenes to execute adaptive actions in dynamic environments. While recent studies on latent-reasoning Vision-Language-Action (VLA) models have demonstrated the capability to capture fine-grained physical dynamics, they remain predominantly confined to static imitation learning, severely limiting their adaptability and generalization. In this paper, we present LaST-R1, a novel reinforcement learning (RL) post-training framework designed to effectively harness "latent reasoning-before-acting" policies. Specifically, we propose Latent-to-Action Policy Optimization (LAPO), a core RL algorithm that jointly optimizes the latent reasoning process and the action generation. By explicitly embedding latent Chain-of-Thought (CoT) reasoning directly within the RL optimization loop, LAPO stimulates profound physical world modeling, which in turn drives robust execution in interactive environments. Furthermore, an adaptive latent CoT mechanism is introduced, allowing the policy to dynamically modulate its reasoning horizon based on diverse environment states. Experiments show that LaST-R1 achieves a near-perfect 99.9% average success rate on the LIBERO benchmark with only one-shot supervised warm-up, significantly improving convergence speed and performance over prior state-of-the-art (SOTA) methods. In real-world deployments, LaST-R1 yields up to a 22.5% average improvement over SOTA supervised fine-tuning approach across four complex tasks, including both single-arm and dual-arm settings. Finally, LaST-R1 demonstrates strong generalization across simulated and real-world environments.

cs.RO↗

arXiv2025

Influence of thermal noise on the field-driven dynamics of the non-collinear antiferromagnet Mn3Sn

$\mathrm{Mn_3Sn}(0\overline{1}\overline{1}0)[0001]$ experiences a tensile strain when grown epitaxially on $\mathrm{MgO}(110)[001]$, and thus the energy landscape changes from six-fold symmetry to two-fold symmetry. External magnetic field further breaks the symmetry and the resulting energy landscape is sensitive to the field orientation relative to the easy axis. In the presence of thermal noise, the relaxation of the magnetic octupole moment in a strained Mn$_3$Sn film is composed of four distinct escape processes involving the two saddle points and two equilibrium states in the energy landscape. Here, we apply harmonic transition-state theory to derive analytical expressions for the inter-well escape time and octupole moment relaxation time, both influenced by an external symmetry-breaking magnetic field and finite thermal noise in the intermediate-to-high damping regime. The analytical predictions are in strong agreement with comprehensive numerical simulations based on coupled LLG equations. The results presented here are crucial toward realizing Mn$_3$Sn's applications in random number generation and probabilistic computing.

cond-mat.mes-hall↗

arXiv2025

AC-DiT: Adaptive Coordination Diffusion Transformer for Mobile Manipulation

Recently, mobile manipulation has attracted increasing attention for enabling language-conditioned robotic control in household tasks. However, existing methods still face challenges in coordinating mobile base and manipulator, primarily due to two limitations. On the one hand, they fail to explicitly model the influence of the mobile base on manipulator control, which easily leads to error accumulation under high degrees of freedom. On the other hand, they treat the entire mobile manipulation process with the same visual observation modality (e.g., either all 2D or all 3D), overlooking the distinct multimodal perception requirements at different stages during mobile manipulation. To address this, we propose the Adaptive Coordination Diffusion Transformer (AC-DiT), which enhances mobile base and manipulator coordination for end-to-end mobile manipulation. First, since the motion of the mobile base directly influences the manipulator's actions, we introduce a mobility-to-body conditioning mechanism that guides the model to first extract base motion representations, which are then used as context prior for predicting whole-body actions. This enables whole-body control that accounts for the potential impact of the mobile base's motion. Second, to meet the perception requirements at different stages of mobile manipulation, we design a perception-aware multimodal conditioning strategy that dynamically adjusts the fusion weights between various 2D visual images and 3D point clouds, yielding visual features tailored to the current perceptual needs. This allows the model to, for example, adaptively rely more on 2D inputs when semantic information is crucial for action prediction, while placing greater emphasis on 3D geometric information when precise spatial understanding is required. We validate AC-DiT through extensive experiments on both simulated and real-world mobile manipulation tasks.

cs.RO↗

arXiv2025

RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation

In this paper, we introduce RoboMIND (Multi-embodiment Intelligence Normative Data for Robot Manipulation), a dataset containing 107k demonstration trajectories across 479 diverse tasks involving 96 object classes. RoboMIND is collected through human teleoperation and encompasses comprehensive robotic-related information, including multi-view observations, proprioceptive robot state information, and linguistic task descriptions. To ensure data consistency and reliability for imitation learning, RoboMIND is built on a unified data collection platform and a standardized protocol, covering four distinct robotic embodiments: the Franka Emika Panda, the UR5e, the AgileX dual-arm robot, and a humanoid robot with dual dexterous hands. Our dataset also includes 5k real-world failure demonstrations, each accompanied by detailed causes, enabling failure reflection and correction during policy learning. Additionally, we created a digital twin environment in the Isaac Sim simulator, replicating the real-world tasks and assets, which facilitates the low-cost collection of additional training data and enables efficient evaluation. To demonstrate the quality and diversity of our dataset, we conducted extensive experiments using various imitation learning methods for single-task settings and state-of-the-art Vision-Language-Action (VLA) models for multi-task scenarios. By leveraging RoboMIND, the VLA models achieved high manipulation success rates and demonstrated strong generalization capabilities. To the best of our knowledge, RoboMIND is the largest multi-embodiment teleoperation dataset collected on a unified platform, providing large-scale and high-quality robotic training data. Our project is at https://x-humanoid-robomind.github.io/.

cs.RO↗

arXiv2024

Spintronic devices and applications using noncollinear chiral antiferromagnets

Antiferromagnetic materials have a vanishingly small net magnetization, which generates weak dipolar fields and makes them robust against external magnetic perturbation and rapid magnetization dynamics, as dictated by the geometric mean of their exchange and anisotropy energies. However, experimental and theoretical techniques to detect and manipulate the antiferromagnetic order in a fully electrical manner must be developed to enable advanced spintronic devices with antiferromagnets (AFMs) as their active spin-dependent elements. Among the various AFMs, conducting AFMs offer high electrical and thermal conductivities and strong electron-spin-phonon interactions. Noncollinear metallic AFMs with negative chirality, including Mn3Sn, Mn3Ge, and Mn3GaN, offer rich physics that arises from their topology. In this review article, we introduce the crystal structure and the physical phenomena observed in negative chirality AFMs. Experimental and theoretical advances related to current-induced dynamics on the spin structure of Mn3Sn are discussed. We then present a potential AFM spintronic device that can serve as a non-volatile memory, high-frequency signal generator, neuron emulator, and even a probabilistic bit, depending on the design parameters and the input stimuli, i.e., amplitude and pulse width of the injected spin current and the external magnetic field. In this device, spin-orbit torques can be used to manipulate the order parameter, while the device state can be read via tunneling magnetoresistance. We also present analytic models that relate the performance characteristics of the device with its design parameters, thus enabling a rapid technology-device assessment. Effects of Joule heating and thermal noise on the device characteristics are briefly discussed. We close the paper by summarizing the status of research and present our outlook in this rapidly evolving research field.

cond-mat.mtrl-sci↗

arXiv2024

Impact of strain on the SOT-driven dynamics of thin film Mn$_3$Sn

Mn$_3$Sn, a metallic antiferromagnet with an anti-chiral 120$^\circ$ spin structure, generates intriguing magneto-transport signatures such as a large anomalous Hall effect, spin-polarized current with novel symmetries, anomalous Nernst effect, and magneto-optic Kerr effect. When grown epitaxially as MgO(110)[001]$\parallel$ Mn$_3$Sn($0\bar{1}\bar{1}0$)[0001], Mn$_3$Sn experiences a uniaxial tensile strain, which changes the bulk six-fold anisotropy landscape to a perpendicular magnetic anisotropy with two stable states. In this work, we investigate the field-assisted spin orbit-torque (SOT)-driven response of the order parameter in single-domain Mn$_3$Sn with uniaxial tensile strain. We find that for a non-zero external magnetic field, the order parameter can be switched between the two stable states if the magnitude of the input current is between two field-dependent critical currents. Below the lower critical current, the order parameter exhibits a stationary state in the vicinity of the initial stable state. On the other hand, above the higher critical current, the order parameter shows oscillatory dynamics which could be tuned from the 100's of megahertz to the gigahertz range. We obtain approximate expressions of the two critical currents and find them to agree very well with the numerical simulations for experimentally relevant magnetic fields. We also obtain unified functional form of the switching time versus the input current for different magnetic fields. Finally, we show that for lower values of Gilbert damping ($α\leq 2\times 10^{-3}$), the critical currents and the final steady states depend significantly on the damping constant. The numerical and analytic results presented in our work can be used by both theorists and experimentalists to understand the SOT-driven order dynamics in PMA Mn$_3$Sn and design future experiments and devices.

cond-mat.mtrl-sci↗

arXiv2023

Order parameter dynamics in Mn$_3$Sn driven by DC and pulsed spin-orbit torques

We numerically investigate and develop analytic models for both the DC and pulsed spin-orbit-torque (SOT)-driven response of order parameter in single-domain Mn$_3$Sn, which is a metallic antiferromagnet with an anti-chiral 120$^\circ$ spin structure. We show that DC currents above a critical threshold can excite oscillatory dynamics of the order parameter in the gigahertz to terahertz frequency spectrum. Detailed models of the oscillation frequency versus input current are developed and found to be in excellent agreement with the numerical simulations of the dynamics. In the case of pulsed excitation, the magnetization can be switched from one stable state to any of the other five stable states in the Kagome plane by tuning the duration or the amplitude of the current pulse. Precise functional forms of the final switched state versus the input current are derived, offering crucial insights into the switching dynamics of Mn$_3$Sn. The readout of the magnetic state can be carried out via either the anomalous Hall effect, or the recently demonstrated tunneling magnetoresistance in an all-Mn$_3$Sn junction. We also discuss possible disturbance of the magnetic order due to heating that may occur if the sample is subject to large currents. Operating the device in pulsed mode or using low DC currents reduces the peak temperature rise in the sample due to Joule heating. Our predictive modeling and simulation results can be used by both theorists and experimentalists to explore the interplay of SOT and the order dynamics in Mn$_3$Sn, and to further benchmark the device performance.

cond-mat.mtrl-sci↗

arXiv2023

Chain of Thought Prompt Tuning in Vision Language Models

Language-Image Pre-training has demonstrated promising results on zero-shot and few-shot downstream tasks by prompting visual models with natural language prompts. However, most recent studies only use a single prompt for tuning, neglecting the inherent step-to-step cognitive reasoning process that humans conduct in complex task settings, for example, when processing images from unfamiliar domains. Chain of Thought is a simple and effective approximation to human reasoning process and has been proven useful for natural language processing (NLP) tasks. Based on this cognitive intuition, we believe that conducting effective reasoning is also an important problem in visual tasks, and a chain of thought could be a solution to this problem. In this work, we propose a novel chain of thought prompt tuning for vision-language modeling. Extensive experiments show that our method not only generalizes better in image classification tasks, has greater transferability beyond a single dataset, and has stronger domain generalization performance, but also performs much better in imagetext retrieval and visual question answering, which require more reasoning capabilities. We are the first to successfully adapt chain-of-thought prompting that combines visual and textual embeddings. We will release our codes

cs.CV↗