arXiv subjects

Mi Yan

Publications and source records attributed to Mi Yan.

At least 19 recordsLinked to original sources

arXiv2026

GIF: Agentic Generation of Interactive and Functional Object Compositions for Robot Learning

Robot manipulation foundation models require scalable evaluation and data generation across diverse scenarios, with simulation providing an environment for both. Automated scene generation offers a promising path, yet prior work has largely emphasized coarse-grained scene layouts rather than fine-grained functional object compositions. Motivated by this gap, we present GIF, an agentic Generation framework for Interactive and Functional object compositions. In this framework, we recast this problem as disentangled reconstruction followed by relative pose recovery. CoGen produces instance-disentangled meshes with coarse initial poses leveraging complementary strengths of 2D and 3D generative models. GPRM refines the relative pose under joint geometric and physical guidance, and a VLM verifier selects the candidate that best matches the structured specification. We further construct a benchmark spanning eight representative contact-geometry classes and compare with state-of-the-art generators; GIF improves both asset quality and relation matching, while reducing collision rate to below 1%. Finally, we synthesize data for policy learning, revealing diversity scaling in both simulation and real-world deployment.

cs.RO↗

arXiv2026

ZETA: A Controlled Study of Zero-Shot Cross-Embodiment VLA Transfer for Tabletop Manipulation

Zero-shot generalization to unseen embodiments is important for generalizable vision-language-action (VLA) models as robot hardware evolves and task-specific data collection remains costly. However, a systematic understanding of this problem remains limited, in part because the literature lacks a unified zero-shot transfer definition and controlled evaluation settings that isolate embodiment changes from differences in tasks, scenes, or protocols. To address this gap, we first distinguish strict zero-shot transfer, where the target embodiment is absent from all training data, from pretrain-exposed zero-shot transfer, where it appears only during pretraining. We then introduce a controlled benchmark spanning 14 held-out target embodiments across simulation and real-world validation. Within this framework, we conduct a controlled analysis of four factors: state-action representations, pretraining embodiment diversity, auxiliary co-training objectives, and target-embodiment exposure. Experimental results show that local end-effector (EEF) state-action representations, the source embodiment diversity, and auxiliary co-training improve cross-embodiment transfer by around 15, 18, and 7 percentage points, respectively. We further find that adding only 5% target-embodiment data during pretraining improves average target-embodiment progress by 13.4 percentage points, showing that strict and pretrain-exposed zero-shot transfer are distinct and should be reported separately. Together, these findings provide practical guidance for evaluating and improving cross-embodiment VLA transfer in stationary tabletop manipulation with two-finger grippers, while motivating future investigation of broader settings including mobile-base control, dexterous hands, and long-horizon tasks.

cs.RO↗

arXiv2026

KPGrasp: Scalable Keypoint Flow Matching for Dexterous Grasp Generation

Generating high-quality dexterous grasps remains challenging for learning-based methods, which often depend on carefully tuned contact losses or costly contact-based test-time refinement. We present KPGrasp, a flow-matching framework that learns dexterous grasp priors from large-scale data rather than relying on contact losses or contact-based test-time refinement. KPGrasp couples an all-Euclidean 3D hand-keypoint parameterization with a simple yet scalable Transformer flow model. The parameterization avoids the drawbacks of the conventional mixed SE(3) pose and joint-angle output space, expresses grasps in the same frame as the object point cloud, and thus enables native spatial reasoning; the Transformer flow model is trained with only the standard flow-matching loss and scales effectively with data, model capacity, and batch size. Experiments demonstrate state-of-the-art performance on two simulation benchmarks. On the Dexonomy benchmark, it reaches a 76.3% grasp success rate, improving over the strongest directly comparable baseline by 47.4% while reducing penetration depth to 2.4 mm. The same model also achieves the best average performance on the DexGrasp Anything benchmark without fine-tuning. For batched inference, KPGrasp requires only 0.032 s per grasp. Finally, real-world experiments on 20 diverse objects demonstrate that the pipeline can be deployed in a real-world setup.

cs.RO↗

arXiv2026

Emerging Extrinsic Dexterity in Cluttered Scenes via Dynamics-aware Policy Learning

Extrinsic dexterity leverages environmental contact to overcome the limitations of prehensile manipulation. However, achieving such dexterity in cluttered scenes remains challenging and underexplored, as it requires selectively exploiting contact among multiple interacting objects with inherently coupled dynamics. Existing approaches lack explicit modeling of such complex dynamics and therefore fall short in non-prehensile manipulation in cluttered environments, which in turn limits their practical applicability in real-world environments. In this paper, we introduce a Dynamics-Aware Policy Learning (DAPL) framework that can facilitate policy learning with a learned representation of contact-induced object dynamics in cluttered environments. This representation is learned through explicit world modeling and used to condition reinforcement learning, enabling extrinsic dexterity to emerge without hand-crafted contact heuristics or complex reward shaping. We evaluate our approach in both simulation and the real world. Our method outperforms prehensile manipulation, human teleoperation, and prior representation-based policies by over 25% in success rate on unseen simulated cluttered scenes with varying densities. The real-world success rate reaches around 50% across 10 cluttered scenes, while a practical grocery deployment further demonstrates robust sim-to-real transfer and applicability.

cs.RO↗

arXiv2026

LDA-1B: Scaling Latent Dynamics Action Model via Universal Embodied Data Ingestion

Recent robot foundation models largely rely on large-scale behavior cloning, which imitates expert actions but discards transferable dynamics knowledge embedded in heterogeneous embodied data. While the Unified World Model (UWM) formulation has the potential to leverage such diverse data, existing instantiations struggle to scale to foundation-level due to coarse data usage and fragmented datasets. We introduce LDA-1B, a robot foundation model that scales through universal embodied data ingestion by jointly learning dynamics, policy, and visual forecasting, assigning distinct roles to data of varying quality. To support this regime at scale, we assemble and standardize EI-30k, an embodied interaction dataset comprising over 30k hours of human and robot trajectories in a unified format. Scalable dynamics learning over such heterogeneous data is enabled by prediction in a structured DINO latent space, which avoids redundant pixel-space appearance modeling. Complementing this representation, LDA-1B employs a multi-modal diffusion transformer to handle asynchronous vision and action streams, enabling stable training at the 1B-parameter scale. Experiments in simulation and the real world show LDA-1B outperforms prior methods (e.g., $\pi_{0.5}$) by up to 21\%, 48\%, and 23\% on contact-rich, dexterous, and long-horizon tasks, respectively. Notably, LDA-1B enables data-efficient fine-tuning, gaining 10\% by leveraging 30\% low-quality trajectories typically harmful and discarded.

cs.RO↗

arXiv2026

Any3D-VLA: Enhancing VLA Robustness via Diverse Point Clouds

Existing Vision-Language-Action (VLA) models typically take 2D images as visual input, which limits their spatial understanding in complex scenes. How can we incorporate 3D information to enhance VLA capabilities? We conduct a pilot study across different observation spaces and visual representations. The results show that explicitly lifting visual input into point clouds yields representations that better complement their corresponding 2D representations. To address the challenges of (1) scarce 3D data and (2) the domain gap induced by cross-environment differences and depth-scale biases, we propose Any3D-VLA. It unifies the simulator, sensor, and model-estimated point clouds within a training pipeline, constructs diverse inputs, and learns domain-agnostic 3D representations that are fused with the corresponding 2D representations. Simulation and real-world experiments demonstrate Any3D-VLA's advantages in improving performance and mitigating the domain gap. Our project homepage is available at https://xianzhefan.github.io/Any3D-VLA.github.io.

cs.CV↗

arXiv2025

StereoVLA: Enhancing Vision-Language-Action Models with Stereo Vision

While Vision-Language-Action (VLA) models excel in generalist manipulation, they often lack fine-grained spatial awareness and show limited viewpoint robustness. This limitation largely stems from the reliance on pretrained RGB encoders, which lack explicit geometric cues and prioritize semantic alignment over geometric representation. We argue that effective visual representations for VLA models must jointly encode both semantic and geometric information. In this paper, we introduce StereoVLA, the first VLA model to incorporate rich geometric cues from large-scale synthetic stereo data. StereoVLA employs a Geometric-and-Semantic (GeoSem) vision encoder that extracts geometric cues from subtle stereo-view disparities for precise spatial perception, while simultaneously capturing semantic features from pixel observations to support language-conditioned manipulation. Additionally, we introduce two synergistic co-training objectives: Interaction-Region Depth Estimation for precise spatial reasoning, and Camera Parameter Estimation to implicitly align perception and action coordinate systems. Compared with baselines that employ various input modalities, StereoVLA achieves a 33.4% absolute gain in success rate in real-world experiments and demonstrates robustness to near-hemispheric camera perspectives. Project page: https://shengliangd.github.io/StereoVLA-Webpage.

cs.RO↗

arXiv2025

Compressibility measurement of the thermal MI--BG transition in an optical lattice

Disorder can be applied to transform conducting to insulating states by localizing individual quantum particles. The interplay between disorder and interactions in many-particle systems leads to a richer tapestry of quantum phase transitions. Here, we report the measurement in an ultracold lattice gas of a disorder-induced transition from a state with small disorder-independent compressibility to a state for which compressibility increases with disorder. At zero temperature this is the transition from a Mott insulator (MI) to a Bose glass (BG), both of which are insulating states. This transformation is observed using measurements of core compressibility. By determining how double occupancy changes with atom number, we identify the threshold disorder strength required to switch from disorder-independent MI-like to disorder-dependent BG-like compressible behavior.

cond-mat.quant-gas↗

arXiv2025

GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data

Embodied foundation models are gaining increasing attention for their zero-shot generalization, scalability, and adaptability to new tasks through few-shot post-training. However, existing models rely heavily on real-world data, which is costly and labor-intensive to collect. Synthetic data offers a cost-effective alternative, yet its potential remains largely underexplored. To bridge this gap, we explore the feasibility of training Vision-Language-Action models entirely with large-scale synthetic action data. We curate SynGrasp-1B, a billion-frame robotic grasping dataset generated in simulation with photorealistic rendering and extensive domain randomization. Building on this, we present GraspVLA, a VLA model pretrained on large-scale synthetic action data as a foundational model for grasping tasks. GraspVLA integrates autoregressive perception tasks and flow-matching-based action generation into a unified Chain-of-Thought process, enabling joint training on synthetic action data and Internet semantics data. This design helps mitigate sim-to-real gaps and facilitates the transfer of learned actions to a broader range of Internet-covered objects, achieving open-vocabulary generalization in grasping. Extensive evaluations across real-world and simulation benchmarks demonstrate GraspVLA's advanced zero-shot generalizability and few-shot adaptability to specific human preferences. We will release SynGrasp-1B dataset and pre-trained weights to benefit the community.

cs.RO↗

arXiv2024

MaskClustering: View Consensus based Mask Graph Clustering for Open-Vocabulary 3D Instance Segmentation

Open-vocabulary 3D instance segmentation is cutting-edge for its ability to segment 3D instances without predefined categories. However, progress in 3D lags behind its 2D counterpart due to limited annotated 3D data. To address this, recent works first generate 2D open-vocabulary masks through 2D models and then merge them into 3D instances based on metrics calculated between two neighboring frames. In contrast to these local metrics, we propose a novel metric, view consensus rate, to enhance the utilization of multi-view observations. The key insight is that two 2D masks should be deemed part of the same 3D instance if a significant number of other 2D masks from different views contain both these two masks. Using this metric as edge weight, we construct a global mask graph where each mask is a node. Through iterative clustering of masks showing high view consensus, we generate a series of clusters, each representing a distinct 3D instance. Notably, our model is training-free. Through extensive experiments on publicly available datasets, including ScanNet++, ScanNet200 and MatterPort3D, we demonstrate that our method achieves state-of-the-art performance in open-vocabulary 3D instance segmentation. Our project page is at https://pku-epic.github.io/MaskClustering.

cs.CV↗

arXiv2022

Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wild

In this work, we tackle the challenging task of jointly tracking hand object pose and reconstructing their shapes from depth point cloud sequences in the wild, given the initial poses at frame 0. We for the first time propose a point cloud based hand joint tracking network, HandTrackNet, to estimate the inter-frame hand joint motion. Our HandTrackNet proposes a novel hand pose canonicalization module to ease the tracking task, yielding accurate and robust hand joint tracking. Our pipeline then reconstructs the full hand via converting the predicted hand joints into a template-based parametric hand model MANO. For object tracking, we devise a simple yet effective module that estimates the object SDF from the first frame and performs optimization-based tracking. Finally, a joint optimization step is adopted to perform joint hand and object reasoning, which alleviates the occlusion-induced ambiguity and further refines the hand pose. During training, the whole pipeline only sees purely synthetic data, which are synthesized with sufficient variations and by depth simulation for the ease of generalization. The whole pipeline is pertinent to the generalization gaps and thus directly transferable to real in-the-wild data. We evaluate our method on two real hand object interaction datasets, e.g. HO3D and DexYCB, without any finetuning. Our experiments demonstrate that the proposed method significantly outperforms the previous state-of-the-art depth-based hand and object pose estimation and tracking methods, running at a frame rate of 9 FPS.

cs.CV↗

arXiv2021

Domain Adaptation on Point Clouds via Geometry-Aware Implicits

As a popular geometric representation, point clouds have attracted much attention in 3D vision, leading to many applications in autonomous driving and robotics. One important yet unsolved issue for learning on point cloud is that point clouds of the same object can have significant geometric variations if generated using different procedures or captured using different sensors. These inconsistencies induce domain gaps such that neural networks trained on one domain may fail to generalize on others. A typical technique to reduce the domain gap is to perform adversarial training so that point clouds in the feature space can align. However, adversarial training is easy to fall into degenerated local minima, resulting in negative adaptation gains. Here we propose a simple yet effective method for unsupervised domain adaptation on point clouds by employing a self-supervised task of learning geometry-aware implicits, which plays two critical roles in one shot. First, the geometric information in the point clouds is preserved through the implicit representations for downstream tasks. More importantly, the domain-specific variations can be effectively learned away in the implicit space. We also propose an adaptive strategy to compute unsigned distance fields for arbitrary point clouds due to the lack of shape models in practice. When combined with a task loss, the proposed outperforms state-of-the-art unsupervised domain adaptation methods that rely on adversarial domain alignment and more complicated self-supervised tasks. Our method is evaluated on both PointDA-10 and GraspNet datasets. The code and trained models will be publicly available.

cs.CV↗

arXiv2019

Machine Learning and the Internet of Things Enable Steam Flood Optimization for Improved Oil Production

Recently developed machine learning techniques, in association with the Internet of Things (IoT) allow for the implementation of a method of increasing oil production from heavy-oil wells. Steam flood injection, a widely used enhanced oil recovery technique, uses thermal and gravitational potential to mobilize and dilute heavy oil in situ to increase oil production. In contrast to traditional steam flood simulations based on principles of classic physics, we introduce here an approach using cutting-edge machine learning techniques that have the potential to provide a better way to describe the performance of steam flood. We propose a workflow to address a category of time-series data that can be analyzed with supervised machine learning algorithms and IoT. We demonstrate the effectiveness of the technique for forecasting oil production in steam flood scenarios. Moreover, we build an optimization system that recommends an optimal steam allocation plan, and show that it leads to a 3% improvement in oil production. We develop a minimum viable product on a cloud platform that can implement real-time data collection, transfer, and storage, as well as the training and implementation of a cloud-based machine learning model. This workflow also offers an applicable solution to other problems with similar time-series data structures, like predictive maintenance.

stat.ML↗

arXiv2017

Spin-Orbit Driven Transitions Between Mott Insulators and Finite Momentum Superfluids of Bosons in Optical Lattices

Synthetic spin-orbit coupling in ultracold atomic gases can be taken to extremes rarely found in solids. We study a two dimensional Hubbard model of bosons in an optical lattice in the presence of spin-orbit coupling strong enough to drive direct transitions from Mott insulators to superfluids. Here we find phase-modulated superfluids with finite momentum that are generated entirely by spin-orbit coupling. We investigate the rich phase patterns of the superfluids, which may be directly probed using time-of-flight imaging of the spin-dependent momentum distribution.

cond-mat.quant-gas↗

arXiv2017

Equilibration Dynamics of Strongly Interacting Bosons in 2D Lattices with Disorder

Motivated by recent optical lattice experiments [J.-y. Choi et al., Science 352, 1547 (2016)], we study the dynamics of strongly interacting bosons in the presence of disorder in two dimensions. We show that Gutzwiller mean-field theory (GMFT) captures the main experimental observations, which are a result of the competition between disorder and interactions. Our findings highlight the difficulty in distinguishing glassy dynamics, which can be captured by GMFT, and many-body localization, which cannot be captured by GMFT, and indicate the need for further experimental studies of this system.

cond-mat.quant-gas↗

arXiv2017

Dynamics of Disordered States in the Bose-Hubbard Model with Confinement

Observations of center of mass dynamics offer a straightforward method to identify strongly interacting quantum phases of atoms placed in optical lattices. We theoretically study the dynamics of states derived from the disordered Bose-Hubbard model in a trapping potential. We find that the edge states in the trap allow center of mass motion even with insulating states in the center. We identify short and long-time scale mechanisms for edge state transport in insulating phases. We also argue that the center of mass velocity can aid in identifying a Bose-glass phase. Our zero temperature results offer important insights into mechanisms of transport of atoms in trapped optical lattices while putting bounds on center of mass dynamics expected at non-zero temperature.

cond-mat.quant-gas↗

arXiv2015

Dzyaloshinskii-Moriya Interaction and Spiral Order in Spin-orbit Coupled Optical Lattices

We show that the recent experimental realization of spin-orbit coupling in ultracold atomic gases can be used to study different types of spin spiral order and resulting multiferroic effects. Spin-orbit coupling in optical lattices can give rise to the Dzyaloshinskii-Moriya (DM) spin interaction which is essential for spin spiral order. By taking into account spin-orbit coupling and an external Zeeman field, we derive an effective spin model in the Mott insulator regime at half filling and demonstrate that the DM interaction in optical lattices can be made extremely strong with realistic experimental parameters. The rich finite temperature phase diagrams of the effective spin models for fermions and bosons are obtained via classical Monte Carlo simulations.

cond-mat.quant-gas↗

arXiv2014

Mass scaling and non-adiabatic effects in photoassociation spectroscopy of ultracold strontium atoms

We report photoassociation spectroscopy of ultracold $^{86}$Sr atoms near the intercombination line and provide theoretical models to describe the obtained bound state energies. We show that using only the molecular states correlating with the $^1S_0$$+$$^3P_1$ asymptote is insufficient to provide a mass scaled theoretical model that would reproduce the bound state energies for all isotopes investigated to date: $^{84}$Sr, $^{86}$Sr and $^{88}$Sr. We attribute that to the recently discovered avoided crossing between the $^1S_0$$+$$^3P_1$ $0_u^+$ ($^3Π_u$) and $^1S_0$$+$$^1D_2$ $0_u^+$ ($^1Σ^+_u$) potential curves at short range and we build a mass scaled interaction model that quantitatively reproduces the available $0_u^+$ and $1_u$ bound state energies for the three stable bosonic isotopes. We also provide isotope-specific two-channel models that incorporate the rotational (Coriolis) mixing between the $0_u^+$ and $1_u$ curves which, while not mass scaled, are capable of quantitatively describing the vibrational splittings observed in experiment. We find that the use of state-of-the-art ab initio potential curves significantly improves the quantitative description of the Coriolis mixing between the two -8 GHz bound states in $^{88}$Sr over the previously used model potentials. We show that one of the recently reported energy levels in $^{84}$Sr does not follow the long range bound state series and theorize on the possible causes. Finally, we give the Coriolis mixing angles and linear Zeeman coefficients for all of the photoassociation lines. The long range van der Waals coefficients $C_6(0_u^+)=3868(50)$~a.u. and $C_6(1_u)=4085(50)$~a.u. are reported.

physics.atom-ph↗