arXiv subjects

Jingyang He

Publications and source records attributed to Jingyang He.

13 recordsLinked to original sources

arXiv2026

HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL

Humanoid robots hold great promise as general-purpose agents in human-centered environments, yet generalist vision-language-action (VLA) foundation models are not readily applicable to humanoid whole-body loco-manipulation. The high dimensionality and interdependence of humanoid motions make it challenging for conventional single-stage VLA architectures to coordinate locomotion, waist posture, and dual-arm manipulation effectively. Moreover, policies trained through offline behavior cloning can remain suboptimal during real-world deployment. Although online reinforcement learning can refine policies through real-world interaction, directly tuning large VLA backbones demands excessive computation and may introduce safety risks during real-robot exploration. To address these bottlenecks, we introduce HAF (Humanoid Adaptation Framework), a two-part framework consisting of HAF-VLA and HAF-Steer that transfers off-the-shelf generalist VLA foundation models to humanoid whole-body loco-manipulation. HAF-VLA is a hierarchical action-flow generator built on a pretrained flow-matching VLA. It splits full-body action denoising into three sequential stages with stage embeddings and cross-stage KV caches that retain kinematic dependencies, avoiding incoherent whole-body actions from one-shot generation. On top of the frozen HAF-VLA, HAF-Steer is a latent offline-to-online RL pipeline that leverages flow-matching invertibility and DCT-based dimensionality reduction to restrict RL optimization to a compact noise subspace and train a regularized SAC policy. This avoids updating the large VLA backbone and enables efficient real-world policy refinement. Evaluated on seven real-world humanoid loco-manipulation tasks, HAF surpasses vanilla single-stage VLA baselines and improves whole-body coordination and task performance. Project website: https://grange007.github.io/HAF .

cs.RO↗

arXiv2026

Demo-JEPA: Joint-Embedding Predictive Architecture for One-shot Cross-Embodiment Imitation

Robotic imitation learning is often treated as reproducing demonstrated actions, but actions are inherently embodiment-specific. When demonstrations come from humans or robots with different morphology, kinematics, or action spaces, this action-centric view requires shared action spaces, heuristic retargeting, or large-scale multi-embodiment co-training. We instead view demonstrations as implicit specifications of future goals: the target agent should infer what state the demonstrator is trying to realize, rather than how the demonstrator executes it. We propose Demo-JEPA, a cross-embodiment imitation framework that decouples demonstration intent from embodiment-specific execution. Built on a JEPA-based world model, Demo-JEPA translates source visual demonstrations into target-compatible future latent trajectories in a shared predictive representation space. The target agent then uses these latent trajectories as subgoals and realizes them through planning under its own learned forward dynamics. Because Demo-JEPA avoids action-level correspondence and requires only visual demonstrations plus the target agent's own interaction experience, it supports flexible imitation across heterogeneous embodiments. Experiments on RLBench and real-world manipulation tasks show that Demo-JEPA matches specialized in-domain planners and generalizes to unseen tasks and embodiment configurations where prior methods fail.

cs.RO↗

arXiv2026

VEGA: Visual Encoder Grounding Alignment for Spatially-Aware Vision-Language-Action Models

Precise spatial reasoning is fundamental to robotic manipulation, yet the visual backbones of current vision-language-action (VLA) models are predominantly pretrained on 2D image data without explicit 3D geometric supervision, resulting in representations that lack accurate spatial awareness. Existing implicit spatial grounding methods partially address this by aligning VLA features with those of 3D-aware foundation models, but they rely on empirical layer search and perform alignment on LLM-level visual tokens where spatial structure has already been entangled with linguistic semantics, limiting both generalizability and geometric interpretability. We propose VEGA (Visual Encoder Grounding Alignment), a simple yet effective framework that directly aligns the output of the VLA's visual encoder with spatially-aware features from DINOv2-FiT3D, a DINOv2 model fine-tuned with multi-view consistent 3D Gaussian Splatting supervision. By performing alignment at the visual encoder output level, VEGA grounds spatial awareness before any linguistic entanglement occurs, offering a more interpretable and principled alignment target. The alignment is implemented via a lightweight projector trained with a cosine similarity loss alongside the standard action prediction objective, and is discarded at inference time, introducing no additional computational overhead. Extensive experiments on simulation benchmark and real-world manipulation tasks demonstrate that VEGA consistently outperforms existing implicit spatial grounding baselines, establishing a new state-of-the-art among implicit spatial grounding methods for VLA models.

cs.RO↗

arXiv2025

RoboMIND 2.0: A Multimodal, Bimanual Mobile Manipulation Dataset for Generalizable Embodied Intelligence

While data-driven imitation learning has revolutionized robotic manipulation, current approaches remain constrained by the scarcity of large-scale, diverse real-world demonstrations. Consequently, the ability of existing models to generalize across long-horizon bimanual tasks and mobile manipulation in unstructured environments remains limited. To bridge this gap, we present RoboMIND 2.0, a comprehensive real-world dataset comprising over 310K dual-arm manipulation trajectories collected across six distinct robot embodiments and 739 complex tasks. Crucially, to support research in contact-rich and spatially extended tasks, the dataset incorporates 12K tactile-enhanced episodes and 20K mobile manipulation trajectories. Complementing this physical data, we construct high-fidelity digital twins of our real-world environments, releasing an additional 20K-trajectory simulated dataset to facilitate robust sim-to-real transfer. To fully exploit the potential of RoboMIND 2.0, we propose MIND-2 system, a hierarchical dual-system frame-work optimized via offline reinforcement learning. MIND-2 integrates a high-level semantic planner (MIND-2-VLM) to decompose abstract natural language instructions into grounded subgoals, coupled with a low-level Vision-Language-Action executor (MIND-2-VLA), which generates precise, proprioception-aware motor actions.

cs.RO↗

arXiv2025

Recover Experimental Data with Selection Bias using Counterfactual Logic

Selection bias, arising from the systematic inclusion or exclusion of certain samples, poses a significant challenge to the validity of causal inference. While Bareinboim et al. introduced methods for recovering unbiased observational and interventional distributions from biased data using partial external information, the complexity of the backdoor adjustment and the method's strong reliance on observational data limit its applicability in many practical settings. In this paper, we formally discover the recoverability of $P(Y^*_{x^*})$ under selection bias with experimental data. By explicitly constructing counterfactual worlds via Structural Causal Models (SCMs), we analyze how selection mechanisms in the observational world propagate to the counterfactual domain. We derive a complete set of graphical and theoretical criteria to determine that the experimental distribution remain unaffected by selection bias. Furthermore, we propose principled methods for leveraging partially unbiased observational data to recover $P(Y^*_{x^*})$ from biased experimental datasets. Simulation studies replicating realistic research scenarios demonstrate the practical utility of our approach, offering concrete guidance for mitigating selection bias in applied causal inference.

stat.ME↗

arXiv2024

RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation

In this paper, we introduce RoboMIND (Multi-embodiment Intelligence Normative Data for Robot Manipulation), a dataset containing 107k demonstration trajectories across 479 diverse tasks involving 96 object classes. RoboMIND is collected through human teleoperation and encompasses comprehensive robotic-related information, including multi-view observations, proprioceptive robot state information, and linguistic task descriptions. To ensure data consistency and reliability for imitation learning, RoboMIND is built on a unified data collection platform and a standardized protocol, covering four distinct robotic embodiments: the Franka Emika Panda, the UR5e, the AgileX dual-arm robot, and a humanoid robot with dual dexterous hands. Our dataset also includes 5k real-world failure demonstrations, each accompanied by detailed causes, enabling failure reflection and correction during policy learning. Additionally, we created a digital twin environment in the Isaac Sim simulator, replicating the real-world tasks and assets, which facilitates the low-cost collection of additional training data and enables efficient evaluation. To demonstrate the quality and diversity of our dataset, we conducted extensive experiments using various imitation learning methods for single-task settings and state-of-the-art Vision-Language-Action (VLA) models for multi-task scenarios. By leveraging RoboMIND, the VLA models achieved high manipulation success rates and demonstrated strong generalization capabilities. To the best of our knowledge, RoboMIND is the largest multi-embodiment teleoperation dataset collected on a unified platform, providing large-scale and high-quality robotic training data. Our project is at https://x-humanoid-robomind.github.io/.

cs.RO↗

arXiv2023

Optical Second Harmonic Generation in Anisotropic Multilayers with Complete Multireflection of Linear and Nonlinear Waves using #SHAARP.ml Package

Optical second harmonic generation (SHG) is a nonlinear optical effect widely used for nonlinear optical microscopy and laser frequency conversion. Closed-form analytical solution of the nonlinear optical responses is essential for evaluating the optical responses of new materials whose optical properties are unknown a priori. A recent open-source code, SHAARP(si), can provide such closed form solutions for crystals with arbitrary symmetries, orientations, and anisotropic properties at a single interface. However, optical components are often in the form of slabs, thin films on substrates, and multilayer heterostructures with multiple reflections of both the fundamental and up to ten different SHG waves at each interface, adding significant complexity. Many approximations have therefore been employed in the existing analytical approaches, such as slowly varying approximation, weak reflection of the nonlinear polarization, transparent medium, high crystallographic symmetry, Kleinman symmetry, easy crystal orientation along a high-symmetry direction, phase matching conditions and negligible interference among nonlinear waves, which may lead to large errors in the reported material properties. To avoid these approximations, we have developed an open-source package named Second Harmonic Analysis of Anisotropic Rotational Polarimetry in Multilayers (SHAARP(ml)). The reliability and accuracy are established by experimentally benchmarking with both the SHG polarimetry and Maker fringes predicted from the package using standard materials.

physics.optics↗

arXiv2023

Large Enhancements in Optical and Piezoelectric Properties in Ferroelectric Zn1-xMgxO Thin Films through Engineering Electronic and Ionic Anharmonicities

Multifunctionality as a paradigm requires materials exhibiting multiple superior properties. Integrating second-order optical nonlinearity and large bandgap with piezoelectricity could, for example, enable broadband, strain-tunable photonics. Though very different phenomena at distinct frequencies, both second-order optical nonlinearity and piezoelectricity are third-rank polar tensors present only in acentric crystal structures. However, simultaneously enhancing both phenomena is highly challenging since it involves competing effects with tradeoffs. Recently, a large switchable ferroelectric polarization of ~ 80 uC cm-2 was reported in Zn1-xMgxO films. Here, ferroelectric Zn1-xMgxO is demonstrated to be a platform that hosts simultaneously a 30% increase in the electronic bandgap, a 50% enhancement in the second harmonic generation coefficients, and a near 200% improvement in the piezoelectric coefficients over pure ZnO. These enhancements are shown to be due to a 400% increase in the electronic anharmonicity and a ~200% decrease in the ionic anharmonicity with Mg substitution. Precisely controllable periodic ferroelectric domain gratings are demonstrated down to 800 nm domain width, enabling ultraviolet quasi-phase-matched optical harmonic generation as well as domain-engineered piezoelectric devices.

cond-mat.mtrl-sci↗

arXiv2022

High-throughput screening assisted discovery of a stable layered anti-ferromagnetic semiconductor: CdFeP2Se6

Recent advances in two-dimensional (2D) magnetism have heightened interest in layered magnetic materials due to their potential for spintronics. In particular, layered semiconducting antiferromagnets exhibit intriguing low-dimensional semiconducting behavior with both charge and spin as carrier controls. However, synthesis of these compounds is challenging and remains rare. Here, we conducted firstprinciples based high-throughput search to screen potentially stable mixed metal phosphorous trichalcogenides (MM'P2X6, where M and M' are transition metals and X is a chalcogenide) that have a wide range of tunable bandgaps and interesting magnetic properties. Among the potential candidates, we successfully synthesized a stable semiconducting layered magnetic material, CdFeP2Se6, that exhibits a short-range antiferromagnetic order at TN = 21 K with an indirect band gap of 2.23 eV. Our work suggests that highthroughput screening assisted synthesis be an effective method for layered magnetic materials discovery.

cond-mat.mtrl-sci↗

arXiv2022

SHAARP: An Open-Source Package for Analytical and Numerical Modeling of Optical Second Harmonic Generation in Anisotropic Crystals

Optical second harmonic generation is a second-order nonlinear process that combines two photons of a given frequency into a third photon at twice the frequency. Due to the symmetry constraints, it is widely used as a sensitive probe to detect broken inversion symmetry and local polar order. Analytical modeling of the electric-dipole SHG response is essential to extract fundamental properties of materials from experiments. However, complexity builds up dramatically in the analytical model when the probed crystal is of a low bulk crystal symmetry, with a low-symmetry surface orientation, exhibits absorption and dispersion, and consists of multiple interfaces. As a result, there is a largely uneven landscape in the literature on the SHG modeling of new materials, involving numerous approximations and a wide range of (in)accuracies, leading to a rather scattered dataset of reported SHG nonlinear susceptibility. Towards streamlining the reliability and accuracy of this process, we have developed an open-source package called the Second Harmonic Analysis of Anisotropic Rotational Polarimetry (SHAARP) which derives analytical solutions and performs numerical simulations of reflection SHG from a single interface for homogeneous crystals. Five key generalizations in SHG modeling are implemented, including all crystal symmetries down to triclinic, any crystal orientation, complex dielectric tensor (refractive indices) with frequency dispersion, and general polarization states of the light. SHAARP enables accurate anisotropic modeling of SHG response for a broad range of materials systems. The method is extendible to multiple interfaces. The code is free to download from https://github.com/Rui-Zu/SHAARP

physics.optics↗

arXiv2022

SnP$_2$S$_6$: A Promising Infrared Nonlinear Optical Crystal with Strong Non-Resonant Second Harmonic Generation and Phase-matchability

High-power infrared laser systems with broadband tunability are of great importance due to their wide range of applications in spectroscopy and free-space communications. These systems require nonlinear optical (NLO) crystals for wavelength up/down conversion using sum/difference frequency generation, respectively. NLO crystals need to satisfy many competing criteria, including large nonlinear optical susceptibility, large laser induced damage threshold (LIDT), wide transparency range and phase-matchability. Here, we report bulk single crystals of SnP_2S_6 with a large non-resonant SHG coefficient of d33= 53 pm/V at 1550nm and a large LIDT of 350 GW/cm^2 for femtosecond laser pulses. It also exhibits a broad transparency range from 0.54 {\mu}m to 8.5{\mu}m (bandgap of ~2.3 eV) and can be both Type I and Type II phase-matched. The complete linear and SHG tensors are measured as well as predicted by first principles calculations, and they are in excellent agreement. A proximate double-resonance condition in the electronic band structure for both the fundamental and the SHG light is shown to enhance the non-resonant SHG response. Therefore, SnP2S6 is an outstanding candidate for infrared laser applications.

physics.optics↗

arXiv2021

Giant Non-resonant Infrared Second Order Nonlinearity in $\gamma$-NaAsSe$_2$

Infrared laser systems are vital for applications in spectroscopy, communications, and biomedical devices, where infrared nonlinear optical (NLO) crystals are required for broadband frequency down-conversion. Such crystals need to have high non-resonant NLO coefficients, a large bandgap, low absorption coefficient, phase-matchability among other competing demands, e.g., a larger bandgap leads to smaller NLO coefficients. Here, we report the successful growth of single crystals of $\gamma$-NaAsSe$_2$ that exhibit a giant second harmonic generation (SHG) susceptibility of d$_{11}$=590 pm V$^{-1}$ at 2$\mu$m wavelength; this is ~ eighteen times larger than that of commercial AgGaSe$_2$ while retaining a similar bandgap of ~1.87eV, making it an outstanding candidate for quasi-phase-matched devices utilizing d$_{11}$. In addition, $\gamma$-NaAsSe$_2$ is both Type I and Type II phase-matchable, and has a transparency range up to 16$\mu$m wavelength. Thus $\gamma$-NaAsSe2 is a promising bulk NLO crystal for infrared laser applications.

physics.optics↗

arXiv2020

Evidence for a magnetic-field induced ideal type-II Weyl state in antiferromagnetic topological insulator Mn(Bi1-xSbx)2Te4

The discovery of Weyl semimetals (WSMs) has fueled tremendous interest in condensed matter physics. WSMs require breaking of either inversion symmetry (IS) or time-reversal symmetry (TRS); they can be categorized into type-I and type-II WSMs, characterized by un-tilted and strongly tilted Weyl cones respectively. Type-I WSMs with breaking of IS or TRS and type-II WSMs with IS breaking have been realized experimentally, but TRS-breaking type-II WSM still remains elusive. In this article, we report an ideal TRS-breaking type-II WSM with only one pair of Weyl nodes observed in the antiferromagnetic topological insulator Mn(Bi1-xSbx)2Te4 under magnetic fields. This state is manifested by a large intrinsic anomalous Hall effect, a non-trivial $\mathrm{{\pi}}$ Berry phase of the cyclotron orbit and a large positive magnetoresistance in the ferromagnetic phase at an optimal sample composition. Our results establish a promising platform for exploring the physics underlying the long-sought, ideal TRS breaking type-II WSM.

cond-mat.mtrl-sci↗