SearcharxivSearch

arXiv subjects

Jaewook Yoo

Publications and source records attributed to Jaewook Yoo.

4 recordsLinked to original sources

Co-VLA: Coordination-Aware Structured Action Modeling for Dual-Arm Vision-Language-Action Systems

Vision-language-action (VLA) models show strong capabilities in single and dual-arm robotic manipulation. Prior works show coordinated bimanual behaviors can emerge from end-to-end learning, leveraging large vision-language backbones with continuous action prediction. However, as bimanual tasks become tightly coupled and execution constraints become critical, implicit coordination alone is insufficient to ensure reliable, interpretable, and stable behavior. In this work, we propose Co-VLA, a coordination-aware bimanual manipulation framework introducing explicit structural priors into VLA models. We instantiate our method on a state-of-the-art vision-language backbone by replacing its monolithic action head with a Structured Action Expert (SAE) designed for bimanual coordination. Specifically, we introduce explicit structure at the action generation level with a modular coordination-aware loss that shapes shared and residual latents according to task-specific structures. The shared latent encodes task-level coordination intent, while residual latents capture execution adjustments for each arm. At deployment, a Latent-Aware Controller (LAC) interprets the learned representations to modulate synchronization strength, execution asymmetry, smoothness, and safety constraints in real time. LAC operates at the joint-command level and remains compatible with standard control pipelines without requiring force or impedance control. Experiments across simulation and real-world benchmarks show Co-VLA significantly outperforms monolithic baselines, achieving a 27% success rate gain in tight-coordination tasks, more than doubling performance in OOD real-world scenarios (from 13% to 27%), and reducing task completion time by up to 25%.

cs.RO

MoSE: Skill-by-Skill Mixture-of-Experts Learning for Embodied Autonomous Machines

To meet the growing demand for smarter, faster, and more efficient embodied AI solutions, we introduce a novel Mixture-of-Expert (MoE) method that significantly boosts reasoning and learning efficiency for embodied autonomous systems. General MoE models demand extensive training data and complex optimization, which limits their applicability in embodied AI such as autonomous driving (AD) and robotic manipulation. In this work, we propose a skill-oriented MoE called MoSE, which mimics the human learning and reasoning process skill-by-skill, step-by-step. We introduce a skill-oriented routing mechanism that begins with defining and annotating specific skills, enabling experts to identify the necessary competencies for various scenarios and reasoning tasks, thereby facilitating skill-by-skill learning. To better align with multi-step planning in human reasoning and in end-to-end driving models, we build a hierarchical skill dataset and pretrain the router to encourage the model to think step-by-step. Unlike other multi-round dialogues, MoSE integrates valuable auxiliary tasks (e.g. perception-prediction-planning for AD, and high-level and low-level planning for robots) in one single forward process without introducing any extra computational cost. With less than 3B sparsely activated parameters, our model effectively grows more diverse expertise and outperforms models on both AD corner-case reasoning tasks and robot reasoning tasks with less than 40% of the parameters.

cs.AI

How Can Objects Help Video-Language Understanding?

Do we still need to represent objects explicitly in multimodal large language models (MLLMs)? To one extreme, pre-trained encoders convert images into visual tokens, with which objects and spatiotemporal relationships may be implicitly modeled. To the other extreme, image captions by themselves provide strong empirical performances for understanding tasks, despite missing fine-grained spatiotemporal information. To answer this question, we introduce ObjectMLLM, a framework capable of leveraging arbitrary computer vision algorithm to extract and integrate structured visual representation. Through extensive evaluations on six video question answering benchmarks, we confirm that explicit integration of object-centric representation remains necessary. Surprisingly, we observe that the simple approach of quantizing the continuous, structured object information and representing them as plain text performs the best, offering a data-efficient approach to integrate other visual perception modules into MLLM design. Our code and models are released at https://github.com/brown-palm/ObjectMLLM.

cs.CV

First Demonstration of HZO/beta-Ga2O3 Ferroelectric FinFET with Improved Memory Window

We have experimentally demonstrated the effectiveness of beta-gallium oxide (beta-Ga2O3) ferroelectric fin field-effect transistors (Fe-FinFETs) for the first time. Atomic layer deposited (ALD) hafnium zirconium oxide (HZO) is used as the ferroelectric layer. The HZO/beta-Ga2O3 Fe-FinFETs have wider counterclockwise hysteresis loops in the transfer characteristics than that of conventional planar FET, achieving record-high memory window (MW) of 13.9 V in a single HZO layer. When normalized to the actual channel width, FinFETs show an improved ION/IOFF ratio of 2.3x10^7 and a subthreshold swing value of 110 mV/dec. The enhanced characteristics are attributed to the low-interface state density (Dit), showing good interface properties between the beta-Ga2O3 and HZO layer. The enhanced polarization due to larger electric fields across the entire ferroelectric layer in FinFETs is validated using Sentaurus TCAD. After 5x10^6 program/erase (PGM/ERS) cycles, the MW was maintained at 9.2 V, and the retention time was measured up to 3x10^4 s with low degradation. Therefore, the ultrawide bandgap (UWBG) Fe-FinFET was shown to be one of the promising candidates for high-density non-volatile memory devices.

physics.app-ph