Searcharxiv⌕ Search

arXiv subjects

Yuanshuo Tan

Publications and source records attributed to Yuanshuo Tan.

4 recordsLinked to original sources

Wear and Go: Calibration Free Sparse Inertial Motion Capture via Online Sensor Setup Estimation

Sparse inertial motion capture typically assumes that each sensor's anatomical identity and mounting orientation are known. We present MOSAIC (Motion-based Sensor Assignment and Inertial Calibration), a framework that recovers this setup from ordinary motion without a calibration pose. Our central insight is that body pose changes continuously while the sensor setup persists, allowing movements that are individually ambiguous to provide complementary evidence over time. A setup network processes unordered sensor streams with temporal and cross-sensor attention, then accumulates assignment and rotation evidence into a session-level estimate. The recovered setup canonicalizes measurements for frozen pretrained pose estimators, separating sensor configuration from motion reconstruction. For sensors worn at prescribed body sites, evaluation on real inertial recordings with randomized setups shows near-calibrated pose accuracy for bounded offsets and substantial benefits from accumulation under arbitrary orientations. Transfer across pose networks supports setup recovery as a reusable interface between interchangeable sensors and pretrained pose models.

cs.CV↗

Kinematic-Tactile Aligned Dexterous Retargeting with Contact Wrench Space Alignment

Human demonstrations provide a natural and scalable source of data for learning dexterous manipulation policies. However, existing retargeting methods primarily transfer human kinematics, while the tactile information underlying physical interaction is rarely preserved across embodiments. To address this limitation, we present a hierarchical three-stage framework for kinematic-tactile aligned dexterous retargeting. We use object-centric contact wrench space as a physical representation for tactile alignment between human and robot hands. The framework first aligns hand kinematics and contact geometry, and then constructs a human wrench support profile from tactile forces and contact geometry. Robot fingertip force targets are subsequently optimized to cover the human wrench support and converted into joint position corrections, yielding directly replayable robot hand trajectories. We evaluate the framework in simulation and real robot experiments across three pouring conditions. In the real robot experiments, our method achieves a 96.7\% replay success rate and higher wrench transfer fidelity than representative retargeting baselines. The framework enables physically grounded transfer of kinematic and tactile information from human demonstrations to robot hands, providing a path toward scalable learning of contact-rich dexterous manipulation.

cs.RO↗

OccluDex: Hierarchical 3D Visuo-Tactile Representation Learning for Egocentric Dexterous Manipulation under Self-Occlusion

Reliable dexterous manipulation requires continuous estimation of object geometry and hand-object contact throughout interaction. With egocentric sensing, however, the manipulating hand frequently occludes task-relevant object surfaces and contact regions, reducing the visual evidence available for state estimation and thereby making robust closed-loop control and generalization to unseen object geometries particularly challenging. To address this, we present OccluDex, a hierarchical 3D visuo-tactile representation learning framework that integrates global geometric structure with local contact information for robust manipulation under dynamic self-occlusion during hand-object interaction. OccluDex adopts multi-scale masked autoencoding to progressively encode partial 3D geometry and fuses tactile contact tokens with high-level geometric features through cross-modal attention. The encoder is pretrained from synchronized human visuo-tactile demonstrations and transferred as a frozen perceptual backbone for downstream reinforcement learning. We evaluate OccluDex on a faucet rotation task, requiring one full clockwise handle revolution, and a tabletop object reorientation task, requiring a 180-degree tabletop object reorientation without toppling. In simulation experiments, OccluDex demonstrated 12.6% higher accuracy for unseen objects and 8.3% higher accuracy for previously seen objects than the strongest state-of-the-art baseline models. Physical experiments were further performed with a Shadow Hand to demonstrate successful zero-shot sim-to-real generalization on unseen physical objects. This results could enable humanoid egocentric object manipulation for seen and unseen objects even when the manipulating robotic hand occludes vision.

cs.RO↗

Full-Body Golf Swing Kinematic Reconstruction From a Smartwatch IMU

Quantitative measurement of the golf swing is critical for evaluating technique and enabling individualized feedback. However, existing methods are impractical to use on the golf course: optical motion capture is laboratory-bound, camera-based methods require impractical camera placement, and multi-sensor inertial measurement unit (IMU) systems require multi-segment setup and calibration. We thus propose a single wrist-worn IMU approach for estimating full-body joint angles during golf swings. The proposed Wrist-IMU Temporal Kinematic Network (WIT-KinNet) combines IMU embedding, feature-wise linear modulation (FiLM), and temporal convolutional encoding. Thirty-six golfers, comprising beginner and skilled players, performed full, half, and quarter swings using seven club types: driver, 3-wood, 5-hybrid, 5-iron, 7-iron, 9-iron, and sand wedge. The proposed WIT-KinNet was evaluated under subject-wise cross-validation using synchronized smartwatch IMU data and ground-truth kinematics derived from an optical motion capture (OMC) system, with OMC-free IMU calibration and IMU-based swing segmentation. The proposed approach achieved a mean absolute error of $8.53\pm2.05^\circ$ across full-body joint angles. High temporal correlation was observed for pelvic rotation and upper torso rotation ($r=0.99$ for both), with X-factor and S-factor also showing strong correlations ($r=0.97$ for both). Linear mixed-effects models of the error revealed that swing amplitude significantly affected estimation error across all five trunk--pelvis variables ($p<0.05$). The results demonstrate the feasibility of full-body golf swing kinematic estimation from a commercial smartwatch, providing a potential solution for post-swing biomechanical analysis.

cs.CV↗