arXiv subjects

Yinqiao Wang

Publications and source records attributed to Yinqiao Wang.

At least 19 recordsLinked to original sources

arXiv2026

ATOM: Geometry-Aware Microgesture towards Object-Agnostic Tangible Interaction

This paper presents ATOM, an integrated framework towards agnostic and tangible object interactions with microgestures. Our goal is to support microgesture interactions across different everyday objects, with the capability to automatically leverage the geometric affordance of each object. We formulate a fingertip-aware detection pipeline to leverage generative 2D and 3D models for geometry enhancement and refinement. We then introduce a usability-based method to prioritize the detected elements based on their ergonomic suitability for interactions. Building on this foundation, we further develop an AR system to transform everyday handheld objects into tangible user interfaces with 0D, 1D, and 2D microgesture interactions. Across transitions among everyday cooking objects of varying shapes and sizes, ATOM outperformed ablation baselines in task completion, usability (SUS), and workload (NASA-TLX). A further study with 10 objects demonstrates ATOM's generalizability across objects and grasps, highlighting its potential towards fluid, object-agnostic tangible interaction in real-world AR scenarios.

cs.HC↗

arXiv2026

LiverPlan: A Stage-Adaptive Immersive Visual Analytics Framework for Anatomical Liver Surgical Planning

Anatomical liver resection (ALR) surgery is the most important treatment for liver cancer, yet preoperative planning demands complex, multi-stage clinical reasoning under competing safety constraints. Current 2D desktop tools are not well equipped to support this process, exhibiting three fundamental limitations: reliance on monolithic interfaces that fail to adapt to the distinct cognitive demands of each planning stage; a perceptual bottleneck caused by limited anatomical spatial representation and missing plane-vessel intersection visualization; and an attention bottleneck stemming from fragmented critical safety criteria display across separate views. We present LiverPlan, a stage-adaptive immersive visual analytics framework for ALR planning, grounded in an 8-month collaboration with two expert hepatobiliary surgeons. Decomposing the surgical planning process into three sequential yet cognitively distinct stages, LiverPlan externalizes the cognitive demand of each stage via tailored techniques: (1) context-preserving focus and hue-preserving rendering for anatomical discovery; (2) direct 3D resection plane manipulation coupled with real-time, embedded visual feedback on critical safety criteria during plan refinement; and (3) explicit plane-vessel intersection visualization for anticipatory surgery preparation. A within-subjects study with eight hepatobiliary surgeons against a desktop baseline shows large-effect-size improvements in task completion time, perceived cognitive workload, and system usability on controlled planning tasks. Moreover, our study reveals broader insights: LiverPlan reduces cognitive burden and encourages a shift in surgeons from merely satisfying safety criteria to actively optimizing them, suggesting that explicit visualization of spatial relationships lowers the cognitive barrier to complex surgical planning.

cs.HC↗

arXiv2026

CHOIR: Contact-aware 4D Hand-Object Interaction Reconstruction

We ask whether everyday open-world monocular videos can be turned into reusable 4D interaction primitives: articulated hand motion, object shape with 6D pose over time, and the when/where of contact. Such a capability would enable scalable mining of real interactions and, beyond reconstruction, support scene-aware synthesis and planning. However, reconstructing hand-object interaction (HOI) from challenging monocular videos remains difficult: methods often assume known objects or curated scenes, and separately estimated hands and objects easily become misaligned under clutter, occlusion, and unseen object geometries. Targeting this setting, we present CHOIR, a Contact-aware HOI Reconstruction framework for a monocular camera, using contact as an explicit coupling signal between hands and objects. CHOIR first initializes a coarse, contact-agnostic 4D HOI sequence from open-world visual priors. It then introduces a generative HOI spatial rectification module to predict ray-depth corrections and rectify hand-object relative placement, then derive initial per-frame contact correspondences on the rectified geometry. Last, a contact-aware joint optimization with dynamically updated contact constraints enforces geometric, temporal, and contact consistency. Experiments on controlled and challenging videos show that CHOIR improves object reconstruction, physical plausibility, and temporal consistency over state-of-the-art methods.

cs.CV↗

arXiv2026

A Collaborative Extended Reality Prototype for 3D Surgical Planning and Visualization

We present a collaborative extended reality (XR) prototype for 3D surgical planning and visualization. Our system consists of three key modules: XR-based immersive surgical planning, cloud-based data management, and coordinated stereoscopic 3D displays for interactive visualization. We describe the overall workflow, core functionalities, implementations and setups. By conducting user studies on a liver resection surgical planning case, we demonstrate the effectiveness of our prototype and provide practical insights to inspire future advances in medical XR collaboration.

cs.HC↗

arXiv2026

EgoHandICL: Egocentric 3D Hand Reconstruction with In-Context Learning

Robust 3D hand reconstruction in egocentric vision is challenging due to depth ambiguity, self-occlusion, and complex hand-object interactions. Prior methods mitigate these issues by scaling training data or adding auxiliary cues, but they often struggle in unseen contexts. We present EgoHandICL, the first in-context learning (ICL) framework for 3D hand reconstruction that improves semantic alignment, visual consistency, and robustness under challenging egocentric conditions. EgoHandICL introduces complementary exemplar retrieval guided by vision-language models (VLMs), an ICL-tailored tokenizer for multimodal context, and a masked autoencoder (MAE)-based architecture trained with hand-guided geometric and perceptual objectives. Experiments on ARCTIC and EgoExo4D show consistent gains over state-of-the-art methods. We also demonstrate real-world generalization and improve EgoVLM hand-object interaction reasoning by using reconstructed hands as visual prompts. Code and data: https://github.com/Nicous20/EgoHandICL

cs.CV↗

arXiv2025

Coordinated 2D-3D Visualization of Volumetric Medical Data in XR with Multimodal Interactions

Volumetric medical imaging technologies produce detailed 3D representations of anatomical structures. However, effective medical data visualization and exploration pose significant challenges, especially for individuals with limited medical expertise. We introduce a novel XR-based system with two key innovations: (1) a coordinated visualization module integrating Multi-layered Multi-planar Reconstruction with 3D mesh models and (2) a multimodal interaction framework combining hand gestures with LLM-enabled voice commands. We conduct preliminary evaluations, including a 15-participant user study and expert interviews, to demonstrate the system's abilities to enhance spatial understanding and reduce cognitive load. Experimental results show notable improvements in task completion times, usability metrics, and interaction effectiveness enhanced by LLM-driven voice control. While identifying areas for future refinement, our findings highlight the potential of this immersive visualization system to advance medical training and clinical practice. Our demo application and supplemental materials are available for download at: https://osf.io/bpjq5/.

cs.HC↗

arXiv2025

Jamming as a topological satisfiability transition with contact number hyperuniformity and criticality

The jamming transition between flow and amorphous-solid states exhibits paradoxical properties characterized by hyperuniformity (suppressed spatial fluctuations) and criticality (hyperfluctuations), whose origin remains unclear. Here we model the jamming transition by a topological satisfiability transition in a minimum network model with simultaneously hyperuniform distributions of contacts, diverging length scales and scale-free clusters. We show that these phenomena stem from isostaticity and mechanical stability: the former imposes a global equality, and the latter local inequalities on arbitrary sub-systems. This dual constraint bounds contact number fluctuations from both above and below, limiting them to scale with the surface area. The hyperuniform and critical exponents of the network model align with those of frictionless jamming, suggesting a new universality class of non-equilibrium phase transitions. Our results provide a minimal, dynamics-independent framework for jamming criticality and hyperuniformity in disordered systems.

cond-mat.soft↗

arXiv2025

Hand-Shadow Poser

Hand shadow art is a captivating art form, creatively using hand shadows to reproduce expressive shapes on the wall. In this work, we study an inverse problem: given a target shape, find the poses of left and right hands that together best produce a shadow resembling the input. This problem is nontrivial, since the design space of 3D hand poses is huge while being restrictive due to anatomical constraints. Also, we need to attend to the input's shape and crucial features, though the input is colorless and textureless. To meet these challenges, we design Hand-Shadow Poser, a three-stage pipeline, to decouple the anatomical constraints (by hand) and semantic constraints (by shadow shape): (i) a generative hand assignment module to explore diverse but reasonable left/right-hand shape hypotheses; (ii) a generalized hand-shadow alignment module to infer coarse hand poses with a similarity-driven strategy for selecting hypotheses; and (iii) a shadow-feature-aware refinement module to optimize the hand poses for physical plausibility and shadow feature preservation. Further, we design our pipeline to be trainable on generic public hand data, thus avoiding the need for any specialized training dataset. For method validation, we build a benchmark of 210 diverse shadow shapes of varying complexity and a comprehensive set of metrics, including a novel DINOv2-based evaluation metric. Through extensive comparisons with multiple baselines and user studies, our approach is demonstrated to effectively generate bimanual hand poses for a large variety of hand shapes for over 85% of the benchmark cases.

cs.CG↗

arXiv2025

HiLoTs: High-Low Temporal Sensitive Representation Learning for Semi-Supervised LiDAR Segmentation in Autonomous Driving

LiDAR point cloud semantic segmentation plays a crucial role in autonomous driving. In recent years, semi-supervised methods have gained popularity due to their significant reduction in annotation labor and time costs. Current semi-supervised methods typically focus on point cloud spatial distribution or consider short-term temporal representations, e.g., only two adjacent frames, often overlooking the rich long-term temporal properties inherent in autonomous driving scenarios. In driving experience, we observe that nearby objects, such as roads and vehicles, remain stable while driving, whereas distant objects exhibit greater variability in category and shape. This natural phenomenon is also captured by LiDAR, which reflects lower temporal sensitivity for nearby objects and higher sensitivity for distant ones. To leverage these characteristics, we propose HiLoTs, which learns high-temporal sensitivity and low-temporal sensitivity representations from continuous LiDAR frames. These representations are further enhanced and fused using a cross-attention mechanism. Additionally, we employ a teacher-student framework to align the representations learned by the labeled and unlabeled branches, effectively utilizing the large amounts of unlabeled data. Experimental results on the SemanticKITTI and nuScenes datasets demonstrate that our proposed HiLoTs outperforms state-of-the-art semi-supervised methods, and achieves performance close to LiDAR+Camera multimodal approaches. Code is available on https://github.com/rdlin118/HiLoTs

cs.CV↗

arXiv2025

UniHOPE: A Unified Approach for Hand-Only and Hand-Object Pose Estimation

Estimating the 3D pose of hand and potential hand-held object from monocular images is a longstanding challenge. Yet, existing methods are specialized, focusing on either bare-hand or hand interacting with object. No method can flexibly handle both scenarios and their performance degrades when applied to the other scenario. In this paper, we propose UniHOPE, a unified approach for general 3D hand-object pose estimation, flexibly adapting both scenarios. Technically, we design a grasp-aware feature fusion module to integrate hand-object features with an object switcher to dynamically control the hand-object pose estimation according to grasping status. Further, to uplift the robustness of hand pose estimation regardless of object presence, we generate realistic de-occluded image pairs to train the model to learn object-induced hand occlusions, and formulate multi-level feature enhancement techniques for learning occlusion-invariant features. Extensive experiments on three commonly-used benchmarks demonstrate UniHOPE's SOTA performance in addressing hand-only and hand-object scenarios. Code will be released on https://github.com/JoyboyWang/UniHOPE_Pytorch.

cs.CV↗

arXiv2024

HandBooster: Boosting 3D Hand-Mesh Reconstruction by Conditional Synthesis and Sampling of Hand-Object Interactions

Reconstructing 3D hand mesh robustly from a single image is very challenging, due to the lack of diversity in existing real-world datasets. While data synthesis helps relieve the issue, the syn-to-real gap still hinders its usage. In this work, we present HandBooster, a new approach to uplift the data diversity and boost the 3D hand-mesh reconstruction performance by training a conditional generative space on hand-object interactions and purposely sampling the space to synthesize effective data samples. First, we construct versatile content-aware conditions to guide a diffusion model to produce realistic images with diverse hand appearances, poses, views, and backgrounds; favorably, accurate 3D annotations are obtained for free. Then, we design a novel condition creator based on our similarity-aware distribution sampling strategies to deliberately find novel and realistic interaction poses that are distinctive from the training set. Equipped with our method, several baselines can be significantly improved beyond the SOTA on the HO3D and DexYCB benchmarks. Our code will be released on https://github.com/hxwork/HandBooster_Pytorch.

cs.CV↗

arXiv2024

SiMA-Hand: Boosting 3D Hand-Mesh Reconstruction by Single-to-Multi-View Adaptation

Estimating 3D hand mesh from RGB images is a longstanding track, in which occlusion is one of the most challenging problems. Existing attempts towards this task often fail when the occlusion dominates the image space. In this paper, we propose SiMA-Hand, aiming to boost the mesh reconstruction performance by Single-to-Multi-view Adaptation. First, we design a multi-view hand reconstructor to fuse information across multiple views by holistically adopting feature fusion at image, joint, and vertex levels. Then, we introduce a single-view hand reconstructor equipped with SiMA. Though taking only one view as input at inference, the shape and orientation features in the single-view reconstructor can be enriched by learning non-occluded knowledge from the extra views at training, enhancing the reconstruction precision on the occluded regions. We conduct experiments on the Dex-YCB and HanCo benchmarks with challenging object- and self-caused occlusion cases, manifesting that SiMA-Hand consistently achieves superior performance over the state of the arts. Code will be released on https://github.com/JoyboyWang/SiMA-Hand Pytorch.

cs.CV↗

arXiv2023

Distinct elastic properties and their origins in glasses and gels

Glasses and gels, widely encountered amorphous solids with diverse industrial and everyday applications, share intriguing similarities such as rigidity without crystalline order and dynamic slowing down during aging. However, the underlying differences between these two fascinating materials have remained elusive. Here we uncover distinct elastic properties concerning observation and aging times in glasses and gels, while delving into the underlying mechanisms. In glasses, we observe a gradual decrease in the shear modulus, while the bulk modulus remains constant throughout the observation time. In contrast, gels exhibit a decrease in both the shear and bulk moduli over the observation time. Additionally, during aging, glasses exhibit a steady trend of stiffening, while gels demonstrate initial stiffening followed by softening. By unravelling the intricate relationship between structure, dynamics, and elasticity, we attribute these differences to mechanisms that minimize free energy: structural ordering in glasses and interface reduction in gels. Our work not only uncovers the distinct behaviors of glasses and gels but also sheds light on the origin and evolution of elasticity in non-equilibrium disordered solids, offering significant implications for the application and design of amorphous materials.

cond-mat.soft↗

arXiv2023

A review on shear jamming

Jamming is a ubiquitous phenomenon that appears in many soft matter systems, including granular materials, foams, colloidal suspensions, emulsions, polymers, and cells -- when jamming occurs, the system undergoes a transition from flow-like to solid-like states. Conventionally, the jamming transition occurs when the system reaches a threshold jamming density under isotropic compression, but recent studies reveal that jamming can also be induced by shear. Shear jamming has attracted much interest in the context of non-equilibrium phase transitions, mechanics and rheology of amorphous materials. Here we review the phenomenology of shear jamming and its related physics. We first describe basic observations obtained in experiments and simulations, and results from theories. Shear jamming is then demonstrated as a "bridge" that connects the rheology of athermal soft spheres and thermal hard spheres. Based on a generalized jamming phase diagram, a universal description is provided for shear jamming in frictionless and frictional systems. We further review the isostaticity and criticality of the shear jamming transition, and the elasticity of shear jammed solids. The broader relevance of shear jamming is discussed, including its relation to other phenomena such as shear hardening, dilatancy, fragility, and discrete shear thickening.

cond-mat.soft↗

arXiv2023

The yielding of granular matter is marginally stable and critical

The mechanical yield of dense granular materials is a fascinating rheological phenomenon, beyond which stress no longer increases with strain at a sufficiently large deformation. Understanding the behavior of mechanical responses associated with yielding is a fundamental goal in granular physics, and other related fields including glassy physics, material sciences, geophysics, and active matter biophysics. However, despite nearly half a century of theoretical efforts, the nature of yielding in amorphous solids remains largely elusive compared to its crystalline counterpart. Here, we experimentally investigate the mechanical responses of two-dimensional bidisperse jammed disks subjected to volume-invariant pure shear, focusing on the behavior of yielding. We show that the microscopic mechanical and geometrical features of configurations under shear can be characterized by two critical exponents of weak-force and small-gap distributions originally proposed for the isotropic jamming transition. We find that the yielding transition satisfies the condition of marginal mechanical stability through a scaling relationship between the two exponents, and after yielding global instability emerges. The criticality of yielding is revealed by a significant peak of susceptibility that quantifies the fluctuation of a glass overlap order parameter. Moreover, we find a distinct transition before yielding, which is associated with the onset of structural anisotropy.

cond-mat.soft↗

arXiv2022

Experimental evidence of detailed balance in granular systems

The principle of detailed balance (DB) states that every kinetic transition in a system with many micro-states, $μ$, is balanced, on average, with the opposite transition, $μ_i\leftrightharpoonsμ_j$. Since its introduction by Boltzmann, this principle has been used by luminaries, such as Einstein, Eddington, Kramers, Pauli, Ehrenfest, Dirac, Onsager, and many others to derive significant results that underpin much of our scientific understanding. The current belief is that DB is satisfied only in equilibrium systems, while non-equilibrium steady states can only be balanced by cycles, such as $A\to B\to C\to A$. We show here experimentally that DB can exist and is commonly and robustly satisfied in a family of quasi-statically cyclically sheared granular systems. We further study the approach to DB as a function of system size and time. Given the significant impact that this principle has had on equilibrium systems, we believe that this discovery paves the way for better models of the dynamics of non-equilibrium systems.

cond-mat.soft↗

arXiv2022

Experimental observations of marginal criticality in granular materials

Two drastically different theories predict the marginal criticality of jamming. The full replica symmetry breaking (fullRSB) theory [1-4] predicts the power-law distributions of weak contact forces and small inter-particle gaps in infinite-dimensional hard-sphere glass, with two nontrivial exponents $θ_f=0.42311...$ and $γ=0.41269...$, respectively. While the marginal mechanical stability (MMS) analysis [5-8] predicts that the isostatic random packings of hard frictionless spheres under external stress are marginally stable and provides inequality relationships for the exponents of the weak-force and inter-particle-gap distributions. Here we measure precisely contact forces and particle positions in isotropic jammed bidisperse photoelastic disks and find the clear power-law distributions of weak forces and small inter-particle gaps, with both exponents $θ_f=0.44(2)$ and $γ= 0.43(3) $ in an excellent agreement with the fullRSB theory. As the jammed packing subject to area-conserved cyclic pure shear approaches the yielding point, the two exponents change substantially from those of the isotropic case but they still satisfy the scaling relationship provided by the MMS argument. Our results provide strong experimental evidences for the robustness of the infinite-dimensional theory and the MMS analysis in real-world amorphous materials.

cond-mat.soft↗

arXiv2021

Critical behaviors of jamming in cyclically sheared frictional hard granular particles

In stark contrast to the jamming of frictionless hard-sphere (hard-disk if in two dimensions) packings, the critical jamming of frictional packings is much more elusive and still under intense debate. Here we show that frictional hard-disk packings self-organize near a critical jamming state when subjected to quasi-static steady-state cyclic shear, displaying scale-free fluctuations in particle velocity fields as characterized by the power spectra $E(k)\propto k^{-α}$ for both longitudinal and transverse modes in wavevector space, with $α\approx2.0\pm0.15$, and the nearly flat spectra $E(ω)\propto const.$ in angular frequency domain. Our findings agree quantitatively with the predictions of the Langevin-type effective medium theory of S. Henkes and coworkers with two diverging length scales associated respectively with the longitudinal and transverse modes, showing a hallmark of critical behaviors of jamming. Moreover, our findings are consistent with the conceptual framework of general isostaticity but with a key difference that the system self-organizes to a critical jamming state through a strong coupling of mechanical structure and dynamics. Our findings are important in providing microscopic mechanism in understanding the nonlocal rheologies and guide principles in developing constitutive relations of real granular materials.

cond-mat.soft↗