arXiv subjects

Yiming Wang

Publications and source records attributed to Yiming Wang.

At least 145 records · Page 8Linked to original sources

arXiv2025

Zero-Shot Human-Object Interaction Synthesis with Multimodal Priors

Human-object interaction (HOI) synthesis is important for various applications, ranging from virtual reality to robotics. However, acquiring 3D HOI data is challenging due to its complexity and high cost, limiting existing methods to the narrow diversity of object types and interaction patterns in training datasets. This paper proposes a novel zero-shot HOI synthesis framework without relying on end-to-end training on currently limited 3D HOI datasets. The core idea of our method lies in leveraging extensive HOI knowledge from pre-trained Multimodal Models. Given a text description, our system first obtains temporally consistent 2D HOI image sequences using image or video generation models, which are then uplifted to 3D HOI milestones of human and object poses. We employ pre-trained human pose estimation models to extract human poses and introduce a generalizable category-level 6-DoF estimation method to obtain the object poses from 2D HOI images. Our estimation method is adaptive to various object templates obtained from text-to-3D models or online retrieval. A physics-based tracking of the 3D HOI kinematic milestone is further applied to refine both body motions and object poses, yielding more physically plausible HOI generation results. The experimental results demonstrate that our method is capable of generating open-vocabulary HOIs with physical realism and semantic diversity.

cs.GR↗

arXiv2025

Can Text-to-Video Generation help Video-Language Alignment?

Recent video-language alignment models are trained on sets of videos, each with an associated positive caption and a negative caption generated by large language models. A problem with this procedure is that negative captions may introduce linguistic biases, i.e., concepts are seen only as negatives and never associated with a video. While a solution would be to collect videos for the negative captions, existing databases lack the fine-grained variations needed to cover all possible negatives. In this work, we study whether synthetic videos can help to overcome this issue. Our preliminary analysis with multiple generators shows that, while promising on some tasks, synthetic videos harm the performance of the model on others. We hypothesize this issue is linked to noise (semantic and visual) in the generated videos and develop a method, SynViTA, that accounts for those. SynViTA dynamically weights the contribution of each synthetic video based on how similar its target caption is w.r.t. the real counterpart. Moreover, a semantic consistency loss makes the model focus on fine-grained differences across captions, rather than differences in video appearance. Experiments show that, on average, SynViTA improves over existing methods on VideoCon test sets and SSv2-Temporal, SSv2-Events, and ATP-Hard benchmarks, being a first promising step for using synthetic videos when learning video-language models.

cs.CV↗

arXiv2025

Numerical simulation of wormhole propagation with the mixed hybridized discontinuous Galerkin finite element method

The acid treatment of carbonate reservoirs is a widely employed technique for enhancing the productivity of oil and gas reservoirs. In this paper, we present a novel combined hybridized mixed discontinuous Galerkin (HMDG) finite element method to simulate the dissolution process near the wellbore, commonly referred to as the wormhole phenomenon. The primary contribution of this work lies in the application of hybridization techniques to both the pressure and concentration equations. Additionally, an upwind scheme is utilized to address convection-dominant scenarios, and a ``cut-off" operator is introduced to maintain the boundedness of porosity. Compared to traditional discontinuous Galerkin methods, the proposed approach results in a global system with fewer unknowns and sparser stencils, thereby significantly reducing computational costs. We analyze the existence and uniqueness of the new combined method and derive optimal error estimates using the developed technique. Numerical examples are provided to validate the theoretical analysis.

math.NA↗

arXiv2025

One VLM to Keep it Learning: Generation and Balancing for Data-free Continual Visual Question Answering

Vision-Language Models (VLMs) have shown significant promise in Visual Question Answering (VQA) tasks by leveraging web-scale multimodal datasets. However, these models often struggle with continual learning due to catastrophic forgetting when adapting to new tasks. As an effective remedy to mitigate catastrophic forgetting, rehearsal strategy uses the data of past tasks upon learning new task. However, such strategy incurs the need of storing past data, which might not be feasible due to hardware constraints or privacy concerns. In this work, we propose the first data-free method that leverages the language generation capability of a VLM, instead of relying on external models, to produce pseudo-rehearsal data for addressing continual VQA. Our proposal, named as GaB, generates pseudo-rehearsal data by posing previous task questions on new task data. Yet, despite being effective, the distribution of generated questions skews towards the most frequently posed questions due to the limited and task-specific training data. To mitigate this issue, we introduce a pseudo-rehearsal balancing module that aligns the generated data towards the ground-truth data distribution using either the question meta-statistics or an unsupervised clustering method. We evaluate our proposed method on two recent benchmarks, \ie VQACL-VQAv2 and CLOVE-function benchmarks. GaB outperforms all the data-free baselines with substantial improvement in maintaining VQA performance across evolving tasks, while being on-par with methods with access to the past data.

cs.CV↗

arXiv2025

Collaborative Instance Object Navigation: Leveraging Uncertainty-Awareness to Minimize Human-Agent Dialogues

Language-driven instance object navigation assumes that human users initiate the task by providing a detailed description of the target instance to the embodied agent. While this description is crucial for distinguishing the target from visually similar instances in a scene, providing it prior to navigation can be demanding for human. To bridge this gap, we introduce Collaborative Instance object Navigation (CoIN), a new task setting where the agent actively resolve uncertainties about the target instance during navigation in natural, template-free, open-ended dialogues with human. We propose a novel training-free method, Agent-user Interaction with UncerTainty Awareness (AIUTA), which operates independently from the navigation policy, and focuses on the human-agent interaction reasoning with Vision-Language Models (VLMs) and Large Language Models (LLMs). First, upon object detection, a Self-Questioner model initiates a self-dialogue within the agent to obtain a complete and accurate observation description with a novel uncertainty estimation technique. Then, an Interaction Trigger module determines whether to ask a question to the human, continue or halt navigation, minimizing user input. For evaluation, we introduce CoIN-Bench, with a curated dataset designed for challenging multi-instance scenarios. CoIN-Bench supports both online evaluation with humans and reproducible experiments with simulated user-agent interactions. On CoIN-Bench, we show that AIUTA serves as a competitive baseline, while existing language-driven instance navigation methods struggle in complex multi-instance scenes. Code and benchmark will be available upon acceptance at https://intelligolabs.github.io/CoIN/

cs.AI↗

arXiv2025

Penalized Principal Component Analysis for Large-dimension Factor Model with Group Pursuit

This paper investigates the intrinsic group structures within the framework of large-dimensional approximate factor models, which portrays homogeneous effects of the common factors on the individuals that fall into the same group. To this end, we propose a fusion Penalized Principal Component Analysis (PPCA) method and derive a closed-form solution for the $\ell_2$-norm optimization problem. We also show the asymptotic properties of our proposed PPCA estimates. With the PPCA estimates as an initialization, we identify the unknown group structure by a combination of the agglomerative hierarchical clustering algorithm and an information criterion. Then the factor loadings and factor scores are re-estimated conditional on the identified latent groups. Under some regularity conditions, we establish the consistency of the membership estimators as well as that of the group number estimator derived from the information criterion. Theoretically, we show that the post-clustering estimators for the factor loadings and factor scores with group pursuit achieve efficiency gains compared to the estimators by conventional PCA method. Thorough numerical studies validate the established theory and a real financial example illustrates the practical usefulness of the proposed method.

stat.ME↗

arXiv2025

Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation

LLM self-evaluation relies on the LLM's own ability to estimate response correctness, which can greatly improve its deployment reliability. In this research track, we propose the Chain-of-Embedding (CoE) in the latent space to enable LLMs to perform output-free self-evaluation. CoE consists of all progressive hidden states produced during the inference time, which can be treated as the latent thinking path of LLMs. We find that when LLMs respond correctly and incorrectly, their CoE features differ, these discrepancies assist us in estimating LLM response correctness. Experiments in four diverse domains and seven LLMs fully demonstrate the effectiveness of our method. Meanwhile, its label-free design intent without any training and millisecond-level computational cost ensures real-time feedback in large-scale scenarios. More importantly, we provide interesting insights into LLM response correctness from the perspective of hidden state changes inside LLMs.

cs.CL↗

arXiv2025

ARCHED: A Human-Centered Framework for Transparent, Responsible, and Collaborative AI-Assisted Instructional Design

Integrating Large Language Models (LLMs) in educational technology presents unprecedented opportunities to improve instructional design (ID), yet existing approaches often prioritize automation over pedagogical rigor and human agency. This paper introduces ARCHED (AI for Responsible, Collaborative, Human-centered Education Instructional Design), a structured multi-stage framework that ensures human educators remain central in the design process while leveraging AI capabilities. Unlike traditional AI-generated instructional materials that lack transparency, ARCHED employs a cascaded workflow aligned with Bloom's taxonomy. The framework integrates specialized AI agents - one generating diverse pedagogical options and another evaluating alignment with learning objectives - while maintaining educators as primary decision-makers. This approach addresses key limitations in current AI-assisted instructional design, ensuring transparency, pedagogical foundation, and meaningful human agency. Empirical evaluations demonstrate that ARCHED enhances instructional design quality while preserving educator oversight, marking a step forward in responsible AI integration in education.

cs.CY↗

arXiv2025

Amplified multipartite entanglement witnessed in a quantum critical metal

Strong correlations in matter promote a landscape of quantum phases and associated quantum critical points. For metallic systems, there is increasing recognition that the quantum criticality goes beyond the Landau framework and, thus, novel means are needed to characterize the quantum critical fluid. Here we do so by studying an entanglement quantity, the quantum Fisher information, in a strange metal system, focusing on the exemplary case of an Anderson/Kondo lattice model near its Kondo destruction quantum critical point. The spin quantum Fisher information peaks at the quantum critical point and indicates a strongly entangled ground state. Our results are supported by the quantum Fisher information extracted from inelastic neutron scattering measurements in heavy fermion metals. Our work elucidates the loss of quasiparticles in strange metals, opens a quantum information avenue to advance the understanding of metallic quantum criticality in a broad range of strongly correlated systems, and points to a novel regime of quantum matter to realize amplified entanglement.

cond-mat.str-el↗

arXiv2025

SplatFormer: Point Transformer for Robust 3D Gaussian Splatting

3D Gaussian Splatting (3DGS) has recently transformed photorealistic reconstruction, achieving high visual fidelity and real-time performance. However, rendering quality significantly deteriorates when test views deviate from the camera angles used during training, posing a major challenge for applications in immersive free-viewpoint rendering and navigation. In this work, we conduct a comprehensive evaluation of 3DGS and related novel view synthesis methods under out-of-distribution (OOD) test camera scenarios. By creating diverse test cases with synthetic and real-world datasets, we demonstrate that most existing methods, including those incorporating various regularization techniques and data-driven priors, struggle to generalize effectively to OOD views. To address this limitation, we introduce SplatFormer, the first point transformer model specifically designed to operate on Gaussian splats. SplatFormer takes as input an initial 3DGS set optimized under limited training views and refines it in a single forward pass, effectively removing potential artifacts in OOD test views. To our knowledge, this is the first successful application of point transformers directly on 3DGS sets, surpassing the limitations of previous multi-scene training methods, which could handle only a restricted number of input views during inference. Our model significantly improves rendering quality under extreme novel views, achieving state-of-the-art performance in these challenging scenarios and outperforming various 3DGS regularization techniques, multi-scene models tailored for sparse view synthesis, and diffusion-based frameworks.

cs.CV↗

arXiv2025

Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs

We introduce Phi-4-Mini and Phi-4-Multimodal, compact yet highly capable language and multimodal models. Phi-4-Mini is a 3.8-billion-parameter language model trained on high-quality web and synthetic data, significantly outperforming recent open-source models of similar size and matching the performance of models twice its size on math and coding tasks requiring complex reasoning. This achievement is driven by a carefully curated synthetic data recipe emphasizing high-quality math and coding datasets. Compared to its predecessor, Phi-3.5-Mini, Phi-4-Mini features an expanded vocabulary size of 200K tokens to better support multilingual applications, as well as group query attention for more efficient long-sequence generation. Phi-4-Multimodal is a multimodal model that integrates text, vision, and speech/audio input modalities into a single model. Its novel modality extension approach leverages LoRA adapters and modality-specific routers to allow multiple inference modes combining various modalities without interference. For example, it now ranks first in the OpenASR leaderboard to date, although the LoRA component of the speech/audio modality has just 460 million parameters. Phi-4-Multimodal supports scenarios involving (vision + language), (vision + speech), and (speech/audio) inputs, outperforming larger vision-language and speech-language models on a wide range of tasks. Additionally, we experiment to further train Phi-4-Mini to enhance its reasoning capabilities. Despite its compact 3.8-billion-parameter size, this experimental version achieves reasoning performance on par with or surpassing significantly larger models, including DeepSeek-R1-Distill-Qwen-7B and DeepSeek-R1-Distill-Llama-8B.

cs.CL↗

arXiv2025

SEAL: Safety Enhanced Trajectory Planning and Control Framework for Quadrotor Flight in Complex Environments

For quadrotors, achieving safe and autonomous flight in complex environments with wind disturbances and dynamic obstacles still faces significant challenges. Most existing methods address wind disturbances in either trajectory planning or control, which may lead to hazardous situations during flight. The emergence of dynamic obstacles would further worsen the situation. Therefore, we propose an efficient and reliable framework for quadrotors that incorporates wind disturbance estimations during both the planning and control phases via a generalized proportional integral observer. First, we develop a real-time adaptive spatial-temporal trajectory planner that utilizes Hamilton-Jacobi (HJ) reachability analysis for error dynamics resulting from wind disturbances. By considering the forward reachability sets propagation on an Euclidean Signed Distance Field (ESDF) map, safety is guaranteed. Additionally, a Nonlinear Model Predictive Control (NMPC) controller considering wind disturbance compensation is implemented for robust trajectory tracking. Simulation and real-world experiments verify the effectiveness of our framework. The video and supplementary material will be available at https://github.com/Ma29-HIT/SEAL/.

eess.SY↗

arXiv2025

Normalization through Fine-tuning: Understanding Wav2vec 2.0 Embeddings for Phonetic Analysis

Phonetic normalization plays a crucial role in speech recognition and analysis, ensuring the comparability of features derived from raw audio data. However, in the current paradigm of fine-tuning pre-trained large transformer models, phonetic normalization is not deemed a necessary step; instead, it is implicitly executed within the models. This study investigates the normalization process within transformer models, especially wav2vec 2.0. Through a comprehensive analysis of embeddings from models fine-tuned for various tasks, our results demonstrate that fine-tuning wav2vec 2.0 effectively achieves phonetic normalization by selectively suppressing task-irrelevant information. We found that models fine-tuned for multiple tasks retain information for both tasks without compromising performance, and that suppressing task-irrelevant information is not necessary for effective classification. These findings provide new insights into how phonetic normalization can be flexibly achieved in speech models and how it is realized in human speech perception.

cs.CL↗

arXiv2025

ForceMimic: Force-Centric Imitation Learning with Force-Motion Capture System for Contact-Rich Manipulation

In most contact-rich manipulation tasks, humans apply time-varying forces to the target object, compensating for inaccuracies in the vision-guided hand trajectory. However, current robot learning algorithms primarily focus on trajectory-based policy, with limited attention given to learning force-related skills. To address this limitation, we introduce ForceMimic, a force-centric robot learning system, providing a natural, force-aware and robot-free robotic demonstration collection system, along with a hybrid force-motion imitation learning algorithm for robust contact-rich manipulation. Using the proposed ForceCapture system, an operator can peel a zucchini in 5 minutes, while force-feedback teleoperation takes over 13 minutes and struggles with task completion. With the collected data, we propose HybridIL to train a force-centric imitation learning model, equipped with hybrid force-position control primitive to fit the predicted wrench-position parameters during robot execution. Experiments demonstrate that our approach enables the model to learn a more robust policy under the contact-rich task of vegetable peeling, increasing the success rates by 54.5% relatively compared to state-ofthe-art pure-vision-based imitation learning. Hardware, code, data and more results can be found on the project website at https://forcemimic.github.io.

cs.RO↗

arXiv2025

Do Large Language Models Truly Understand Geometric Structures?

Geometric ability is a significant challenge for large language models (LLMs) due to the need for advanced spatial comprehension and abstract thinking. Existing datasets primarily evaluate LLMs on their final answers, but they cannot truly measure their true understanding of geometric structures, as LLMs can arrive at correct answers by coincidence. To fill this gap, we introduce the GeomRel dataset, designed to evaluate LLMs' understanding of geometric structures by isolating the core step of geometric relationship identification in problem-solving. Using this benchmark, we conduct thorough evaluations of diverse LLMs and identify key limitations in understanding geometric structures. We further propose the Geometry Chain-of-Thought (GeoCoT) method, which enhances LLMs' ability to identify geometric relationships, resulting in significant performance improvements.

cs.CL↗

arXiv2025

Local and Non-local Entanglement Witnesses of Fermi Liquid

There is a growing interest both in utilizing entanglement means to characterize many-body systems and in uncovering their entanglement depth. Motivated by recent findings that the spin quantum Fisher information witnesses amplified multipartite entanglement of strange metals and characterizes their loss of quasiparticles, we study the quantum Fisher information in various cases of Fermi liquid. We show that local operators generically do not witness any multipartite entanglement in a Fermi liquid, but non-local many-body operators do. Our results point to novel experimental means to detect the entanglement depth of metallic fermionic systems and, in general, open a new avenue to the emerging exploration of entanglement in quantum materials.

cond-mat.str-el↗

arXiv2025

Orbital Signatures of Density Wave Transition in La3Ni2O7-delta and La2PrNi2O7-delta RP-Nickelates Probed via in-situ X-ray Absorption Near-edge Spectroscopy

The report of superconductivity (SC) with Tc~80 K in bilayer Ruddlesden-Popper (RP) nickelate La3Ni2O7-delta have sparked considerable investigations on its normal state properties and SC mechanism under pressure and at low temperature. It is believed that the density wave (DW) at ~150 K plays an important role in SC emergence, but its nature remains largely underexplored. Here, we utilized temperature-dependent in-situ Ni K-edge X-ray Absorption Near-edge Spectroscopy (XANES) to probe the Ni-3d/4p electronic states of La3Ni2O7-delta and La2PrNi2O7-delta samples down to 4.8 K, enabling us to witness the evolution of both in-plane d_(x^2-y^2)/p_x (p_y) and out-of-plane d_(3z^2-r^2)/p_z orbitals of NiO6 octahedron across the DW transition. Main edge energy associated with Ni 4p orbital shows an anomalous decline near DW transition, signifying the occurrence of lattice distortions as a hallmark of charge density wave. Below DW transition, the enlarged crystal field splitting (CFS) indicates an enhanced NiO6 octahedral distortion. Intriguingly, magnetic Pr substituents could activate the mutual interplay of d_(x^2-y^2) and d_(3z^2-r^2) orbitals. We discussed its relevance to the favored bulk SC in the pressurized polycrystalline La2PrNi2O7-delta than pristine.

cond-mat.supr-con↗

arXiv2025

Orbital-selective correlation effects and superconducting pairing symmetry in a multiorbital $t$-$J$ model for bilayer nickelates

The recent discovery of superconductivity in La$_3$Ni$_2$O$_7$ raises key questions about its mechanism and the nature of pairing symmetry. This system is believed to be described by a bilayer two-orbital Hubbard model. The considerations of orbital-selective Mott correlations motivate a bilayer two-orbital $t$-$J$ model and, accordingly, we study the superconducting pairing in this model. We obtain an overall phase diagram of superconductivity, where the leading channel has either extended $s$-wave or $d_{x^2-y^2}$-wave symmetry. Our analysis highlights how the orbital-selective correlations affect the superconducting pairing via the interlayer exchange couplings and low-energy electronic structure. In particular, we find that the dominant orbital for the pairing may change between $z^2$ and $x^2-y^2$ when the position of the bonding $z^2$ band is varied by tuning either the $c$-axis lattice constant or electron concentration strength. We discuss the implications of these results for the superconductivity in both bulk La$_{3}$Ni$_{2}$O$_{7}$ and its thin film counterpart.

cond-mat.supr-con↗