arXiv subjects

Xun Yang

Publications and source records attributed to Xun Yang.

At least 55 records · Page 3Linked to original sources

arXiv2025

EverybodyDance: Bipartite Graph-Based Identity Correspondence for Multi-Character Animation

Consistent pose-driven character animation has achieved remarkable progress in single-character scenarios. However, extending these advances to multi-character settings is non-trivial, especially when position swap is involved. Beyond mere scaling, the core challenge lies in enforcing correct Identity Correspondence (IC) between characters in reference and generated frames. To address this, we introduce EverybodyDance, a systematic solution targeting IC correctness in multi-character animation. EverybodyDance is built around the Identity Matching Graph (IMG), which models characters in the generated and reference frames as two node sets in a weighted complete bipartite graph. Edge weights, computed via our proposed Mask-Query Attention (MQA), quantify the affinity between each pair of characters. Our key insight is to formalize IC correctness as a graph structural metric and to optimize it during training. We also propose a series of targeted strategies tailored for multi-character animation, including identity-embedded guidance, a multi-scale matching strategy, and pre-classified sampling, which work synergistically. Finally, to evaluate IC performance, we curate the Identity Correspondence Evaluation benchmark, dedicated to multi-character IC correctness. Extensive experiments demonstrate that EverybodyDance substantially outperforms state-of-the-art baselines in both IC and visual fidelity.

cs.CV↗

arXiv2025

Mock Observations for the CSST Mission: Multi-Channel Imager--Instrument Simulation

The Chinese Space Station Survey Telescope (CSST), a two-meter aperture astronomical space telescope under China's manned space program, is equipped with multiple back-end scientific instruments. As an astronomical precision measurement module of the CSST, the Multi-Channel Imager (MCI) can cover a wide wavelength range from ultraviolet to near-infrared with three-color simultaneous high-precision photometry and imaging, which meets the scientific requirements for various fields. The diverse scientific objectives of MCI require not only a robust airborne platform, advanced optical systems, and observing facilities but also comprehensive software support for scientific operations and research. To this end, it is essential to develop realistic observational simulation software to thoroughly evaluate the MCI data stream and provide calibration tools for future scientific investigations. The MCI instrument simulation software will serve as a foundation for the development of the MCI data processing pipeline and will facilitate improvements in both hardware and software, as well as in the observational operation strategy, in alignment with the mission's scientific goals. In conclusion, we present a comprehensive overview of the MCI instrument simulation and some corresponding performances of the MCI data processing pipeline.

astro-ph.IM↗

arXiv2025

Mock Observations for the CSST Mission: Integral Field Spectrograph--Instrument Simulation

The Chinese Space Station Survey Telescope (CSST) is a next-generation Stage-IV facility renowned for its wide field of view, high image quality, and multi-band observational capabilities. Among the five instruments onboard the CSST, the Integral Field Spectrograph (IFS) offers the unique ability to simultaneously capture spatial and spectral information across a field of view of no less than $6^{''}\times6^{''}$. Key advantages of the IFS include a high spatial resolution of $0.2^{''}$ and a broad spectral coverage from 350 to 1000 nm, making it an ideal instrument for studying physical processes in the vicinity of supermassive black holes within galaxies. To more accurately assess the technical and scientific performance of the CSST-IFS, it is essential to develop a simulation tool that incorporates realistic effects from all optical components. Such a simulation will form an integral part of the CSST-IFS data and pipeline system, enabling the development of the data reduction pipeline well ahead of actual observations. This paper presents an end-to-end simulation workflow for the CSST-IFS, incorporating a wide range of instrumental effects that may influence its spectral and imaging performance. The simulation accounts for optical diffraction effects introduced by all components, such as image slicers and slit array, as well as sub-pixel effects from gratings. It also includes various detector noises, frame-shifting effects, and charge-transfer inefficiency. Real observational conditions--such as target Doppler shift, cosmic rays, and other in-orbit operational effects--are also considered. We describe the technical implementation of the simulation and present results that quantitatively characterize key instrument parameters.

astro-ph.IM↗

arXiv2025

AffordBot: 3D Fine-grained Embodied Reasoning via Multimodal Large Language Models

Effective human-agent collaboration in physical environments requires understanding not only what to act upon, but also where the actionable elements are and how to interact with them. Existing approaches often operate at the object level or disjointedly handle fine-grained affordance reasoning, lacking coherent, instruction-driven grounding and reasoning. In this work, we introduce a new task: Fine-grained 3D Embodied Reasoning, which requires an agent to predict, for each referenced affordance element in a 3D scene, a structured triplet comprising its spatial location, motion type, and motion axis, based on a task instruction. To solve this task, we propose AffordBot, a novel framework that integrates Multimodal Large Language Models (MLLMs) with a tailored chain-of-thought (CoT) reasoning paradigm. To bridge the gap between 3D input and 2D-compatible MLLMs, we render surround-view images of the scene and project 3D element candidates into these views, forming a rich visual representation aligned with the scene geometry. Our CoT pipeline begins with an active perception stage, prompting the MLLM to select the most informative viewpoint based on the instruction, before proceeding with step-by-step reasoning to localize affordance elements and infer plausible interaction motions. Evaluated on the SceneFun3D dataset, AffordBot achieves state-of-the-art performance, demonstrating strong generalization and physically grounded reasoning with only 3D point cloud input and MLLMs.

cs.CV↗

arXiv2025

Mock Observations for the CSST Mission: End-to-End Performance Modeling of Optical System

This study presents a comprehensive end-to-end simulation analysis of the optical imaging performance of the China Survey Space Telescope (CSST) under in-orbit conditions. An integrated system model incorporating five static and two dynamic error sub-models was established. Wavefront errors were calculated for each sub-model and compared to the integrated system error to quantify the individual contributions to image degradation. At the detector level, wavefront error, point spread function (PSF), and ellipticity were evaluated across the full field of view (FOV). The average radius of 80\% encircled energy (REE80) of the PSF under full-error conditions was determined for 25 field points, yielding a value of 0.114 arcseconds. Furthermore, the calculations indicate a correlation between the wavefront distribution and the ellipticity distribution within the optical system. By optimizing the wavefront distribution, it is possible to adjust the ellipticity distribution of the PSF across the full FOV. The end-to-end simulation approach adopted in this paper provides a theoretical foundation for improving the image quality in large-aperture, off-axis space telescopes.

astro-ph.IM↗

arXiv2025

EgoBlind: Towards Egocentric Visual Assistance for the Blind

We present EgoBlind, the first egocentric VideoQA dataset collected from blind individuals to evaluate the assistive capabilities of contemporary multimodal large language models (MLLMs). EgoBlind comprises 1,392 first-person videos from the daily lives of blind and visually impaired individuals. It also features 5,311 questions directly posed or verified by the blind to reflect their in-situation needs for visual assistance. Each question has an average of 3 manually annotated reference answers to reduce subjectiveness. Using EgoBlind, we comprehensively evaluate 16 advanced MLLMs and find that all models struggle. The best performers achieve an accuracy near 60\%, which is far behind human performance of 87.4\%. To guide future advancements, we identify and summarize major limitations of existing MLLMs in egocentric visual assistance for the blind and explore heuristic solutions for improvement. With these efforts, we hope that EgoBlind will serve as a foundation for developing effective AI assistants to enhance the independence of the blind and visually impaired. Data and code are available at https://github.com/doc-doc/EgoBlind.

cs.CV↗

arXiv2025

Audio Does Matter: Importance-Aware Multi-Granularity Fusion for Video Moment Retrieval

Video Moment Retrieval (VMR) aims to retrieve a specific moment semantically related to the given query. To tackle this task, most existing VMR methods solely focus on the visual and textual modalities while neglecting the complementary but important audio modality. Although a few recent works try to tackle the joint audio-vision-text reasoning, they treat all modalities equally and simply embed them without fine-grained interaction for moment retrieval. These designs are counter-practical as: Not all audios are helpful for video moment retrieval, and the audio of some videos may be complete noise or background sound that is meaningless to the moment determination. To this end, we propose a novel Importance-aware Multi-Granularity fusion model (IMG), which learns to dynamically and selectively aggregate the audio-vision-text contexts for VMR. Specifically, after integrating the textual guidance with vision and audio separately, we first design a pseudo-label-supervised audio importance predictor that predicts the importance score of the audio, and accordingly assigns weights to mitigate the interference caused by noisy audio. Then, we design a multi-granularity audio fusion module that adaptively fuses audio and visual modalities at local-, event-, and global-level, fully capturing their complementary contexts. We further propose a cross-modal knowledge distillation strategy to address the challenge of missing audio modality during inference. To evaluate our method, we further construct a new VMR dataset, i.e., Charades-AudioMatter, where audio-related samples are manually selected and re-organized from the original Charades-STA to validate the model's capability in utilizing audio modality. Extensive experiments validate the effectiveness of our method, achieving state-of-the-art with audio-video fusion in VMR methods. Our code is available at https://github.com/HuiGuanLab/IMG.

cs.IR↗

arXiv2025

Dual Learning with Dynamic Knowledge Distillation and Soft Alignment for Partially Relevant Video Retrieval

Almost all previous text-to-video retrieval works ideally assume that videos are pre-trimmed with short durations containing solely text-related content. However, in practice, videos are typically untrimmed in long durations with much more complicated background content. Therefore, in this paper, we focus on the more practical yet challenging task of Partially Relevant Video Retrieval (PRVR), which aims to retrieve partially relevant untrimmed videos with the given query. To tackle this task, we propose a novel framework that distills generalization knowledge from a powerful large-scale vision-language pre-trained model and transfers it to a lightweight, task-specific PRVR network. Specifically, we introduce a Dual Learning framework with Dynamic Knowledge Distillation (DL-DKD++), where a large teacher model provides supervision to a compact dual-branch student network. The student model comprises two branches: an inheritance branch that absorbs transferable knowledge from the teacher, and an exploration branch that learns task-specific information from the PRVR dataset to address domain gaps. To further enhance learning, we incorporate a dynamic soft-target construction mechanism. By replacing rigid hard-target supervision with adaptive soft targets that evolve during training, our method enables the model to better capture the fine-grained, partial relevance between videos and queries. Experiment results demonstrate that our proposed model achieves state-of-the-art performance on TVR, ActivityNet, and Charades-STA datasets for PRVR. The code is available at https://github.com/HuiGuanLab/DL-DKD.

cs.CV↗

arXiv2025

Benchmarking and Bridging Emotion Conflicts for Multimodal Emotion Reasoning

Despite their strong performance in multimodal emotion reasoning, existing Multimodal Large Language Models (MLLMs) often overlook the scenarios involving emotion conflicts, where emotional cues from different modalities are inconsistent. To fill this gap, we first introduce CA-MER, a new benchmark designed to examine MLLMs under realistic emotion conflicts. It consists of three subsets: video-aligned, audio-aligned, and consistent, where only one or all modalities reflect the true emotion. However, evaluations on our CA-MER reveal that current state-of-the-art emotion MLLMs systematically over-rely on audio signal during emotion conflicts, neglecting critical cues from visual modality. To mitigate this bias, we propose MoSEAR, a parameter-efficient framework that promotes balanced modality integration. MoSEAR consists of two modules: (1)MoSE, modality-specific experts with a regularized gating mechanism that reduces modality bias in the fine-tuning heads; and (2)AR, an attention reallocation mechanism that rebalances modality contributions in frozen backbones during inference. Our framework offers two key advantages: it mitigates emotion conflicts and improves performance on consistent samples-without incurring a trade-off between audio and visual modalities. Experiments on multiple benchmarks-including MER2023, EMER, DFEW, and our CA-MER-demonstrate that MoSEAR achieves state-of-the-art performance, particularly under modality conflict conditions.

cs.AI↗

arXiv2025

PRVR: Partially Relevant Video Retrieval

In current text-to-video retrieval (T2VR), videos to be retrieved have been properly trimmed so that a correspondence between the videos and ad-hoc textual queries naturally exists. Note in practice that videos circulated on the Internet and social media platforms, while being relatively short, are typically rich in their content. Often, multiple scenes / actions / events are shown in a single video, leading to a more challenging T2VR setting wherein only part of the video content is relevant w.r.t. a given query. This paper presents a first study on this setting which we term Partially Relevant Video Retrieval (PRVR). Considering that a video typically consists of multiple moments, a video is regarded as partially relevant w.r.t. to a given query if it contains a query-related moment. We formulate the PRVR task as a multiple instance learning problem, and propose a Multi-Scale Similarity Learning (MS-SL++) network that jointly learns both clip-scale and frame-scale similarities to determine the partial relevance between video-query pairs. Extensive experiments on three diverse video-text datasets (TVshow Retrieval, ActivityNet-Captions and Charades-STA) demonstrate the viability of the proposed method.

cs.CV↗

arXiv2025

Fiber-integrated NV Magnetometer with Microcontroller-based Software Lock-in Technique

Fiber-integrated nitrogen-vacancy (NV) magnetometers possess high sensitivity, integration, and flexibility, and thus have been explored extensively for industrial applications. While most studies have focused on the optimization of the quantum sensing head, less attention has been paid to the frequently employed professional, expensive, and bulky electronics, which hinder their practical applications. In this article, we fabricate a fiber-integrated NV magnetometer and develop a low-cost microcontroller-based software lock-in technique. In this technique, a microcontroller coordinates efficiently a microwave source chip and an analog-to-digital converter, and a program mimicking the lock-in mechanism realizes microwave frequency-modulated optically detected magnetic resonance of NV centers. As a result, with our setup and technique, we have realized the detection of weak magnetic field with a sensitivity of 93 nT/Hz^{1/2}, which is comparable to what obtained with bulky and professional devices. Furthermore, we demonstrated real-time magnetic field detection, achieving a standard deviation of 488 nT. Our work provides a novel and cost-effective technique for electronic miniaturization, thereby potentially accelerating the industrial application of NV magnetometers.

quant-ph↗

arXiv2025

Fading to Grow: Growing Preference Ratios via Preference Fading Discrete Diffusion for Recommendation

Recommenders aim to rank items from a discrete item corpus in line with user interests, yet suffer from extremely sparse user preference data. Recent advances in diffusion models have inspired diffusion-based recommenders, which alleviate sparsity by injecting noise during a forward process to prevent the collapse of perturbed preference distributions. However, current diffusion-based recommenders predominantly rely on continuous Gaussian noise, which is intrinsically mismatched with the discrete nature of user preference data in recommendation. In this paper, building upon recent advances in discrete diffusion, we propose PreferGrow, a discrete diffusion-based recommender system that models preference ratios by fading and growing user preferences over the discrete item corpus. PreferGrow differs from existing diffusion-based recommenders in three core aspects: (1) Discrete modeling of preference ratios: PreferGrow models relative preference ratios between item pairs, rather than operating in the item representation or raw score simplex. This formulation aligns naturally with the discrete and ranking-oriented nature of recommendation tasks. (2) Perturbing via preference fading: Instead of injecting continuous noise, PreferGrow fades user preferences by replacing the preferred item with alternatives -- physically akin to negative sampling -- thereby eliminating the need for any prior noise assumption. (3) Preference reconstruction via growing: PreferGrow reconstructs user preferences by iteratively growing the preference signals from the estimated ratios. PreferGrow offers a well-defined matrix-based formulation with theoretical guarantees on Markovianity and reversibility, and it demonstrates consistent performance gains over state-of-the-art diffusion-based recommenders across five benchmark datasets, highlighting both its theoretical soundness and empirical effectiveness.

cs.IR↗

arXiv2025

Long-exposure Camera Readout for Coherent Control of Nitrogen-Vacancy Center Spins in Diamond

Camera-based wide-field quantum noise spectroscopy (QNS) with nitrogen-vacancy (NV) center spins in diamond can be used to extract DC and AC magnetic field noise with sub-micrometer spatial resolution, but its realization is currently limited by the conflict between the optimal optical spin readout time (normally below one microsecond) and the minimal camera exposure time (normally tens of microsecond). In the present article, we demonstrate fully coherent control of NV center spins via Rabi oscillation, Ramsey, Hahn echo and relaxometry experiments, i.e. the core of QNS, with a home-built camera-based setup, and achieve an unexpectedly high contrast of 12% with an optical spin readout time of 500 microseconds. We explain the absence of the above conflict in our system with the optical spin readout mechanism under weak laser illumination, where spin-lattice relaxation and weak optical pumping result in a rather slow reduction of contrast with increasing integration time. The revealed mechanism is instructive for the construction of the wide-field QNS with a sCMOS or EMCCD camera, and its application in the studies of magnetic material or superconducting material.

quant-ph↗

arXiv2025

Enhancing Partially Relevant Video Retrieval with Robust Alignment Learning

Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos partially relevant to a given query. The core challenge lies in learning robust query-video alignment against spurious semantic correlations arising from inherent data uncertainty: 1) query ambiguity, where the query incompletely characterizes the target video and often contains uninformative tokens, and 2) partial video relevance, where abundant query-irrelevant segments introduce contextual noise in cross-modal alignment. Existing methods often focus on enhancing multi-scale clip representations and retrieving the most relevant clip. However, the inherent data uncertainty in PRVR renders them vulnerable to distractor videos with spurious similarities, leading to suboptimal performance. To fill this research gap, we propose Robust Alignment Learning (RAL) framework, which explicitly models the uncertainty in data. Key innovations include: 1) we pioneer probabilistic modeling for PRVR by encoding videos and queries as multivariate Gaussian distributions. This not only quantifies data uncertainty but also enables proxy-level matching to capture the variability in cross-modal correspondences; 2) we consider the heterogeneous informativeness of query words and introduce learnable confidence gates to dynamically weight similarity. As a plug-and-play solution, RAL can be seamlessly integrated into the existing architectures. Extensive experiments across diverse retrieval backbones demonstrate its effectiveness.

cs.CV↗

arXiv2025

Omni-Perception: Omnidirectional Collision Avoidance for Legged Locomotion in Dynamic Environments

Agile locomotion in complex 3D environments requires robust spatial awareness to safely avoid diverse obstacles such as aerial clutter, uneven terrain, and dynamic agents. Depth-based perception approaches often struggle with sensor noise, lighting variability, computational overhead from intermediate representations (e.g., elevation maps), and difficulties with non-planar obstacles, limiting performance in unstructured environments. In contrast, direct integration of LiDAR sensing into end-to-end learning for legged locomotion remains underexplored. We propose Omni-Perception, an end-to-end locomotion policy that achieves 3D spatial awareness and omnidirectional collision avoidance by directly processing raw LiDAR point clouds. At its core is PD-RiskNet (Proximal-Distal Risk-Aware Hierarchical Network), a novel perception module that interprets spatio-temporal LiDAR data for environmental risk assessment. To facilitate efficient policy learning, we develop a high-fidelity LiDAR simulation toolkit with realistic noise modeling and fast raycasting, compatible with platforms such as Isaac Gym, Genesis, and MuJoCo, enabling scalable training and effective sim-to-real transfer. Learning reactive control policies directly from raw LiDAR data enables the robot to navigate complex environments with static and dynamic obstacles more robustly than approaches relying on intermediate maps or limited sensing. We validate Omni-Perception through real-world experiments and extensive simulation, demonstrating strong omnidirectional avoidance capabilities and superior locomotion performance in highly dynamic environments.

cs.RO↗

arXiv2025

MoCA: Identity-Preserving Text-to-Video Generation via Mixture of Cross Attention

Achieving ID-preserving text-to-video (T2V) generation remains challenging despite recent advances in diffusion-based models. Existing approaches often fail to capture fine-grained facial dynamics or maintain temporal identity coherence. To address these limitations, we propose MoCA, a novel Video Diffusion Model built on a Diffusion Transformer (DiT) backbone, incorporating a Mixture of Cross-Attention mechanism inspired by the Mixture-of-Experts paradigm. Our framework improves inter-frame identity consistency by embedding MoCA layers into each DiT block, where Hierarchical Temporal Pooling captures identity features over varying timescales, and Temporal-Aware Cross-Attention Experts dynamically model spatiotemporal relationships. We further incorporate a Latent Video Perceptual Loss to enhance identity coherence and fine-grained details across video frames. To train this model, we collect CelebIPVid, a dataset of 10,000 high-resolution videos from 1,000 diverse individuals, promoting cross-ethnicity generalization. Extensive experiments on CelebIPVid show that MoCA outperforms existing T2V methods by over 5% across Face similarity.

cs.CV↗

arXiv2025

End-to-End Humanoid Robot Safe and Comfortable Locomotion Policy

The deployment of humanoid robots in unstructured, human-centric environments requires navigation capabilities that extend beyond simple locomotion to include robust perception, provable safety, and socially aware behavior. Current reinforcement learning approaches are often limited by blind controllers that lack environmental awareness or by vision-based systems that fail to perceive complex 3D obstacles. In this work, we present an end-to-end locomotion policy that directly maps raw, spatio-temporal LiDAR point clouds to motor commands, enabling robust navigation in cluttered dynamic scenes. We formulate the control problem as a Constrained Markov Decision Process (CMDP) to formally separate safety from task objectives. Our key contribution is a novel methodology that translates the principles of Control Barrier Functions (CBFs) into costs within the CMDP, allowing a model-free Penalized Proximal Policy Optimization (P3O) to enforce safety constraints during training. Furthermore, we introduce a set of comfort-oriented rewards, grounded in human-robot interaction research, to promote motions that are smooth, predictable, and less intrusive. We demonstrate the efficacy of our framework through a successful sim-to-real transfer to a physical humanoid robot, which exhibits agile and safe navigation around both static and dynamic 3D obstacles.

cs.RO↗

arXiv2025

QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for Customized Generation

Existing text-to-image models often rely on parameter fine-tuning techniques such as Low-Rank Adaptation (LoRA) to customize visual attributes. However, when combining multiple LoRA models for content-style fusion tasks, unstructured modifications of weight matrices often lead to undesired feature entanglement between content and style attributes. We propose QR-LoRA, a novel fine-tuning framework leveraging QR decomposition for structured parameter updates that effectively separate visual attributes. Our key insight is that the orthogonal Q matrix naturally minimizes interference between different visual features, while the upper triangular R matrix efficiently encodes attribute-specific transformations. Our approach fixes both Q and R matrices while only training an additional task-specific $ΔR$ matrix. This structured design reduces trainable parameters to half of conventional LoRA methods and supports effective merging of multiple adaptations without cross-contamination due to the strong disentanglement properties between $ΔR$ matrices. Experiments demonstrate that QR-LoRA achieves superior disentanglement in content-style fusion tasks, establishing a new paradigm for parameter-efficient, disentangled fine-tuning in generative models. The project page is available at: https://luna-ai-lab.github.io/QR-LoRA/.

cs.CV↗