arXiv subjects

Mingyuan Zhang

Publications and source records attributed to Mingyuan Zhang.

At least 19 recordsLinked to original sources

arXiv2026

MoCapAnything V2: End-to-End Motion Capture for Arbitrary Skeletons

Recent methods for arbitrary-skeleton motion capture from monocular video follow a factorized pipeline, where a Video-to-Pose network predicts joint positions and an analytical inverse-kinematics (IK) stage recovers joint rotations. While effective, this design is inherently limited, since joint positions do not fully determine rotations and leave degrees of freedom such as bone-axis twist ambiguous, and the non-differentiable IK stage prevents the system from adapting to noisy predictions or optimizing for the final animation objective. In this work, we present the first fully end-to-end framework in which both Video-to-Pose and Pose-to-Rotation are learnable and jointly optimized. We observe that the ambiguity in pose-to-rotation mapping arises from missing coordinate system information: the same joint positions can correspond to different rotations under different rest poses and local axis conventions. To resolve this, we introduce a reference pose-rotation pair from the target asset, which, together with the rest pose, not only anchors the mapping but also defines the underlying rotation coordinate system. This formulation turns rotation prediction into a well-constrained conditional problem and enables effective learning. In addition, our model predicts joint positions directly from video without relying on mesh intermediates, improving both robustness and efficiency. Both stages share a skeleton-aware Global-Local Graph-guided Multi-Head Attention (GL-GMHA) module for joint-level local reasoning and global coordination. Experiments on Truebones Zoo and Objaverse show that our method reduces rotation error from ~17 degrees to ~10 degrees, and to 6.54 degrees on unseen skeletons, while achieving ~20x faster inference than mesh-based pipelines. Project page: https://animotionlab.github.io/MoCapAnythingV2/

cs.CV↗

arXiv2026

Programmable generation of optical skyrmions on a silicon photonic chip

Optical skyrmions, characterized by topologically stable and spatially varying polarization textures, show immense potential for robust optical communications and metrology. However, conventional methods for generating optical Stokes skyrmions rely on bulky free-space optics, strictly constraining both system miniaturization and dynamic reconfigurability. Here, we demonstrate the efficient and programmable generation of optical skyrmions and bimerons using a compact silicon photonic chip. By integrating a programmable Mach--Zehnder interferometer mesh with a multi-dimensional grating emitter, we dynamically control the amplitudes, phases, and polarizations of emitted fundamental and orbital angular momentum modes. This architecture allows on-demand electrical switching among a complete library of optical quasi-particle states, including Néel, Bloch, intermediate, and anti-type skyrmions and bimerons. Experimental full-Stokes polarimetry confirms high-fidelity polarization textures with near-unity skyrmion numbers. Our foundry-compatible platform translates complex topological light generation into simple voltage controls, paving the way for next-generation communication and sensing systems based on optical skyrmions.

physics.optics↗

arXiv2026

GlanceWAM: Sparse Test-Time Imagination for World-Action Models

Video generative models provide rich physical priors for robot learning, yet existing world-action models (WAMs) face a fundamental trade-off: synchronous video generation at control rate is latency-prohibitive, while abandoning test-time visual imagination sacrifices task success. We show that visual imagination achieves both real-time inference and superior success rates when generated asynchronously off the critical path and consumed directly in latent space. We introduce GlanceWAM, which decouples imagination from control within a single video DiT: an asynchronous proposer glances ahead on a slow clock to imagine a single lookahead frame seconds into the future in the background, while an action head decodes action chunks at control rate (48 ms) purely in latent space without blocking. Enabled by a non-interfering attention mask that isolates video representations and staleness-robust horizon training that accommodates asynchronous lookahead aging, GlanceWAM breaks the speed-success dilemma. Trained purely on demonstrations, it attains 72.2% on the 24-task RoboCasa kitchen benchmark (surpassing synchronous Cosmos Policy at 67.1% and imagination-free co-training at 64.4%) and 99.0% on LIBERO, executing at 48 ms per chunk on an NVIDIA A100 GPU (24x faster than synchronous baselines). Code is available at https://github.com/linhanwang/GlanceWAM.

cs.CV↗

arXiv2026

$π$-Girsanov: A Generalized Method to Construct Markov State Models from Non-Equilibrium and Multiensemble Biased Simulations

We introduce $π$-Girsanov, a new method for constructing Markov state models from biased enhanced-sampling molecular dynamics simulations based on Girsanov reweighting. The key idea behind this new method is to separate the reweighting of the stationary density from the reweighting of the correlation function. We evaluate the effectiveness of this approach on several analytical potentials and on a model biomolecular system, comparing its performance with the original method. Our results show that $π$-Girsanov not only improves the estimation in a single-ensemble setting, but also resolves key challenges in estimating transition matrices from multiensemble and non-equilibrium biased trajectories. Overall, $π$-Girsanov represents a substantial advance in kinetic reweighting, strengthening the connection between enhanced sampling techniques and Markov state modeling.

physics.bio-ph↗

arXiv2026

Exponential Convex Calibration Dimension for the Multi-Label Jaccard Measure

The per-instance Jaccard score, or intersection over union (IoU), is standard in multi-label classification and binary segmentation. With $s$ labels, its loss matrix has $2^s$ outcomes and reports. Under the convention $\mathrm{Jac}(\varnothing,\varnothing)=1$, we prove that the Jaccard score, shifted-loss, and ordinary loss matrices are nonsingular and that the loss columns have affine dimension $2^s-1$. The proof combines a finite MinHash Gram representation with Boolean Möbius inversion. For exact calibration, we prove $2^{s-1} \leq \mathrm{CCdim}(L^{\mathrm{Jac}}) \leq 2^s-1$. The lower bound uses a factorially weighted distribution with $2^{s-1}+1$ supported outcomes and Bayes-optimal reports. Consequently, every exactly calibrated convex surrogate requires exponentially many prediction coordinates. We also give two polynomial-dimensional approximation guarantees with explicit regret transfers. A new $F_1$-to-Jaccard transfer turns an existing $(s^2+1)$-dimensional $F_1$ surrogate into a polynomial-time rule with asymptotic Jaccard regret at most $3-2\sqrt{2}$. For any $α>0$ and $0<ρ<1$, a MinHash square-loss surrogate attains Jaccard-regret floor $α$ uniformly over arbitrary conditional label distributions. With probability at least $1-ρ$, the direct construction has dimension $O((s^2+s\log(1/ρ))/α^2)$, while a signed variant has dimension $O((s+\log(1/ρ))/α^2)$. Thus zero-regret calibration requires exponential dimension, whereas every fixed additive regret tolerance admits polynomial prediction dimension.

cs.LG↗

arXiv2026

Towards Terabit/$λ$/s Multidimensional Silicon Photonic Engine

Increasing artificial intelligence (AI) workloads drive co-packaged optics (CPO), which integrates optical engines with electronic components. Optical interconnects can extend transmission distances and reduce latency, allowing distributed clusters in AI factories to operate as a unified computational unit. However, escalating data throughput necessitates greater parallelization of light within ultracompact form factors while maintaining stringent energy efficiency and latency constraints. Here, we present a multidimensional silicon photonic engine that achieves a communication capacity exceeding 1.8 terabit/s/lambda/s. By monolithically integrating transceivers, spatial and polarization (de)multiplexers, and optical signal processors on a single chip, we eliminate bulky discrete (de)multiplexers and power-hungry digital signal processing (DSP). In experiments, the photonic engine can be self-configured to identify two, four, or six concurrent spatial and polarization channels per fiber while mitigating dynamic channel crosstalk. Compared with the state-of-art DSP, our approach achieves >5,000-fold reductions in both power consumption and processing latency at a MIMO processing order of six. Furthermore, we demonstrate full-duplex, modulation-format-transparent inter-chip communication over 300-meter fiber. These results represent a paradigm shift for optical engines in future high-performance computing and AI-driven data centers.

physics.optics↗

arXiv2026

Exact Rank and Convex Calibration Dimension Lower Bounds for the Multi-Label F1 Loss

The instance-wise $F_1$ measure is a central performance measure for multi-label classification. For a problem with $s$ labels, it defines a $2^s\times 2^s$ loss matrix. Previous work exhibited $s^2+1$-coordinate affine and shifted low-rank representations and used them to construct quadratic-dimensional convex calibrated surrogates. We determine the exact rank. Under the convention $F_1(\varnothing,\varnothing)=1$, the $F_1$ score matrix, the shifted loss matrix, and the unshifted loss matrix all have rank $s^2-s+2$, while the column-affine dimension of the loss is $s^2-s+1$. The proof factors the nonempty score matrix through subset-incidence matrices and a positive-definite Cauchy matrix. Exact rank does not, by itself, lower-bound the dimension of an arbitrary convex calibrated surrogate. We therefore analyze the Bayes geometry of $F_1$ directly. We construct a distribution for which precisely all supersets of a fixed core label set are Bayes optimal, and show that the corresponding active loss columns, restricted to the witness support, have affine dimension $hn$, where $n=s-\lfloor s/3\rfloor$ and $h=\lceil(s\lfloor s/3\rfloor)^{1/2}\rceil-1$. Applying the feasible-subspace lower bound for convex calibration dimension gives \[ \operatorname{CCdim}(L^{F_1}) \ge \left(\frac{2}{3\sqrt{3}}-o(1)\right)s^2. \] Together with the quadratic upper bound, this establishes $\operatorname{CCdim}(L^{F_1})=Θ(s^2)$.

cs.LG↗

arXiv2026

Impacts of radiative cooling on the images of a black hole shadow and extended jets in two-temperature GRMHD simulations

The recent 230 GHz observations from the Event Horizon Telescope collaboration have successfully imaged the supermassive black hole shadow of the M87 galaxy. However, the relatively high radiative efficiency observed in the hot accretion flow suggests that radiative cooling is non-negligible and should be considered when calculating the electron temperature. In this study, we compare accretion models without and with radiative cooling across a range of mass accretion rates, $\dot{M}_{\mathrm{BH}} = (1.0 - 10) \times 10^{-6}\,\dot{M}_{\mathrm{Edd}}$, aiming to assess the impact of cooling on the disk structure, electron temperature distribution (eDF), black hole shadow morphology, broadband spectral energy distributions (SEDs), and flux variability. We performed general relativistic radiative transfer (GRRT) calculations on two-temperature, radiative, general relativistic magnetohydrodynamic (GRMHD) simulations, employing different electron heating prescriptions and nonthermal eDFs, analyzing the radiation transfer due to synchrotron emission at 230 GHz with inclination angle of $163^\circ$. These simulations are targeted toward M87$^{*}$. By comparing density profiles, eDFs, GRRT images, SEDs, and time variability between models, we find that the radiative cooling sharply decreases the electron temperature in the dense inner disk around the equatorial plane ($r\lesssim 10\,r_\mathrm{g}$), while slightly reducing jet sheath temperature. Cooling leads to a dimmer disk, more extended and brighter jets, and reduced total flux. For a given accretion rate, cooling reduces the high-frequency flux. Time variability originates primarily from the midplane in both non-cooling and cooling cases and decreases as accretion rates rise. Although currently below the dynamic range of EHT observations, the features identified in this study could be resolved by next-generation arrays such as the ngEHT.

astro-ph.HE↗

arXiv2026

From Morphology to Variability: Radiative Cooling Effects on Horizon-Scale Polarization in Two-Temperature GRMHD Simulations

Polarization signatures provide a new window to investigate the effects of radiative cooling in the horizon-scale accretion flows. Morphology and variability of polarization offer quantifiable diagnostics of how cooling modifies the polarised emission from two-temperature GRMHD simulations. We find that cooling enhances the effective Faraday depth, leading to stronger large-scale Faraday scrambling, particularly at higher accretion rates. In contrast, depolarization associated with higher-order photons is comparable between cooling and non-cooling models. Radiative cooling also increases the intrinsic asymmetry in both the ring structure and the polarization pattern. This effect is quantified by enhanced power in non-axisymmetric azimuthal modes ($β_m$, $m \neq 2$) relative to the dominant quadrupolar component $β_2$. The increased asymmetry is directly linked to stronger temporal variability of the polarization angle $\angleβ_2$, including frequent sign reversals that are absent in non-cooling models. The radial profile of $\angle β_2$ further localizes the physical origin of these effects, distinguishing regions dominated by Faraday rotation from those influenced by photon ring contributions, and providing a clear separation between cooling and non-cooling cases. Additional tests including a non-thermal electron population indicate that the polarization structure at 230 GHz is largely insensitive to the detailed form of the electron distribution functions. Our results demonstrate that horizon-scale polarization asymmetry, variability, and radial structure encode robust signatures of radiative cooling. These findings highlight the diagnostic power of time-resolved polarimetry and high-resolution imaging for constraining radiative processes in black hole accretion flows with EHT-like observations.

astro-ph.HE↗

arXiv2026

MUSE: A Unified Agentic Harness for MLLMs

Despite rapid progress, multimodal large language models (MLLMs) still fail on tasks that humans solve effortlessly, such as navigating a grid maze from a screenshot or selecting the correct puzzle piece. Rather than retraining the model, we ask a complementary question: how much capability can be elicited from a frozen MLLM purely by improving the execution scaffold around it? We introduce MUSE, a multimodal unified structured execution harness that wraps any off-the-shelf MLLM with composable modules for task representation, visual processing, perception tool use, structured parsing, deterministic verification, and verifier-guided repair, without any model retraining. We evaluate MUSE across diverse benchmarks spanning visual spatial planning, visual perception, multimodal reasoning, and fine-grained visual discrimination, using multiple state-of-the-art MLLMs. MUSE delivers consistent gains over the bare model in all settings, with the largest jumps on challenging instances. Further analysis reveals that many MLLM failures arise from harness-level shortcomings rather than fundamental model deficits, and can be addressed through verifier-guided repair without touching the model. These findings highlight the agentic multimodal harness as a critical yet underexplored design dimension, offering an orthogonal avenue for improving MLLMs beyond model-centric optimization.

cs.CV↗

arXiv2026

MoCapAnything: Unified 3D Motion Capture for Arbitrary Skeletons from Monocular Videos

Motion capture now underpins content creation far beyond digital humans, yet most existing pipelines remain species- or template-specific. We formalize this gap as Category-Agnostic Motion Capture (CAMoCap): given a monocular video and an arbitrary rigged 3D asset as a prompt, the goal is to reconstruct a rotation-based animation such as BVH that directly drives the specific asset. We present MoCapAnything, a reference-guided, factorized framework that first predicts 3D joint trajectories and then recovers asset-specific rotations via constraint-aware inverse kinematics. The system contains three learnable modules and a lightweight IK stage: (1) a Reference Prompt Encoder that extracts per-joint queries from the asset's skeleton, mesh, and rendered images; (2) a Video Feature Extractor that computes dense visual descriptors and reconstructs a coarse 4D deforming mesh to bridge the gap between video and joint space; and (3) a Unified Motion Decoder that fuses these cues to produce temporally coherent trajectories. We also curate Truebones Zoo with 1038 motion clips, each providing a standardized skeleton-mesh-render triad. Experiments on both in-domain benchmarks and in-the-wild videos show that MoCapAnything delivers high-quality skeletal animations and exhibits meaningful cross-species retargeting across heterogeneous rigs, enabling scalable, prompt-driven 3D motion capture for arbitrary assets. Project page: https://animotionlab.github.io/MoCapAnything/

cs.CV↗

arXiv2026

Distorted or Fabricated? A Survey on Hallucination in Video LLMs

Despite significant progress in video-language modeling, hallucinations remain a persistent challenge in Video Large Language Models (Vid-LLMs), referring to outputs that appear plausible yet contradict the content of the input video. This survey presents a comprehensive analysis of hallucinations in Vid-LLMs and introduces a systematic taxonomy that categorizes them into two core types: dynamic distortion and content fabrication, each comprising two subtypes with representative cases. Building on this taxonomy, we review recent advances in the evaluation and mitigation of hallucinations, covering key benchmarks, metrics, and intervention strategies. We further analyze the root causes of dynamic distortion and content fabrication, which often result from limited capacity for temporal representation and insufficient visual grounding. These insights inform several promising directions for future work, including the development of motion-aware visual encoders and the integration of counterfactual learning techniques. This survey consolidates scattered progress to foster a systematic understanding of hallucinations in Vid-LLMs, laying the groundwork for building robust and reliable video-language systems. An up-to-date curated list of related works is maintained at https://github.com/hukcc/Awesome-Video-Hallucination .

cs.CV↗

arXiv2026

Self-Configuring Universal Multichannel and Multidimensional Integrated Photonic Processing Engine

Arbitrary manipulation of light across multiple physical dimensions is essential for harnessing its parallelism in fundamental research and advanced applications, such as optical interconnects, computing, imaging, sensing, and quantum networks. However, creating a universal device capable of arbitrary operations of multidimensional optical beams has been challenging, primarily due to their complex mutual interferences and dynamic transmission characteristics. In this study, we experimentally demonstrate a self-configuring integrated photonic processor designed for the arbitrary manipulations of multiple optical waves over their spatial and polarization dimensions. Despite the random nature of the input speckle, the photonic processor relies on an optical singular-value decomposition engine to sort all orthogonal input beams and implement arbitrary processing over both spatial and polarization dimensions precisely. Notably, the photonic processor can be self programmed in situ, enabling versatile functionalities such as beam shaping, optical switching, and reconfigurable optical add-drop multiplexing. Our findings advance the manipulation of multidimensional optical beams through a scalable, CMOS-compatible integration approach, paving the way for fully exploiting the parallelism of light in various applications.

physics.optics↗

arXiv2026

Bridging Semantic and Kinematic Conditions with Diffusion-based Discrete Motion Tokenizer

Prior motion generation largely follows two paradigms: continuous diffusion models that excel at kinematic control, and discrete token-based generators that are effective for semantic conditioning. To combine their strengths, we propose a three-stage framework comprising condition feature extraction (Perception), discrete token generation (Planning), and diffusion-based motion synthesis (Control). Central to this framework is MoTok, a diffusion-based discrete motion tokenizer that decouples semantic abstraction from fine-grained reconstruction by delegating motion recovery to a diffusion decoder, enabling compact single-layer tokens while preserving motion fidelity. For kinematic conditions, coarse constraints guide token generation during planning, while fine-grained constraints are enforced during control through diffusion-based optimization. This design prevents kinematic details from disrupting semantic token planning. On HumanML3D, our method significantly improves controllability and fidelity over MaskControl while using only one-sixth of the tokens, reducing trajectory error from 0.72 cm to 0.08 cm and FID from 0.083 to 0.029. Unlike prior methods that degrade under stronger kinematic constraints, ours improves fidelity, reducing FID from 0.033 to 0.014.

cs.CV↗

arXiv2026

Boosting Large Language Models with Mask Fine-Tuning

The large language model (LLM) is typically integrated into the mainstream optimization protocol. No work has questioned whether maintaining the model integrity is \textit{indispensable} for promising performance. In this work, we introduce Mask Fine-Tuning (MFT), a novel LLM fine-tuning paradigm demonstrating that carefully breaking the model's structural integrity can surprisingly improve performance without updating model weights. MFT learns and applies binary masks to well-optimized models, using the standard LLM fine-tuning objective as supervision. Based on fully fine-tuned models, MFT uses the same fine-tuning datasets to achieve consistent performance gains across domains and backbones (e.g., an average gain of \textbf{2.70 / 4.15} in IFEval with LLaMA2-7B / 3.1-8B). Detailed ablation studies and analyses examine the proposed MFT from different perspectives, such as sparse ratio and loss surface. Additionally, by deploying it on well-trained models, MFT is compatible with collaborating with other LLM optimization procedures to enhance the general model. Furthermore, this study extends the functionality of the masking operation beyond its conventional network-pruning context for model compression to a broader model capability scope.

cs.CL↗

arXiv2026

InfiniteDance: Scalable 3D Dance Generation Towards in-the-wild Generalization

Although existing 3D dance generation methods perform well in controlled scenarios, they often struggle to generalize in the wild. When conditioned on unseen music, existing methods often produce unstructured or physically implausible dance, largely due to limited music-to-dance data and restricted model capacity. This work aims to push the frontier of generalizable 3D dance generation by scaling up both data and model design. (1) On the data side, we develop a fully automated pipeline that reconstructs high-fidelity 3D dance motions from monocular videos. To eliminate the physical artifacts prevalent in existing reconstruction methods, we introduce a Foot Restoration Diffusion Model (FRDM) guided by foot-contact and geometric constraints that enforce physical plausibility while preserving kinematic smoothness and expressiveness, resulting in a diverse, high-quality multimodal 3D dance dataset totaling 100.69 hours. (2) On model design, we propose Choreographic LLaMA (ChoreoLLaMA), a scalable LLaMA-based architecture. To enhance robustness under unfamiliar music conditions, we integrate a retrieval-augmented generation (RAG) module that injects reference dance as a prompt. Additionally, we design a slow/fast-cadence Mixture-of-Experts (MoE) module that enables ChoreoLLaMA to smoothly adapt motion rhythms across varying music tempos. Extensive experiments across diverse dance genres show that our approach surpasses existing methods in both qualitative and quantitative evaluations, marking a step toward scalable, real-world 3D dance generation. Code, models, and data will be released.

cs.CV↗

arXiv2026

DiMo: Discrete Diffusion Modeling for Motion Generation and Understanding

Prior masked modeling motion generation methods predominantly study text-to-motion. We present DiMo, a discrete diffusion-style framework, which extends masked modeling to bidirectional text--motion understanding and generation. Unlike GPT-style autoregressive approaches that tokenize motion and decode sequentially, DiMo performs iterative masked token refinement, unifying Text-to-Motion (T2M), Motion-to-Text (M2T), and text-free Motion-to-Motion (M2M) within a single model. This decoding paradigm naturally enables a quality-latency trade-off at inference via the number of refinement steps. We further improve motion token fidelity with residual vector quantization (RVQ) and enhance alignment and controllability with Group Relative Policy Optimization (GRPO). Experiments on HumanML3D and KIT-ML show strong motion quality and competitive bidirectional understanding under a unified framework. In addition, we demonstrate model ability in text-free motion completion, text-guided motion prediction and motion caption correction without architectural change. Additional qualitative results are available on our project page: https://animotionlab.github.io/DiMo/.

cs.CV↗

arXiv2026

NECromancer: Breathing Life into Skeletons via BVH Animation

Motion tokenization is a key component of generalizable motion models, yet most existing approaches are restricted to species-specific skeletons, limiting their applicability across diverse morphologies. We propose NECromancer (NEC), a universal motion tokenizer that operates directly on arbitrary BVH skeletons. NEC consists of three components: (1) an Ontology-aware Skeletal Graph Encoder (OwO) that encodes structural priors from BVH files, including joint semantics, rest-pose offsets, and skeletal topology, into skeletal embeddings; (2) a Topology-Agnostic Tokenizer (TAT) that compresses motion sequences into a universal, topology-invariant discrete representation; and (3) the Unified BVH Universe (UvU), a large-scale dataset aggregating BVH motions across heterogeneous skeletons. Experiments show that NEC achieves high-fidelity reconstruction under substantial compression and effectively disentangles motion from skeletal structure. The resulting token space supports cross-species motion transfer, composition, denoising, generation with token-based models, and text-motion retrieval, establishing a unified framework for motion analysis and synthesis across diverse morphologies. Demo page: https://animotionlab.github.io/NECromancer/

cs.CV↗