Searcharxiv⌕ Search

arXiv subjects

Yujia Zhang

Publications and source records attributed to Yujia Zhang.

At least 19 recordsLinked to original sources

CamChoreo: Benchmarking Temporally Grounded Compositional Camera Motion Understanding for Vision Language Models

Understanding camera motion is fundamental to video perception, with applications in spatial intelligence and controllable video generation. Existing camera motion evaluations typically assign labels to entire clips, overlooking within-shot changes and simultaneous movements. We therefore formulate camera motion understanding as temporally grounded, compositional recognition, requiring models to localize motion-consistent intervals and identify all active movements. We introduce CamChoreo, the first benchmark for this task, comprising 4,229 diverse single-shot clips with expert temporal annotations using 20 direction-aware labels. Nearly half of its segments contain compound camera motion. We also construct a separate training corpus of approximately 43K clips with the same annotation format. On CamChoreo, supervised fine-tuning (SFT) on this corpus achieves 62.1/64.1 micro F1 with 4B/8B backbones, outperforming advanced closed-source vision language models (VLMs). To complement the primarily semantic representations of VLM visual encoders with cross-frame geometry, we introduce CamInject, which injects features from a frozen 3D foundation model into the VLM. CamInject improves over SFT but requires running the costly 3D model at inference. We instead propose CamDistill, which distills the same geometric knowledge into lightweight camera tokens during training and removes the 3D model at inference. CamDistill improves micro F1 over same-data SFT by 5.4/3.7 points on 4B/8B backbones, closely approaching CamInject. CamDistill-8B also demonstrates strong zero-shot transfer on the external camera motion understanding benchmarks CameraBench and CameraMotionVQA. Together, these resources and CamDistill enable benchmarking and improving temporally grounded, compositional camera motion understanding.

cs.CV↗

Knowledge in the Harness, Reliability in the Weights: Teacher-Free Co-Evolution of an LLM Agent

Small open models suit private customer-service deployments but trail frontier models in two ways: they miss what the written policy requires in a given situation, and they act inconsistently on what they know. We study whether such a model can close both gaps from its own deployment logs and their reference outcomes, without a stronger teacher model. Our starting point is a Gemma 4 12B agent that reads its domain policy from on-demand skills rather than from its system prompt. In each round, the deployed system serves a fresh batch of tasks; the base model diagnoses failed conversations and writes rules into the skills, and a rule survives only if it passes code checks, counterexample self-review, a paired retest, and a noise-calibrated validation gate. Reinforcement learning (GRPO) then trains a LoRA adapter on complete multi-turn, tool-using conversations. On the retail and airline domains of $τ^2$-bench, without training or selecting on official tasks, the agent after two rounds improves on this starting point by 12.5 pass$^1$ points in retail and 19.5 in airline, and by 18.4 and 22.0 pass$^4$ points. It also exceeds the benchmark's reference agent, which runs the same model with the whole policy in its system prompt, by 11.8 and 8.0 pass$^1$ points and by 22.8 and 16.0 pass$^4$ points, closing most of the gap to three frontier models under the same protocol. A $2\times2$ ablation shows that the rules rescue tasks the agent never solved and carry policy knowledge that RL with the rules in context does not internalize, whereas the trained weights make solved tasks reliable.

cs.CL↗

Scalable Bayesian Optimization of Composite Functions for Image-Based Inverse Problems in Materials Characterization

Estimating physical parameters from scientific images is a common inverse problem in materials characterization that often relies on expensive physics-based simulations. In electron microscopy, specimen thickness and crystal mistilt are critical parameters that govern how electrons scatter through the sample, and therefore the accuracy of any atomic-scale structure recovered from it. They are commonly inferred by matching experimental position-averaged convergent-beam electron diffraction (PACBED) patterns to simulated ones, but grid searches scale poorly and neural-network methods require extensive pretraining that may not transfer to new conditions. Here, we propose scalable Bayesian optimization of composite functions (SBOCF), a simulation-efficient method that exploits the known composite structure of the image-matching objective and the intermediate information contained in simulated images. By representing PACBED images with patch-level summaries and two correction terms, SBOCF preserves the original pixel-wise objective while reducing the number of modeled outputs from 24,649 to 11. Under a budget of 50 simulator evaluations, SBOCF outperformed standard Bayesian optimization with expected improvement on synthetic SrTiO3 benchmarks with thick and thin specimens, reducing the median final SSE by up to 290 times in the thick-sample case. On experimental data, SBOCF produced parameter estimates consistent with previously reported values without task-specific pretraining. For a simulated mistilted specimen, using the SBOCF estimates in a downstream ptychographic reconstruction recovered sharp atoms that were otherwise blurred. These results establish SBOCF as a promising approach for inverse problems involving expensive simulators and high-dimensional structured outputs.

cs.LG↗

Steer Unified Audio Editing Toward Reliable Source Selection

Unified audio editors support diverse operations, yet often struggle to modify the intended source while preserving non-target content. We show that source selection in audio editing is shaped by early denoising states, while incorrect early selections can hardly be corrected during subsequent denoising. Motivated by these findings, we propose Source-selection Training via Early-trajectory Emphasis and Recovery (STEER), a two-stage post-training algorithm for reliable selective audio editing. STEER-Emphasis prioritizes training on task-critical early states to reduce source-selection errors, while STEER-Recovery learns corrective denoising from self-generated rollout states to recover from early deviations. To systematically assess source-selection reliability, we introduce the Selective Audio Editing Benchmark, enabling large-scale automatic evaluation of target manipulation and non-target preservation. Applying STEER to our pretrained multitask base editor gives SteerSound, a unified audio editor with leading selective editing performance among open-source audio editors. Experiments on our base editor, UNISON, and SAM Audio demonstrate that STEER consistently improves both selective editing capability and overall editing performance. Early-trajectory emphasis further brings substantial improvements in transcript-conditioned target speaker removal and separation, highlighting the broader applicability of this training principle.

cs.SD↗

See, Foresee, Act: Intent-Aware World Modeling for Proactive Retail Assistance

World models allow agents to anticipate action consequences, but proactive assistance also requires interpreting incomplete observations and deciding whether to intervene before a user asks for help. We propose See-Foresee-Act and instantiate it as the Proactive Intent World Model (PIWM) for retail assistance. See preserves role-relevant observable evidence alongside AIDA-BDI customer states; Foresee predicts one-step consequences for candidate actions; Act explicitly compares those consequences, including non-intervention. GuidanceSalesBench supplies predominantly synthetic supervision linking observations, states, candidate outcomes, and decisions. On 509 Full test examples, consequence-based selection achieves 63.65% decision accuracy, compared with 45.38% for direct decision output, an improvement of 18.27 percentage points. These results support candidate-consequence comparison for decision reasoning in this retail task. Code and data will be released after consent verification and upon publication.

cs.CL↗

Towards reactor-relevant electromagnetic turbulence in the MAST-U spherical tokamak

We investigate electromagnetic ion-scale turbulence in a high-performing H-mode plasma of the Mega Ampere Spherical Tokamak Upgrade (MAST-U) with local gyrokinetic simulations, asking whether it approaches the non-zonal transition: the nonlinear transition to a high-flux state above a critical value of $q^{2}β_{e}$ found in simulations of STEP, where $q$ is the safety factor and $β_{e}$ the ratio of electron thermal pressure to magnetic pressure. We examine two flux surfaces: one at $t = 531\,\mathrm{ms}$, close to peak normalised pressure $β_N$, at normalised poloidal flux $ψ_N = 0.50$; and one at $t = 591\,\mathrm{ms}$ and $ψ_N = 0.60$, the ideal-ballooning-stable surface of largest $q^{2}β_e$ inside $ψ_N = 0.8$ at that time. On both surfaces the dominant ion-scale instabilities are microtearing and electromagnetic ballooning-like modes, as in STEP. Without equilibrium flow shear, at least one of the surfaces lies above the large flux threshold with every set of kinetic species we simulate, with ion-scale heat fluxes far exceeding power balance, in turbulence characterised by a highly stochastic magnetic field. Strong flow shear driven by neutral-beam injection in the real experiment suppresses this turbulence and holds the plasma below the transition. When the carbon impurity is included in the simulation, the ion-scale flux is then close to power balance on both surfaces. Without rotation, gyrokinetics suggests the transition would limit the accessible profiles. This could degrade MAST-U plasma performance using heating schemes that lack a momentum source, and this will be tested in forthcoming electron Bernstein wave heating experiments.

physics.plasm-ph↗

InteractAgent: Agentic Human Motion Interaction with Memory-Augmented Reflection

Long-horizon human-scene interaction requires coordinated spatial reasoning, physical feasibility, and the ability to recover from compounding execution errors. Text-driven motion models are largely scene-agnostic and lack closed-loop correction, while geometry-conditioned approaches depend on paired scene-motion training or explicit 3D preprocessing that limits generalization to novel layouts and open-vocabulary goals. We present InteractAgent, an agentic multimodal LLM system for long-horizon human-scene interaction that plans, executes, observes, and reflects in a closed loop, using a single MLLM as both planner and evaluator and a frozen MotionLLM as the executor. InteractAgent integrates Online Observation for step-wise recovery decisions (continue/redo/replan) and Memory-Augmented Reflection for multi-round plan refinement with adaptive stopping. On UniHSI, InteractAgent improves task success and reduces penetration compared with training-free baselines. Code: https://github.com/AIGeeksGroup/InteractAgent.

cs.RO↗

CineRecon: Benchmarking Frontier Models for Agentic Cinematic Reconstruction

Reconstructing reference videos as editable dynamic 3D scenes is an important path toward controllable video generation for filmmaking: these scenes provide modifiable, renderable visual proxies that enable explicit control over scene layout, subject actions, and camera behavior. To evaluate agents' ability to reconstruct editable dynamic 3D scenes from dynamic reference videos, we introduce CineRecon, a benchmark comprising 50 reference videos and 360 annotated evaluation clips. The reference videos span film, real-world footage, and games. Given a complete video, an agent uses tools to construct, animate, render, and refine its reconstruction. We evaluate this task under three representation settings: Proxy, Simplified, and Detailed. These settings increase geometric and appearance requirements while retaining the same requirements for spatial relations, camera behavior, and subject dynamics. This design tests whether agents can reproduce the events in a video when detailed appearance is unnecessary, and how performance changes as representation requirements increase. For evaluation, we construct checklists that associate observable properties and events with specific entities and times. A vision-language judge compares timestamp-aligned reference and rendered frames across 11 categories, grouped into camera, static, and dynamic consistency dimensions. We evaluate 12 models using their corresponding agent harnesses and find failures across all three dimensions. Subject motions and interactions remain difficult to reproduce even under the Proxy setting.

cs.CV↗

MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cues

Video temporal grounding (VTG), which localizes the start and end times of a queried event in an untrimmed video, is a key test of whether multimodal large language models (MLLMs) understand not only what happens but also when it happens. Although modern MLLMs describe video content fluently, their timestamp predictions remain unreliable, while existing remedies either require costly post-training on temporal annotations or rely on coarse training-free heuristics. In this work, we probe the cross-modal attention of MLLMs and uncover a perception-generation gap. Our key finding is that MLLMs often know the target interval during prefill, but lose this signal when generating the final answer. In the prefill stage, a sparse set of attention heads, which we call Temporal Grounding Heads (TG-Heads), concentrates query-to-video attention on the ground-truth interval. During autoregressive decoding, however, the answer tokens shift attention away from this interval toward visually salient but query-irrelevant segments. This observation motivates an inference-time read-then-regenerate framework. We first convert TG-Head prefill attention into a debiased frame-level relevance signal and extract the high-attention interval it highlights. We then re-invoke the MLLM with visual context restricted to this interval using video cropping. Without parameter updates or architectural changes, our framework consistently improves MiMo-VL, Qwen3-VL, and Molmo2 on three benchmarks, with gains of up to +3.5 mIoU. The project website can be found at https://ddz16.github.io/mllmsknowwhen.github.io/.

cs.CV↗

Learning Spatiotemporal Sensitivity in Video LLMs via Counterfactual Reinforcement Learning

Video large language models (Video LLMs) can achieve strong video-QA accuracy without reliably tracking spatiotemporal dynamics. A model may answer a motion question from static cues, for example, and give the same prediction even after the underlying motion is reversed. Correctness-based reinforcement learning does not directly address this problem because it rewards the final answer without requiring the policy to respond to task-relevant video dynamics. We propose Counterfactual Relational Policy Optimization (CRPO), which explicitly trains Video LLMs to respond to controlled changes in the visual input. For each training example, CRPO constructs a counterfactual video, such as a horizontally flipped or temporally reversed version of the original, and jointly optimizes rollouts from both videos under a shared policy. Factual supervision anchors what the model should answer, while counterfactual supervision constrains when that answer should change: predictions should change when an intervention alters task-relevant dynamics and remain stable when the queried property is preserved. This coupling provides a direct behavioral learning signal without requiring ground-truth labels for transformed videos or annotated spatiotemporal reasoning traces, while discouraging indiscriminate answer changes. To evaluate this property, we introduce DyBench, a paired counterfactual benchmark with 3{,}014 videos and a strict pair-accuracy metric. Across paired spatiotemporal evaluations and standard video benchmarks, CRPO improves sensitivity to motion and temporal changes while improving performance on general video understanding. The gains also extend to segment reordering, a transformation never used during training, suggesting that CRPO learns sensitivity to video dynamics beyond the training interventions. The project website can be found at https://ddz16.github.io/crpo.github.io/ .

cs.CV↗

Temporal Regression-Based Model-Free Sensorless Control of Permanent Magnet Synchronous Motor

To address the widespread sensitivity of surface-mounted permanent magnet synchronous motor (SPMSM) sensorless control to motor parameters, this paper proposes a temporal regression-based model-free sensorless control (TFC) method. First, voltage integrals and current increments over consecutive short intervals are stacked to construct a finite window regression, in which the unknown stator inductance appears as a common scalar coefficient. Second, a projection operator constructed from the stacked current increments eliminates the inductance term, and a least-squares formulation is developed to reconstruct the rotor flux vector. Meanwhile, the analysis of the projected regression and current-flux geometry establishes a two-dimensional direction vector whose components share a common amplitude containing the stator resistance and flux linkage. This amplitude cancels during position extraction. By setting the resistance reference to zero, the proposed method estimates the position without specifying the stator resistance, inductance, or flux linkage. Finally, experimental results verify the effectiveness of the proposed TFC method.

eess.SY↗

Entirely nonlocal quantum magic without entanglement

Nonstabilizerness, or magic, is an archetypal \emph{quantum} resource that is necessary for quantum computational advantage. Here we uncover a phenomenon seemingly at odds with the quantum nature of magic: entirely nonlocal magic (ENM)---magic present only in correlations and absent from each party's marginal---can live without entanglement. We systematically study this separation and show it is universal and operationally reversible: every magical state or channel can be encoded into and recovered from a separable ENM realization using only local stabilizer processing and classical communication. We leverage this mechanism to devise an activation key protocol in which a classical key controls access to non-Clifford operations. We further formulate magic secret sharing, in which computational power inaccessible to any party alone becomes accessible through cooperation. On a superconducting quantum processor, we experimentally demonstrate activation key and network computing primitives, together with separable ENM state preparation and extraction protocols. Together, our results establish that magic can be classically activated, localized, and secret-shared without entanglement, providing new resource-control primitives for distributed quantum computation.

quant-ph↗

Sheet As Token: A Graph-Enhanced Representation for Multi-Sheet Spreadsheet Understanding

Workbook-scale spreadsheet understanding is increasingly important for language-model-based data analysis agents, but remains challenging because relevant information is often distributed across multiple sheets with heterogeneous schemas, layouts, and implicit relationships. Existing retrieval-augmented approaches typically decompose spreadsheets into rows, columns, or blocks to improve scalability; however, such chunk-centric representations can fragment worksheets into isolated text spans and weaken global sheet-level semantics. We propose Sheet As Token (SAT), a graph-enhanced framework that treats each worksheet as a unified semantic unit for multi-sheet spreadsheet retrieval. SAT serializes sparse schema-aware features, including sheet name, shape, and column headers, and encodes each worksheet into a compact dense token. Given a query, SAT retrieves candidates with a BGE-initialized Sheet Encoder and refines them with a gated relational GNN. In strict full-corpus evaluation, SAT reaches 0.9173 NDCG@5 on IndustryTab-614 and 0.9222 on IndustryTab-1K, relative improvements of 44.6% and 46.7% over zero-shot BGE RAG, respectively. SAT therefore improves both retrieval accuracy and serving efficiency: on IndustryTab-1K, it exceeds a Qwen3.5-9B RAG reranker by 12.5% while reducing online latency from 2.61 s to 9.24 ms, approximately 283X faster. These results show that SAT provides accurate and latency-efficient retrieval in the evaluated fixed-corpus setting. Code and data are available at https://github.com/SHITIANYU-hue/SheetasToken .

cs.AI↗

Visualizing spin-polarization of an altermagnet KV$_2$Se$_2$O via spin-selective tunneling

Altermagnetism, a recently identified magnetic phase that combines vanishing net magnetization with momentum-dependent spin splitting, challenges the conventional dichotomy between ferromagnets and antiferromagnets. While several candidate materials have been proposed, direct experimental evidence linking crystal symmetry, electronic structure and d-wave spin polarization remains scarce. Here we report the visualization of a metallic d-wave altermagnet in KV2Se2O. Through spin-selective scanning tunneling microscopy powered by a topological insulator tip, we uncover symmetry-protected momentum-dependent spin splitting that follows a characteristic d-wave form factor. Our results establish KV2Se2O as a tunable platform to study the interplay between spin-valley locking, Fermi-surface instability and unconventional magnetism, and open a pathway toward symmetry-engineered spintronics without net magnetization.

cond-mat.mtrl-sci↗

On the transition to large fluxes and access to second stability in gyrokinetic simulations of electromagnetic turbulence in STEP

This work investigates the nonlinear transition to large heat fluxes observed in local gyrokinetic simulations of electromagnetic turbulence in STEP. Using the stress-balance framework of Zhang et al. (arXiv:2606.04616, arXiv:2607.11789), we confirm that the onset of extreme transport correlates with a critical value of $q^{2}β_{e}$, where $q$ is the safety factor and $β_{e}$ is the ratio of electron thermal pressure to magnetic pressure, and relate this to a limit on the poloidal beta $β_{\mathrm{pol}}$. Crucially, this critical value lies below any relevant linear stability limit in the ($q$, $β_{e}$) space (e.g., the onset of ideal or kinetic ballooning modes). Using an extensive set of nonlinear gyrokinetic simulations, we demonstrate that the transition to large fluxes in STEP is governed by a balance between the electrostatic and magnetic-flutter stresses. We argue, and also show numerically, that larger-major-radius tokamaks reach the electromagnetic non-zonal regime at lower $β_{e}$, making this MHD-controlled saturation limit more accessible in reactor-scale devices than in small spherical tokamaks. We also demonstrate that access to a second-stable regime enables re-saturation at larger values of $β^{\prime}$. We further show that the ideal ballooning mode (IBM) threshold serves as a useful proxy for delineating this second-stable region and also as a qualitative guide for the onset of large fluxes. These results provide a predictive framework for identifying no-go zone predictions from local gyrokinetics and offer new insight into the electromagnetic saturation physics relevant to STEP and other high-$β_{e}$ devices.

physics.plasm-ph↗

Utonia: Toward One Encoder for All Point Clouds

We dream of a future where point clouds from all domains can come together to shape a single model that benefits them all. Toward this goal, we present Utonia, a first step toward training a single self-supervised point transformer encoder across diverse domains, spanning remote sensing, outdoor LiDAR, indoor RGB-D sequences, object-centric CAD models, and point clouds lifted from RGB-only videos. Despite their distinct sensing geometries, densities, and priors, Utonia learns a consistent representation space that transfers across domains. This unification improves perception capability while revealing intriguing emergent behaviors that arise only when domains are trained jointly. Beyond perception, we observe that Utonia representations can also benefit embodied and multimodal reasoning: conditioning vision-language-action policies on Utonia features improves robotic manipulation, and integrating them into vision-language models yields gains on spatial reasoning. We hope Utonia can serve as a step toward foundation models for sparse 3D data, and support downstream applications in AR/VR, robotics, and autonomous driving.

cs.CV↗

A Multi-Modal Framework with Cross-Subject Pseudo-Labeling and Semantic Alignment for Micro-Gesture Recognition

Micro-gestures (MGs) are spontaneous and subtle body movements that frequently convey hidden human emotions. Recognizing MGs in untrimmed videos remains highly challenging due to their extremely low signal-to-noise ratio, severe long-tailed class distribution, and the inherent domain shift encountered in cross-subject evaluation scenarios. In this paper, we propose a comprehensive multi-modal framework for Track 1 of the 4th MiGA-IJCAI Challenge. To capture fine-grained representations, we design a saliency-guided multi-modal extraction pipeline integrating 68-keypoint skeleton joint coordinates, 3D heatmap volumes, and high-resolution RGB visual features. We introduce a gentle square-root smoothed weighting mechanism paired with an Orthogonal Semantic Embedding Loss to protect tail classes without compromising overall recognition capabilities. More importantly, to bridge the cross-subject generalization gap, we propose a Cross-Modal Pseudo-Labeling (CMPL) strategy for unsupervised domain adaptation, which significantly boosts single-modal robustness. A temperature-scaled soft-voting mechanism is finally utilized to alleviate overconfidence during late fusion. Extensive experiments demonstrate that our framework achieves a competitive F1-score of 68.13\%, securing the 4th place.

cs.CV↗

Non-Local and Non-Markovian Effects of a Microscopic Two-Level Defect in Superconducting Quantum Circuits

Microscopic two-level systems (TLS) -- ubiquitous atomic-scale defects in solid-state quantum devices -- are a dominant source of qubit decoherence, yet their role is often considered local and short-memoried. Here, we report the observation of a coherent TLS that couples simultaneously to two spatially distant superconducting qubits. The TLS is identified to reside within the tunable coupler linking the qubits, enabling controllability of the TLS-qubit coupling strength via coupler frequency -- a capability absent in earlier studies. This tunability allows us to systematically probe how TLS distorts qubit dynamics, revisiting the decoherence model in the presence of non-Markovian TLS dephasing noise. This is corroborated by the reconstructed $1/f$ noise spectrum of TLS frequency fluctuation spanning more than ten orders of magnitude (0.1\,mHz -- 1\,MHz) that reveals discrete fluctuator signatures. Quantum process tomography further unveils TLS-induced correlated qubit dynamics, highlighting the long-lived TLS as an effective source of non-Markovianity. Our findings expose a previously overlooked interaction mechanism in scalable quantum architectures: defects embedded in coupling elements can simultaneously affect multiple qubits with variable impact. Beyond immediate implications for system characterization and calibration, this situation provides a powerful testbed for studying defect-driven quantum dynamics, refining error suppression strategies, and advancing architecture design for scalable quantum technologies.

quant-ph↗