Searcharxiv⌕ Search

arXiv subjects

Jian Kang

Publications and source records attributed to Jian Kang.

At least 19 recordsLinked to original sources
arXiv2026

Understanding the relationship between retrospective soundscape attributes and continuous pleasantness and eventfulness ratings

Soundscape studies ask listeners to rate perceived affective quality (PAQ) attributes, such as pleasant and vibrant, after listening to recordings or experiencing acoustic environments. These retrospective ratings summarise an experience during which impressions may change. International Organization for Standardization (ISO) procedures convert these PAQ ratings into Pleasantness and Eventfulness scores. Pleasantness and Eventfulness (P/E) can also be rated directly and continuously during listening, capturing changes in reported perception over time. Retrospective PAQ ratings and continuous direct P/E describe the same soundscape experience from different perspectives. We examine how PAQs relate to direct P/E and how ISO-derived P/E corresponds to direct judgements, using 1,782 paired 30-s trials involving audiovisual recordings from 18 streets. The results show that PAQs retain a predominantly two-dimensional structure. Vibrant and Monotonous lie closer to the Eventfulness-Uneventfulness axis in the direct P/E plane than in the ISO 2025 reference geometry. The full 30-s direct P/E mean has lower calibrated error against ISO-derived P/E than three 10-s means. An empirical mapping using the same 8 PAQs reduces held-out joint root-mean-square error by 4.9% relative to affine-calibrated ISO 2025 scoring. The findings show that retrospective and continuous assessments offer related but distinct soundscape descriptions.

eess.AS↗
arXiv2026

From Transcription to Reformulation: Defining and Benchmarking Grounded Generative Speech Recognition

Speech language models have expanded automatic speech recognition (ASR) from transcription to general speech-to-text generation. Yet existing benchmarks still assume a single ground-truth transcript per utterance, an assumption that struggles when models produce readable, formatted, or structured outputs. In this work, we introduce Grounded Generative Speech Recognition (GSR), reformulating speech recognition as generating semantically grounded text instead of a single transcript. We define a grounded reformulation space that characterizes all valid textual realizations supported by the input speech, and establish a four-level hierarchy of reformulation capabilities, including literal transcription, canonical reformulation, semantic reformulation and speech-grounded structuring. Based on this formulation, we present GSR-Bench, the first comprehensive benchmark designed to evaluate generative speech recognition systems beyond exact transcript matching. GSR-Bench covers diverse scenarios including refined meeting transcription, cross-dialect translation, context-aware speech understanding, and robustness under silent or noisy acoustic conditions. To address the limitations of word-level evaluation metrics, we develop a multi-dimensional evaluation framework combining speech grounding assessment and LLM-based semantic judgment, whose reliability is validated through human preference studies. Extensive experiments across representative architectures, ranging from conventional ASR systems to end-to-end Speech-LLMs, reveal fundamental trade-offs between transcription fidelity and generative reformulation ability. Our analysis further identifies critical failure modes, including acoustic hallucination, unsupported generation, and instruction drift, providing insights into the design and evaluation of future grounded generative speech systems.

cs.CL↗
arXiv2026

XingChen-UniFE: A Unified Generative Speech Front-End via Complementary Acoustic-Semantic Modeling

This paper presents XingChen-UniFE, a unified large audio language model (LALM) for general speech restoration (GSR) and target speaker extraction (TSE). The proposed model employs complementary semantic and acoustic representations through a dual-stream encoder, where a self-supervised learning model provides hierarchical semantic features and a learnable acoustic branch captures fine-grained structures from complex spectra. A bidirectional cross-attention module further facilitates information exchange between the two representations. The fused representation conditions the LALM for discrete speech token generation with task and speaker information. Experiments demonstrate strong performance on both GSR and TSE and robust generalization to unseen degradation conditions.

eess.AS↗
arXiv2026

Each Reward to Its Relevant Tokens: Reward-Specific Token-Level Credit Assignment for Multi-Reward RL

Language models are increasingly trained against several verifiable rewards at once, such as the correctness and the format of a tool call, or the individual constraints of a complex instruction. Reward-decoupled normalization (GDPO) normalizes each reward separately so that different reward combinations keep distinct advantages, but it still spreads every reward's advantage evenly over the whole response, although each verifier checks only a few positions. We propose MRTPO, which assigns the advantage of each reward to the tokens relevant to it, using the information its verifier checks, without auxiliary models or token-level annotations. For each reward, the policy replays its own response with and without that reward's verifier information, and the per-token Jensen-Shannon distance between the two next-token distributions measures each token's relevance and reassigns the advantage. On text-to-SQL, multi-constraint instruction following and tool calling, MRTPO converges faster and achieves better performance than the compared methods.

cs.LG↗
arXiv2026

ForkCast: Predictive Recovery for Vision-Language-Action Policies via Counterfactual Futures

Vision-language-action (VLA) policies perform manipulation tasks using camera images and language instructions, but object slip or visual occlusion can disrupt execution. Recovery is typically learned during policy training or delegated to a monitor that triggers a scripted repair after detecting a failure. We introduce ForkCast, a predictive recovery framework that compares counterfactual futures to determine whether and how to intervene in a frozen VLA policy. At each decision point, ForkCast evaluates three candidates: executing the policy's nominal action chunk (Commit), adjusting the camera viewpoint (Reobserve), and replanning the arm trajectory (Replan). A candidate-conditioned world model forecasts the latent outcome of each candidate at a common future horizon, and a shared task-progress head scores the predicted outcomes. By evaluating the policy's own continuation alongside the repair candidates, a single comparison based on task progress determines both whether and how to intervene, without a failure detector, an intervention threshold, or updates to the VLA backbone. On five simulated RoboTwin 2.0 tasks with injected occlusion and object slip, ForkCast increases pooled success from 13.3% for the frozen $π_{0.5}$ policy to 30.0% and transfers to two additional VLA backbones without updating ForkCast. After adaptation on real-robot candidate rollouts, ForkCast also improves success over the base policy under both clean and visibility-limited conditions. Code will be released to benefit the research community.

cs.RO↗
arXiv2026

Lost in Adaptation: Layer-Selective Recovery of Temporal Reasoning in Video-Language Models

Multimodal adaptation can weaken temporal reasoning (TR) in video-language models (VLMs) even when the visual evidence needed for an answer is correctly perceived. This weakened competence, however, remains partly recoverable from the model's paired pre-adaptation language checkpoint. We introduce MERIT, a gradient-free framework that selectively restores this competence across the language backbone while protecting temporal perception (TP). It assigns each self-attention layer a VLM-dominant or LLM-dominant interpolation and uses the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to search the recipe space under a perception-constrained objective. Across three VLM families and five video benchmarks, MERIT improves reasoning-aligned performance in all 15 model--benchmark evaluations while preserving TP across all three Video-MME settings. Recipes selected using 232 diagnostic examples transfer to four unseen benchmarks, with relative gains up to 27.8%, and outperform full-backbone, uniform-attention, and alpha-tuned random-layer merging. Interventional masking shows that MERIT-selected layers are disproportionately important for TR, while frame-level attribution reveals increased reliance on temporally distributed, causally relevant evidence. These results show that adaptation-induced reasoning degradation can be repaired post hoc through selective reuse of pre-adaptation competence, without learning new parameters or adding inference overhead.

cs.CV↗
arXiv2026

Field Response of the Orbital Moment Probed by Integrated Magnetic Circular Dichroism

Magnetic circular dichroism (MCD) is an optical probe of orbital time reversal symmetry breaking. Although frequency integrated MCD is related to the self-rotation contribution to orbital magnetization at zero field, its magnetic response generally differs from the thermodynamic orbital susceptibility. Here, we define the MCD susceptibility and derive general expressions for multiband systems using an electromagnetic $U(1)$ gauge invariant approach. For insulators, the response contains contributions from the non-Abelian quantum metric, Berry curvature, and field induced interband mixing. For metals, a finite temperature Green function formulation separates the smooth response into Fermi sea and Fermi surface contributions. An independent Landau level calculation confirms the expressions for the zero-field MCD moment and its linear magnetic response. In a continuum model for twisted bilayer graphene, the MCD susceptibility vanishes within the charge neutrality gap, while the thermodynamic orbital susceptibility remains markedly diamagnetic. Near the gap edges, the orbital moment term dominates the response. Features near the van Hove energies are associated with density of states enhancement. These results clarify how the two susceptibilities reflect different orbital processes even for the same electronic structure and thus establish the MCD susceptibility as a distinct probe of orbital magnetic response.

cond-mat.str-el↗
arXiv2026

Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?

Automated failure attribution uses LLMs to identify where and why agentic systems fail. As agents become more capable, their failures become subtler, making automated attribution increasingly important. We introduce Who&When Pro, a large-scale benchmark for automated failure attribution in agentic systems. Using a strictly controlled pipeline that injects a failure only after exactly replaying a successful prefix, we construct 12,326 failed trajectories with golden labels across 3 modalities and 26 benchmarks covering various scenarios. Beyond benchmarking, we conduct extensive experiments and analyses, revealing systematic patterns in how models attribute failures across modalities, protocols, and model families, and providing empirical guidance for future automated failure attribution systems.

cs.AI↗
arXiv2026

Overcoming Prior Barriers: Supervised Fine-Tuning under Long-Tail Distribution

Supervised fine-tuning (SFT) adapts pretrained large language models (LLMs) to downstream tasks, but the required concepts can receive substantially different levels of pretrained support. Frequent concepts are more likely to be well learned, whereas rare concepts may remain weakly represented. We introduce a novel notion named prior barrier to quantify how strongly the pretrained model supports competing concepts over the target concept. We observe that prior barriers follow a long-tail distribution, placing head and tail concepts at different starting points for SFT: head concepts face lower prior barriers, whereas tail concepts require additional instructions to overcome their higher prior barriers. Our theoretical analysis further derives a predictive risk bound for SFT under long-tail prior barriers, explicitly characterizing how the prior barrier and accumulated SFT evidence jointly determine predictive performance. Motivated by this prior barrier-dependent demand, we propose PASS, an adaptive SFT instruction selection method that constructs reference-derived concepts and estimates the distinguishing evidence provided by each instruction, and adaptively allocates the selection budget toward concepts that remain insufficiently covered under the current selection. In this way, PASS jointly considers which instructions can provide useful evidence and where additional supervision is needed under a limited budget. Experiments show that our method consistently outperforms seven state-of-the-art instruction selection methods on four backbone-budget settings. An ablation study further shows that PASS's adaptive allocation consistently improves over uniform allocation.

cs.AI↗
arXiv2026

TS-OPD: Reconciling ASR and QA in Speech Language Models via Task-Specific On-Policy Distillation

Speech Language Models (SLMs) inherit strong instruction-following capabilities from pretrained language models, yet ASR specialization can substantially degrade them. To address this ASR--QA trade-off, we propose Task-Specific On-Policy Distillation (TS-OPD), which leverages models before and after ASR specialization as complementary QA and ASR teachers. The student generates separate task-conditioned trajectories for ASR and QA, each supervised only by its corresponding teacher, thereby reducing direct competition between the two supervision signals. Experiments on basic ASR, contextual ASR, and QA demonstrate that TS-OPD improves recognition while preserving QA capability. Moreover, TS-OPD remains robust across different balancing coefficients and continues to benefit from increased distillation data.

cs.SD↗
arXiv2026

Low-altitude aircraft will reshape noise exposure across global cities

Just as motorisation made road traffic a defining noise source of the twentieth-century city, low-altitude aircraft may reshape urban acoustic exposure in the twenty-first. Yet how this noise interacts with existing sound environments and three-dimensional urban form across global cities remains unclear. We modelled identical low-altitude aircraft operations across ten urban districts spanning all inhabited continents. Horizontal exposure varied markedly across and within cities, depending on road noise conditions and urban morphology, which also strongly influenced overall attenuation. Some semi-enclosed spaces with low noise levels before low-altitude aircraft operations experienced increases above 20 dB(A), creating pronounced local contrasts in urban noise exposure. Vertically, exposure varied with height and acoustic visibility to the flight route, creating vertical exposure inequalities between storeys, façades and buildings at comparable heights. These findings show that low-altitude aircraft can reshape noise exposure across global cities, requiring route assessment to distinguish newly exposed from already exposed areas and to account for three-dimensional exposure.

eess.AS↗
arXiv2026

1% of Tokens Can Be Enough: On Gradient Estimation in On-Policy Distillation

Sparse on-policy distillation (OPD) allocates teacher supervision to a small subset of tokens in student-generated trajectories. However, useful teacher guidance can yield a noisy update when its gradient is estimated from a sampled next token. We study this estimation problem at a fixed prefix in information geometry and propose an information-efficiency ratio (IER) based on a signal-to-noise decomposition. IER characterizes relative gradient estimation error under an optimal scalar baseline. A candidate-set approximation enables token selection based on IER and its combination with existing usefulness scores, while retaining the sampled reverse-KL training objective. On mathematical and medical reasoning tasks, adding IER improves existing selectors in multiple settings, with sparse configurations matching or exceeding full OPD without token selection at small token budgets of 0.1\%--1\%. These results support accounting for both usefulness and gradient-estimation reliability when allocating sparse supervision. Our code is available at https://github.com/BruceSheng1202/IER-OPD.

cs.LG↗
arXiv2026

Sparse Bayesian Modeling of EEG Channel Interactions Improves P300 Brain-Computer Interface Performance

Electroencephalography (EEG)-based P300 brain-computer interfaces (BCIs) enable communication without physical movement by detecting stimulus-evoked neural responses. Accurate and efficient decoding remains challenging due to high dimensionality, temporal dependence, and complex interactions across EEG channels. Existing approaches often treat channels independently or rely on black-box models, limiting interpretability and personalization. We propose a sparse Bayesian time-varying regression framework that explicitly models pairwise EEG channel interactions while performing temporal feature selection, where a relaxed-thresholded Gaussian process prior induces structured sparsity in both channel-specific and interaction effects, enabling interpretable identification of task-relevant channels and channel pairs. Applied to a public P300 speller dataset of 55 participants, our method achieves a 96.4% median character-level accuracy using all sequence repetitions and attains the best overall decoding performance among competing statistical and deep learning approaches. Incorporating channel interactions yields subgroup-specific gains in character-level accuracy, particularly among participants abstained from alcohol (median 7%, up to 14%). Importantly, our approach improves median BCI-Utility by more than 10% at its optimal operating point, achieving peak throughput after only six sequence repetitions. These results demonstrate that explicitly modeling structured EEG channel interactions within a principled Bayesian framework enhances accuracy and user-centric throughput, and supports personalization in P300 BCI systems.

stat.ME↗
arXiv2026

Contribution of remote bands to orbital magnetization in twisted bilayer graphene

Motivated by recent theoretical and experimental works on orbital magnetization $M_{\mathrm{orb}}$ for the interacting system, we develop a gauge-invariant framework to compute $M_{\mathrm{orb}}$ for correlated phases of magic-angle twisted bilayer graphene within self-consistent Hartree-Fock approximation. Based on the projector formulation of the theory of orbital magnetization, we evaluate both $M_{\mathrm{orb}}$ and the self-rotation contribution $m_{\mathrm{SR}}$ directly from the Hartree-Fock Hamiltonian. We demonstrate that, in contrast to topological invariants such as the Chern number, both $M_{\mathrm{orb}}$ and $m_{\mathrm{SR}}$ obtain substantial contributions from remote bands and thus require careful convergence with respect to the number of included remote bands. Applying this approach to correlated phases at integer fillings, we obtain converged $M_{\mathrm{orb}}$ and $m_{\mathrm{SR}}$ for time reversal symmetry broken Chern insulating states at $ν=\pm3$ and for competing correlated phases at other integer fillings. Our results establish a systematic and controlled approach for evaluating orbital magnetization in correlated moiré systems and clarify the crucial role of remote bands in determining their magnetic response.

cond-mat.str-el↗
arXiv2026

Dissecting Hierarchical Reasoning Models: A Mechanistic Study

We study Hierarchical Reasoning Model (HRM), a representative hierarchical Transformer-based latent reasoning model with many variants, on Sudoku, Maze, and ARC-AGI-2. We mechanistically understand how HRM reasons and what information it encodes. Our analyses compare HRM against Transformer baselines with and without recurrent modules, apply causal interventions on recurrent states, and utilize linear probes against random-direction ablations, as well as sparse autoencoders with feature ablations. Our results reveal several key findings: recurrent models outperform one-pass baselines, while single-state recurrent Transformers are comparable to HRM. State interventions further show that the causal contributions of the high- and low-level states vary across task-specific checkpoints and inference stages. Selected task variables are linearly decodable from the recurrent states in HRM, yet ablating probe directions produce effects comparable to random controls. SAE ablations yield larger behavioral changes than probe-direction ablations. However, top-ranked SAE features show no stable advantage over size-matched random subsets at larger ablation sizes or across tasks; the same pattern persists in a Sudoku control with within-step BPTT. Together, we characterize that HRM is essentially implementing constraint-aware iterative refinement on a puzzle-specific solution state, in which the functional contributions of components at different levels vary without relying on a compact, causally important feature set. These results highlight the necessity of studying the different working mechanisms and the importance of developing mechanistic interpretability techniques better suited for latent-space, recursive reasoning models.

cs.LG↗
arXiv2026

TELEVAL: A Benchmark Designed for Spoken Language Models in Chinese Interactive Scenarios

Spoken Language Models (SLMs) are expected to support natural spoken interaction beyond task completion. However, existing SLM benchmarks primarily evaluate semantic correctness in structured settings and provide limited assessment of interactional behavior grounded in acoustic context. To address this gap, we introduce TELEVAL, a large-scale SLM benchmark for Chinese spoken interaction in instruction-free, audio-conditioned settings. TELEVAL evaluates two complementary aspects: (1) Reliable Content Fulfillment, which measures semantic accuracy of SLMs under diverse acoustic and linguistic conditions, and (2) Interactional Appropriateness, which assesses whether models produce natural and appropriate responses by implicitly grounding behavior in auditory cues. Experiments show that while models perform competitively on semantic tasks, their performance degrades under acoustic variability and in interactional settings. We observe consistent degradation from perceptual instability to interactional errors, and further identify a recurring failure pattern, termed the "Caption Trap", where models tend to describe perceived audio signals rather than produce appropriate interactive responses. These results indicate that current SLMs remain insufficiently aligned with the requirements of natural spoken interaction. TELEVAL provides a targeted framework for evaluating and analyzing interactional behavior in SLMs.

cs.CL↗
arXiv2026

Beyond Observed Auxiliary Relations: Environment-Conditioned Modeling for Multi-Behavior Recommendation

Multi-behavior recommendation (MBR) leverages auxiliary behavioral signals, such as clicks and add-to-cart, to enhance target behavior prediction like purchases. While recent graph neural network-based approaches have achieved strong performance by systematically propagating auxiliary behavior signals, they still suffer from two fundamental challenges inherent to auxiliary behaviors: (1) missing auxiliary signals, which hinder generalization to items without auxiliary observations, and (2) unreliable auxiliary signals, which amplify noise misaligned with the target behavior. To address these challenges in a unified manner, we propose BOAR, an environment-conditioned MBR framework that addresses missing and unreliable auxiliary signals through two complementary modules conditioned on auxiliary observability. Extensive experiments demonstrate that BOAR consistently outperforms state-of-the-art baselines, achieving up to 7.82% gains in HR@10 overall and up to 44.2% gains for target items without auxiliary observations, highlighting its ability to capture hidden preferences beyond observed auxiliary relations. Our code is available at: https://github.com/LSH0411/BOAR.

cs.AI↗
arXiv2026

Mixed Membership Model of Low-rank Matrices with Multimodal Extension

Matrix-valued observations arise in multiplex networks, neuroimaging, and other domains where population-level patterns are often low-rank and subjects may express several latent patterns simultaneously. Existing tensor PCA methods provide continuous subject scores but their loading matrices can be difficult to interpret as population prototypes, while low-rank clustering yields interpretable prototypes with hard labels. We introduce a low-rank mixed membership model for matrix-valued data in which the expected value of each subject's matrix is a convex combination of latent low-rank basis matrices. The model yields both interpretable population-level extreme profiles and continuous subject-level memberships. Our multimodal extension shares memberships across modalities with modality-specific basis matrices and can restore identifiability when one modality is insufficient. We establish identifiability under a pure-subject condition, propose a constrained least-squares estimator and scalable algorithm with spectral initialization and low-rank refinement, and derive nonasymptotic error bounds. The estimator achieves a minimax-optimal reconstruction rate up to a logarithmic factor, with separate basis and membership convergence rates under a geometric condition. Simulations corroborate the theoretical rates and show strong performance. In an analysis of Human Connectome Project functional connectivity data, the proposed method identifies interpretable brain connectivity profiles whose estimated memberships are strongly associated with cognitive phenotypes.

stat.ME↗