arXiv subjects

Jinbo Hu

Publications and source records attributed to Jinbo Hu.

16 recordsLinked to original sources

arXiv2026

CoSTALA: Compositional Spatio-Temporal Audio-Language Alignment via Multi-Grain Hierarchical Contrastive Learning

Conventional audio language models (ALMs) have made significant progress in achieving alignment between auditory and textual representations, including recent explorations in spatial audio. However, in daily spatial scenarios, they still cannot effectively process multi-event audio sequences. Current approaches primarily rely on coarse-grained contrastive learning with global auditory and textual features, lacking the resolution to distinguish multiple sequential events. To overcome these limitations, we propose CoSTALA-a novel training paradigm that transitions from purely global alignment to fine-grained spatio-temporal reasoning. By constructing a multi-granularity hierarchical loss function system, we achieve explicit modeling of temporal dependencies, and successfully anchors individual acoustic events to preserve their semantic purity. Extensive experiments demonstrate that CoSTALA significantly establish a powerful new framework for spatio-temporal audio understanding.

cs.SD↗

arXiv2026

Optical pumping of alkali-metal vapor in the quasi-high-pressure regime

Optical pumping is fundamental to high-precision measurement using thermal alkali-metal atoms in vapor cells. In applications such as atomic magnetometry, buffer gases (e.g., $\mathrm{N}_2$ or $\mathrm{He}$) at specific pressures are introduced to quench fluorescence and mitigate wall relaxation. In the high-pressure limit (e.g., the $\mathrm{N}_2$ pressure $p_{\mathrm{N}_2}> 1$~atm), where collisional broadening exceeds hyperfine splittings of the atoms, optical pumping theory provides a clear description of the angular momentum exchange between photons and atomic spins. However, in many magnetic sensing scenarios, the high-pressure approximation becomes inadequate as its pressure conditions are not strictly satisfied. Consequently, an explicit description of optical pumping under realistic pressures is critical for selecting operating points and enhancing system performance. To address this, we develop a unified theoretical framework of optical pumping in the quasi-high-pressure regime, where collisional broadening is comparable to the ground-state hyperfine splitting. We demonstrate that light absorption, spin polarization, and magnetic-resonance linewidth in this regime differ significantly from those predicted by the high-pressure limit and offer favorable operating conditions. Our study extends conventional modeling and offers critical guidance for atomic magnetometry operating under realistic buffer gas pressures.

physics.atom-ph↗

arXiv2025

SciAgent: A Unified Multi-Agent System for Generalistic Scientific Reasoning

Recent advances in large language models have enabled AI systems to achieve expert-level performance on domain-specific scientific tasks, yet these systems remain narrow and handcrafted. We introduce SciAgent, a unified multi-agent system designed for generalistic scientific reasoning-the ability to adapt reasoning strategies across disciplines and difficulty levels. SciAgent organizes problem solving as a hierarchical process: a Coordinator Agent interprets each problem's domain and complexity, dynamically orchestrating specialized Worker Systems, each composed of interacting reasoning Sub-agents for symbolic deduction, conceptual modeling, numerical computation, and verification. These agents collaboratively assemble and refine reasoning pipelines tailored to each task. Across mathematics and physics Olympiads (IMO, IMC, IPhO, CPhO), SciAgent consistently attains or surpasses human gold-medalist performance, demonstrating both domain generality and reasoning adaptability. Additionally, SciAgent has been tested on the International Chemistry Olympiad (IChO) and selected problems from the Humanity's Last Exam (HLE) benchmark, further confirming the system's ability to generalize across diverse scientific domains. This work establishes SciAgent as a concrete step toward generalistic scientific intelligence-AI systems capable of coherent, cross-disciplinary reasoning at expert levels.

cs.AI↗

arXiv2025

MorphoBench: A Benchmark with Difficulty Adaptive to Model Reasoning

With the advancement of powerful large-scale reasoning models, effectively evaluating the reasoning capabilities of these models has become increasingly important. However, existing benchmarks designed to assess the reasoning abilities of large models tend to be limited in scope and lack the flexibility to adapt their difficulty according to the evolving reasoning capacities of the models. To address this, we propose MorphoBench, a benchmark that incorporates multidisciplinary questions to evaluate the reasoning capabilities of large models and can adjust and update question difficulty based on the reasoning abilities of advanced models. Specifically, we curate the benchmark by selecting and collecting complex reasoning questions from existing benchmarks and sources such as Olympiad-level competitions. Additionally, MorphoBench adaptively modifies the analytical challenge of questions by leveraging key statements generated during the model's reasoning process. Furthermore, it includes questions generated using simulation software, enabling dynamic adjustment of benchmark difficulty with minimal resource consumption. We have gathered over 1,300 test questions and iteratively adjusted the difficulty of MorphoBench based on the reasoning capabilities of models such as o3 and GPT-5. MorphoBench enhances the comprehensiveness and validity of model reasoning evaluation, providing reliable guidance for improving both the reasoning abilities and scientific robustness of large models. The code has been released in https://github.com/OpenDCAI/MorphoBench.

cs.AI↗

arXiv2025

SALM: Spatial Audio Language Model with Structured Embeddings for Understanding and Editing

Spatial audio understanding is essential for accurately perceiving and interpreting acoustic environments. However, existing audio-language models exhibit limitations in processing spatial audio and perceiving spatial acoustic scenes. To address this gap, we propose the Spatial Audio Language Model (SALM), a novel framework that bridges spatial audio and language through multi-modal contrastive learning. SALM integrates a text encoder with a dual-branch audio encoder that decomposes spatial sound into semantic and spatial components via structured audio embeddings. Key features of SALM include seamless alignment between spatial audio and natural language, both separate and joint extraction of spatial and semantic representations, zero-shot direction classification, and flexible support for spatial audio editing. Experimental results demonstrate that SALM effectively captures and aligns cross-modal representations, yielding well-structured audio embeddings. Furthermore, SALM enables advanced editing capabilities, such as modifying directional audio using text-based embeddings.

cs.SD↗

arXiv2025

Radial Attention: $O(n\log n)$ Sparse Attention with Energy Decay for Long Video Generation

Recent advances in diffusion models have enabled high-quality video generation, but the additional temporal dimension significantly increases computational costs, making training and inference on long videos prohibitively expensive. In this paper, we identify a phenomenon we term Spatiotemporal Energy Decay in video diffusion models: post-softmax attention scores diminish as spatial and temporal distance between tokens increase, akin to the physical decay of signal or waves over space and time in nature. Motivated by this, we propose Radial Attention, a scalable sparse attention mechanism with $\mathcal{O}(n \log n)$ complexity that translates energy decay into exponentially decaying compute density, which is significantly more efficient than standard $\mathcal{O}(n^2)$ dense attention and more expressive than linear attention. Specifically, Radial Attention employs a simple, static attention mask where each token attends to spatially nearby tokens, with the attention window size shrinking with temporal distance. Moreover, it allows pre-trained video diffusion models to extend their generation length with efficient LoRA-based fine-tuning. Extensive experiments show that Radial Attention maintains video quality across Wan2.1-14B, HunyuanVideo, and Mochi 1, achieving up to a 1.9$\times$ speedup over the original dense attention. With minimal tuning, it enables video generation up to 4$\times$ longer while reducing training costs by up to 4.4$\times$ compared to direct fine-tuning and accelerating inference by up to 3.7$\times$ compared to dense attention inference. Code is released at \href{https://github.com/mit-han-lab/radial-attention}{https://github.com/mit-han-lab/radial-attention}.

cs.CV↗

arXiv2025

Understanding and Optimizing Agentic Workflows via Shapley value

Agentic workflows have become the dominant paradigm for building complex AI systems, orchestrating specialized components, such as planning, reasoning, action execution, and reflection, to tackle sophisticated real-world tasks. However, systematically analyzing and optimizing these workflows remains challenging due to intricate component interdependencies and the lack of principled attribution methods. In this work, we introduce ShapleyFlow, the first framework that employs cooperative game theory to analyze and optimize agentic workflows. By applying the Shapley value to evaluate all possible component configurations, ShapleyFlow enables fine-grained attribution of each component's contribution and facilitates the identification of task-specific optimal configurations. Through a constructed dataset evaluated across 7 scenarios, such as navigation, math and OS, we demonstrate 3 key contributions: (1) Theoretical Framework: a principled game-theoretic approach for the attribution of contributions in agentic workflows. (2) Optimal Workflow Discovery: ShapleyFlow identifies task-specific component configurations that consistently outperform workflows relying on a single LLM across all tested tasks. (3) Comprehensive Analysis: we construct and analyze over 1,500 tasks, providing actionable insights and design guidelines for optimizing workflows across multiple domains.

cs.AI↗

arXiv2024

PSELDNets: Pre-trained Neural Networks on a Large-scale Synthetic Dataset for Sound Event Localization and Detection

Sound event localization and detection (SELD) has seen substantial advancements through learning-based methods. These systems, typically trained from scratch on specific datasets, have shown considerable generalization capabilities. Recently, deep neural networks trained on large-scale datasets have achieved remarkable success in the sound event classification (SEC) field, prompting an open question of whether these advances can be extended to the development of SELD foundation models. In this paper, leveraging the power of pre-trained SEC models, we propose pre-trained SELD networks (PSELDNets) on a large-scale synthetic dataset. The synthetic dataset, generated by convolving sound events with simulated spatial room impulse responses (SRIRs), contains 1,167 hours of audio clips with an ontology of 170 sound classes. These PSELDNets are applied to various SELD scenarios. When we adapt PSELDNets to specific scenarios, particularly in cases of low-resource data, we introduce a data-efficient fine-tuning method, AdapterBit. PSELDNets are evaluated on synthetic-test-set using collected SRIRs from the TAU Spatial Room Impulse Response Database (TAU-SRIR DB) and achieve satisfactory performance. We also carried out experiments to validate the transferability of PSELDNets to three publicly available datasets and our own real-world recordings. The results demonstrate that PSELDNets surpass state-of-the-art systems across all publicly available datasets. Given the need for direction-of-arrival estimation, SELD generally relies on sufficient multi-channel audio clips. However, incorporating the AdapterBit, PSELDNets show more efficient adaptability to various scenarios using minimal multi-channel or even just monophonic audio clips, outperforming traditional fine-tuning approaches.

eess.AS↗

arXiv2024

Experimental verification of the optimal fingerprint method in a Spin Resonance System: Implications for Complex Systems

The optimal fingerprint method (OFM) serves as a potent approach for detecting and attributing climate change. However, direct experimental validation remains challenging due to the system's inherent complexity. Here, we experimentally validate this method using a precisely controlled magnetic resonance system of spins, which serves as a minimal physical analog of forced noise response. Based on linear response theory (LRT), we derived the system's Green's function from spin projection noise measurements and successfully applied it to attribute the magnetic fields, yielding excellent agreement with predictions. Furthermore, our measurements confirm the existence of an optimal detection direction that maximizes the signal-to-noise ratio, a key theoretical prediction underlying the OFM. This work serves as a laboratory demonstration of LRT and OFM in detection and attribution (DA) studies, aiming to connect theoretical models with experimental observations. These findings may provide useful references for climate change science and its broader interdisciplinary applications. in ecosystems, finance, social sciences, quantum sensing, and beyond.

physics.ao-ph↗

arXiv2024

Frequency Shift Caused by Nonuniform Field and Boundary Relaxation in Magnetic Resonance and Comagnetometers

In magnetic resonance experiments, it is widely recognized that a nonuniform magnetic field can lead to an increase in the resonance line width, as well as a reduction in sensitivity and spectral resolution. However, a nonuniform magnetic field can also cause shifts in resonance frequency, which has received far less attention. In this work, we investigate the frequency shift caused by boundary relaxation and nonuniform magnetic field with arbitrary spatial distribution. We find that this frequency shift is spin-species dependent, implying a systematic error in NMR gyroscope and comagnetometers. The first order correction to this systematic error is proportional to the difference of boundary relaxation rate, and dominates for small cells. In contrast, the third and higher order corrections arise from the difference of gyromagnetic ratios of spin species, and dominates for large cells. This insight helps understanding the unexplained isotope shifts in recent NMR gyroscope and new physics searching experiments that utilize comagnetometers. Finally, we propose a tool for wall interaction research based on the frequency shift's dependency on boundary relaxation.

quant-ph↗

arXiv2023

Selective-Memory Meta-Learning with Environment Representations for Sound Event Localization and Detection

Environment shifts and conflicts present significant challenges for learning-based sound event localization and detection (SELD) methods. SELD systems, when trained in particular acoustic settings, often show restricted generalization capabilities for diverse acoustic environments. Furthermore, obtaining annotated samples for spatial sound events is notably costly. Deploying a SELD system in a new environment requires extensive time for re-training and fine-tuning. To overcome these challenges, we propose environment-adaptive Meta-SELD, designed for efficient adaptation to new environments using minimal data. Our method specifically utilizes computationally synthesized spatial data and employs Model-Agnostic Meta-Learning (MAML) on a pre-trained, environment-independent model. The method then utilizes fast adaptation to unseen real-world environments using limited samples from the respective environments. Inspired by the Learning-to-Forget approach, we introduce the concept of selective memory as a strategy for resolving conflicts across environments. This approach involves selectively memorizing target-environment-relevant information and adapting to the new environments through the selective attenuation of model parameters. In addition, we introduce environment representations to characterize different acoustic settings, enhancing the adaptability of our attenuation approach to various environments. We evaluate our proposed method on the development set of the Sony-TAu Realistic Spatial Soundscapes 2023 (STARSS23) dataset and computationally synthesized scenes. Experimental results demonstrate the superior performance of the proposed method compared to conventional supervised learning methods, particularly in localization.

eess.AS↗

arXiv2023

META-SELD: Meta-Learning for Fast Adaptation to the new environment in Sound Event Localization and Detection

For learning-based sound event localization and detection (SELD) methods, different acoustic environments in the training and test sets may result in large performance differences in the validation and evaluation stages. Different environments, such as different sizes of rooms, different reverberation times, and different background noise, may be reasons for a learning-based system to fail. On the other hand, acquiring annotated spatial sound event samples, which include onset and offset time stamps, class types of sound events, and direction-of-arrival (DOA) of sound sources is very expensive. In addition, deploying a SELD system in a new environment often poses challenges due to time-consuming training and fine-tuning processes. To address these issues, we propose Meta-SELD, which applies meta-learning methods to achieve fast adaptation to new environments. More specifically, based on Model Agnostic Meta-Learning (MAML), the proposed Meta-SELD aims to find good meta-initialized parameters to adapt to new environments with only a small number of samples and parameter updating iterations. We can then quickly adapt the meta-trained SELD model to unseen environments. Our experiments compare fine-tuning methods from pre-trained SELD models with our Meta-SELD on the Sony-TAU Realistic Spatial Soundscapes 2023 (STARSSS23) dataset. The evaluation results demonstrate the effectiveness of Meta-SELD when adapting to new environments.

eess.AS↗

arXiv2023

Stability Improvement of Nuclear Magnetic Resonance Gyroscope with Self-Calibrating Parametric Magnetometer

In this paper, we study the stability of nuclear magnetic resonance gyroscope (NMRG), which employs Xe nuclear spins to measure inertial rotation rate. The Xe spin polarization is sensed by an in-situ Rb-magnetometer. The Rb-magnetometer works in a parametric oscillation mode (henceforth referred to as the Rb parametric magnetometer, or Rb-PM), in which the Larmor frequency of the Rb spins is modulated and the transverse components of Xe nuclear spin polarization are measured. As the measurement output of the Rb-PM, the phase of the Xe nuclear spin precession is eventually converted to the Xe nuclear magnetic resonance (NMR) frequencies and the inertial rotation rate. Here we provide a comprehensive study of the NMR phase measured by the Rb-PM, and analyze the influence of various control parameters, including the DC magnetic field, the frequency and phase of the modulation field, and the Rb resonance linewidth, on the stability of the NMR phase. Based on these analysis, we propose and implement a self-calibrating method to compensate the NMR phase drift during the Rb-PM measurement. With the self-calibrating Rb-PM, we demonstrate a significant improvement of the bias stability of NMRG.

physics.atom-ph↗

arXiv2022

Stable Atomic Magnetometer in Parity-Time Symmetry Broken Phase

Random motion of spins is usually detrimental in magnetic resonance experiments. The spin diffusion in non-uniform magnetic fields causes broadening of the resonance and limits the sensitivity and the spectral resolution in applications like magnetic resonance spectroscopy. Here, by observation of the parity-time ($PT$) phase transition of diffusive spins in gradient magnetic fields, we show that the spatial degrees of freedom of atoms could become a resource, rather than harmfulness, for high-precision measurement of weak signals. In the normal phase with zero or low gradient fields, the diffusion results in dissipation of spin precession. However, by increasing the field gradient, the spin system undergoes a $PT$ transition, and enters the $PT$ symmetry broken phase. In this novel phase, the spin precession frequency splits due to spatial localization of the eigenmodes. We demonstrate that, using these spatial-motion-induced split frequencies, the spin system can serve as a stable magnetometer, whose output is insensitive to the inevitable long-term drift of control parameters. This opens a door to detect extremely weak signals in imperfectly controlled environment.

quant-ph↗

arXiv2022

Sound Event Localization and Detection for Real Spatial Sound Scenes: Event-Independent Network and Data Augmentation Chains

Sound event localization and detection (SELD) is a joint task of sound event detection and direction-of-arrival estimation. In DCASE 2022 Task 3, types of data transform from computationally generated spatial recordings to recordings of real-sound scenes. Our system submitted to the DCASE 2022 Task 3 is based on our previous proposed Event-Independent Network V2 (EINV2) with a novel data augmentation method. Our method employs EINV2 with a track-wise output format, permutation-invariant training, and a soft parameter-sharing strategy, to detect different sound events of the same class but in different locations. The Conformer structure is used for extending EINV2 to learn local and global features. A data augmentation method, which contains several data augmentation chains composed of stochastic combinations of several different data augmentation operations, is utilized to generalize the model. To mitigate the lack of real-scene recordings in the development dataset and the presence of sound events being unbalanced, we exploit FSD50K, AudioSet, and TAU Spatial Room Impulse Response Database (TAU-SRIR DB) to generate simulated datasets for training. We present results on the validation set of Sony-TAu Realistic Spatial Soundscapes 2022 (STARSS22) in detail. Experimental results indicate that the ability to generalize to different environments and unbalanced performance among different classes are two main challenges. We evaluate our proposed method in Task 3 of the DCASE 2022 challenge and obtain the second rank in the teams ranking. Source code is released.

eess.AS↗

arXiv2022

A Track-Wise Ensemble Event Independent Network for Polyphonic Sound Event Localization and Detection

Polyphonic sound event localization and detection (SELD) aims at detecting types of sound events with corresponding temporal activities and spatial locations. In this paper, a track-wise ensemble event independent network with a novel data augmentation method is proposed. The proposed model is based on our previous proposed Event-Independent Network V2 and is extended by conformer blocks and dense blocks. The track-wise ensemble model with track-wise output format is proposed to solve an ensemble model problem for track-wise output format that track permutation may occur among different models. The data augmentation approach contains several data augmentation chains, which are composed of random combinations of several data augmentation operations. The method also utilizes log-mel spectrograms, intensity vectors, and Spatial Cues-Augmented Log-Spectrogram (SALSA) for different models. We evaluate our proposed method in the Task of the L3DAS22 challenge and obtain the top ranking solution with a location-dependent F-score to be 0.699. Source code is released.

cs.SD↗