Searcharxiv⌕ Search

SEARCH · Searcharxiv

Search Searcharxiv

Search indexed arXiv papers on artificial intelligence, large language models, computer vision and robotics. Read source abstracts and follow links to arXiv.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11Linked to original sources
arXiv2026

MuSP-Bench: Advanced Multimodal Benchmarking of Music Understanding across Score and Performance

Musicians commonly communicate music through scores and performances. Scores encode musical intent, while performances realize it in sound. To investigate whether models can meaningfully engage with both modalities, we introduce MuSP-Bench, a human-authored benchmark of 490 questions targeting understanding across Musical Scores and Performances. The benchmark distinguishes itself by spanning score-based, performance-based, interpretive, and long-horizon reasoning across classical piano and orchestral works. We evaluate frontier multimodal large language models under multiple input conditions. Our results show that these models struggle substantially to understand scores, while facing even greater challenges when reasoning about performance audio. The benchmark is available at https://musp.vaclis.net/.

cs.MM↗
arXiv2026

Multirate State Space Models for End-to-End Processing of Pulse Density Modulated Speech Signals

Deep neural networks (DNNs) based on state-space models (SSMs) are increasingly applied to speech processing, but typically operate on pulse-code-modulated (PCM) audio. This constrains deployment on low-power, always-on edge devices, which commonly use single-bit pulse-density-modulated (PDM) micro-electromechanical (MEMS) microphones for their noise robustness, low cost, and variable sampling rates that enable low-power operation. In fact, converting PDM to PCM requires low-pass filtering and decimation, imposing costly overhead on resource-constrained hardware. While prior works have attempted to process PDM signals directly, they require long training times and generalize poorly across sampling rates. In this paper, we show that the SSM has two key properties that remediate these issues: its continuous-time parametrization allows it to produce a consistent representation of the input audio signal, regardless of the modulation strategy and sampling rate, and its long-term memory enables this representation to be aggressively downsampled without needing any anti-aliasing operations. We then propose a novel end-to-end PDM speech processing architecture that uses an SSM to encode the input audio signal into a modulation- and sampling-rate-invariant latent representation. We show that our proposed architecture achieves robust speech classification and enhancement gains at low-power sampling-rates (512 kHz) and similar performance to state-of-the-art algorithms operating on PCM data when tested on standard PDM sampling-rates of 2 MHz. Moreover, we show that the SSM's output can be downsampled by more than 65,000 times, thus significantly reducing the number of processing timesteps in downstream layers.

cs.SD↗
arXiv2026

Low-Power End-to-End Cochlear Implant Speech Denoising with Spiking Neural Networks

Cochlear implants (CI) restore hearing for individuals with severe to profound hearing loss. However, CI users often struggle to understand speech in noisy environments. Deep neural networks (DNN) have shown promise in enhancing speech for CI users, yet their high energy demands make them non-ideal for low-power CI processors. Spiking neural networks (SNN), on the other hand, offer comparable performance with significantly lower energy consumption. Hence, we propose a novel SNN inspired by the Deep ACE architecture that simultaneously performs speech enhancement and CI coding. Our model achieves competitive vocoded short-time objective intelligibility (VSTOI) and signal-to-noise ratio improvement (SNRi) scores compared to Deep ACE, while achieving more than a sixfold reduction in energy consumption.

cs.SD↗
arXiv2026

Accurate Plate Reverb Parameter Estimation Using Two-Stage Evolutionary Search

We describe our submission to Task A of the 1st DAFx parameter estimation challenge. The task is to recover the six physical parameters of a simulated metal-plate reverberator -- its dimensions and material properties -- from a single impulse response (IR). We treat this as a black-box optimization: candidate parameter sets are fed to the simulator and scored by a loss against the target IR. The method has two stages. The first uses CMA-ES, an evolutionary optimizer, to recover five of the six parameters, comparing IRs under an amplitude-normalized loss. Amplitude normalization makes the search robust but discards the cue to the sixth parameter, the plate's surface density; a second stage therefore estimates it alone, with a ternary search on the un-normalized loss. As the choice of loss strongly affects the search, we select it beforehand, and analyze why compression in the common multi-scale spectral loss degrades recovery. Finally, we test our method on a validation set of 50 IRs, discuss a pathological failure mode, and ablate to justify having two different stages instead of a unified CMA-ES search.

eess.AS↗
arXiv2026

Evoking Harmony via Convolution

I show how to evoke the pitch-class content of a chord from an arbitrary source sound by convolving the source with an impulse response whose grains are one windowed sinusoid per pitch-class, across each octave of hearing range; while, at the same time, minimizing artifacts. A Csound user-defined opcode, chord_convolver, mixes a dry Dirac component into that response, and applies partitioned convolution once. I contrast the effect with a linear-frequency comb filter and with a generic constant-Q resonator bank, and I demonstrate musical use on a twilight field recording alongside the ruins of Chateau de Lagarde.

cs.SD↗
arXiv2026

No Detectable Change in Side-Level WER from Prompt-Level Context: A Preregistered Ablation on a Production Oral-History Corpus

Supplying context at inference time to a large multimodal model is an inexpensive lever for adapting speech transcription to a domain, and earlier results on smaller models reported large gains. This work tested that mechanism where it ships, in the prompt-conditioning layer of a production oral-history transcription tool, on a sample from its own production corpus. Full prompt-level context did not detectably change side-level word error rate (WER), and none of the four preregistered hypotheses was supported. The design was a within-item paired ablation, preregistered with the analysis code frozen by hash before the confirmatory batch was scored; two disclosed gpt-4o pilot sides had been scored earlier, during scorer development. Nineteen cassette sides, about 10.6 hours of degraded 1970s-80s interview audio, were reprocessed through the production code path under three prompt arms, crossed with two deployed commercial configurations, gpt-4o-transcribe and gemini-2.5-flash, and scored against operator-corrected verbatim references. For gpt-4o-transcribe the median paired difference between the full-context and no-context arms was +0.6 WER points, with a side-resampled interval of [-1.1, +1.0]; the Gemini estimates were too unstable to support a comparable negative inference. A post-hoc rerun found run-to-run pipeline variability larger than the confirmatory differences, so effects of that size cannot be resolved from one transcription per cell. An implementation audit verified the manipulation was live, and sequence-alignment analysis found a small improvement on complete context-listed phrases, too small to materially change side-level WER, and for Gemini coexisting with worsened unlisted-token error. Evaluating context mechanisms therefore requires sequence-aligned term-level, insertion, and speaker-label measures alongside aggregate accuracy.

cs.CL↗
arXiv2026

Encore: Infinite Audio-Video Generation with Adaptive Signal Routing

Existing audio-video generation methods produce well-synchronized clips but are limited to short durations, while long-video generation methods extend duration through chunk-based iterative synthesis yet lack audio entirely. Generating long audio-video jointly is fundamentally harder than either task alone: each chunk must simultaneously maintain video temporal coherence, audio temporal coherence, and cross-modal synchronization, whose conditioning signals enter the model through different pathways. In this work, we present Encore for long-form synchronized audio-video generation. Our key insight is to factor this challenge into: (1) local continuity handled by iterative generation with explicit cross-chunk context propagation, and (2) global consistency enforced via reference audio-video signals with shifted position embedding. Building on this design, we propose Adaptive Signal Routing (ASR), which introduces learnable attention biases within self-attention and learnable residual scales on cross-attention outputs, enabling the model to adaptively modulate the influence of each conditioning signal. Trained end-to-end for joint audio-video generation, Encore also supports infinite-length audio-to-video and video-to-audio synthesis at inference by conditioning on the ground-truth modality throughout the denoising process. Experiments on our extended VerseBench for long audio-video evaluation demonstrate that Encore significantly outperforms existing methods in both generation quality and temporal coherence. Code and data for this paper are at https://github.com/shaohua-pan/Encore.

cs.MM↗
arXiv2026

An Attention-Assisted AI Model for Real-Time Underwater Sound Speed Estimation Leveraging Remote Sensing Sea Surface Temperature Data

The estimation of underwater sound velocity distribution serves as a critical basis for facilitating effective underwater communication and precise positioning, given that variations in sound velocity influence the path of signal transmission. Conventional techniques for the direct measurement of sound velocity, as well as methods that involve the inversion of sound velocity utilizing acoustic field data, necessitate on--site data collection. This requirement not only places high demands on device deployment, but also presents challenges in achieving real-time estimation of sound velocity distribution. In order to construct a real-time sound velocity field and eliminate the need for underwater onsite data measurement operations, we propose a self-attention embedded multimodal data fusion convolutional neural network (SA-MDF-CNN) for real-time underwater sound speed profile (SSP) estimation. The proposed model seeks to elucidate the inherent relationship between remote sensing sea surface temperature (SST) data, the primary component characteristics of historical SSPs, and their spatial coordinates. This is achieved by employing CNNs and attention mechanisms to extract local and global correlations from the input data, respectively. The ultimate objective is to facilitate a rapid and precise estimation of sound velocity distribution within a specified task area. The comparative analysis demonstrates that the proposed approach achieves superior performance in terms of both accuracy and stability, exhibiting reduced error rates and enhanced resistance to disturbances when benchmarked against existing advanced techniques.

eess.SP↗
arXiv2026

Automatic Pronunciation Error Detection and Correction of the Holy Quran's Learners Using Deep Learning

Assessing spoken language is challenging, and quantifying pronunciation metrics for machine learning models is even harder. However, for the Holy Quran, this task is enabled by the rigorous recitation rules (Tajweed) established through the efforts of Muslim scholars, making highly effective assessment possible. Despite this advantage, the scarcity of high-quality annotated data remains a significant barrier. In this work, we bridge these gaps by introducing: (1) A 98% automated pipeline to produce high-quality Quranic datasets -- encompassing collection of recitations from expert reciters, segmentation at pause points (waqf) using our fine-tuned wav2vec2-BERT model, transcription of segments, and transcript verification via our novel Tasmeea algorithm; (2) 848 hours of audio (286K annotated utterances); (3) qdat_bench, a benchmark covering phonemes, diacritization, and Tajweed rules (Ghunnah, Qalqalah, Madd) on real recitation errors containing 159 samples; (4) A novel ASR-based approach for pronunciation error detection utilizing our custom Quran Phonetic Script (QPS) to encode Tajweed rules (unlike the IPA standard for Modern Standard Arabic). QPS uses an 11-level script: phoneme level (encoding Arabic letters with short/long vowels) and sifat level (encoding articulation characteristics of every phoneme). We further present comprehensive modeling with our novel multi-level CTC model, which achieved 0.21% and 1.94% average Phoneme Error Rate (PER) on the test set and qdat_bench respectively, with a 75.8% Tajweed F1 score. We release our work as open-source: https://obadx.github.io/quran-muaalem/en/

eess.AS↗
arXiv2026

Evaluating Loss Functions in Differentiable Out-of-Domain Sound-Matching with Partial Parameter Distance

In out-of-domain (OOD) sound-matching, a synthesizer is optimized to mimic a sound it did not generate. OOD evaluation of loss functions is underexplored in part because the standard "parameter loss" metric requires a shared parameter space between target and imitator, which OOD settings lack. We introduce Partial Parameter Distance (PPD), which applies parameter loss only to the critical parameters that mismatched synthesizers share (e.g., filter cutoffs), enabling automatically evaluated OOD experiments; we verify its results with blinded listening tests. Across seven scenarios involving band-pass filtering, amplitude modulation, and pitch-bending, we evaluate four differentiable loss functions (SIMSE_Spec, L1_Spec, JTFS, DTW_Envelope). Loss-function effectiveness remains tightly coupled to the method of synthesis: SIMSE_Spec excels at filter-cutoff recovery, DTW_Envelope at amplitude-modulation recovery, and JTFS at smooth pitch trajectories. Parameter-based evaluation agrees with listening tests on the top-ranked loss function in five of seven scenarios, demonstrating its utility as a diagnostic tool.

cs.SD↗
arXiv2026

A Mixed-Behavior Vote Model for Multimedia Subjective Quality Votes, Means, and Variances

The relationship between subjective test vote variance and vote mean (or MOS) is well-studied, and the mathematically admissible vote variance region has been previously defined. We propose a reduced admissible variance region called the Unimodal Variance Region (UVR) that better describes real subjective rating behavior of multimedia. Further, subjective vote variance is often modeled as parabolic. We explain that, in practice, the parabolic model often violates the admissible region in the variance vs. MOS plane and we propose alternatives that respect the admissible region. We also present a parametrized random process to model votes that mixes voting processes and produces a realistic range of vote variances within the UVR at any desired MOS. This process was inspired by and comports with voting behavior that is observed in many subjective tests. By modeling vote variance from a subjective experiment, this vote model offers additional interpretable insights into voting behavior observed in a given experiment. We present example results from 16 datasets spanning speech, image, and video subjective quality experiments.

cs.MM↗
arXiv2026

CAD: Conflict-Aware Decoding to Mitigate Cross-Modal Hallucinations in Omnimodal Large Language Models

Omnimodal large language models (Omni-LLMs) integrate audio, video, and text, yet remain vulnerable to cross-modal hallucinations, where one modality improperly influences predictions about another. Existing training-free decoders modulate modality influence through perturbation or relevance weighting, but do not assess predictive compatibility within the joint audio-visual branch. Because joint-branch discrepancies may indicate either harmful interference or useful complementarity, reliable intervention requires assessing both discrepancy magnitude and actionability. To this end, we propose Conflict-Aware Decoding (CAD), a training-free framework comprising Potential Conflict Magnitude Estimation (PCME) and Conflict Actionability Assessment (CAA). PCME quantifies potential conflict using audio-video disagreement and the deviation of the joint prediction from a relevance-weighted unimodal reference. CAA then applies Dempster-Shafer reliability discounting to task-space answer relations, using query relevance and answer decisiveness to determine whether intervention is warranted. When an actionable conflict is identified, CAD selectively reallocates decoding weight from the joint branch to the unimodal branches. Experiments on CMM, AVHBench, WorldSense, and VideoMME show that CAD consistently outperforms the base decoder and competitive training-free methods across multiple audio-visual backbones. On Qwen2.5-Omni-7B, CAD improves overall accuracy by 14.1 and 8.0 percentage points on CMM and AVHBench, respectively, without model retraining.

eess.AS↗
arXiv2026

From local kernels to global form: modeling the emergence of musical content

Markov models are established tools for symbolic music, including non-homogeneous formulations. The narrower contribution examined here is an observation-driven estimation mechanism: overlapping sliding windows derive a trajectory of local transition kernels from one symbolic sequence rather than from an exogenous formal partition. We test this mechanism on 273 logical note events from Debussy's Syrinx (1913), using the often-proposed A-B-A' reading as a reference rather than ground truth. We apply the same validation to absolute-pitch and notated-duration kernels. At $L=6$, both reference boundaries attain the Jensen--Shannon maximum in both dimensions; the duration plateau is substantially narrower (64 of 267 comparisons) than the pitch plateau (210 of 267). Because the theoretical maximum for consecutive sliding-window comparisons is set by window geometry and equals $1/\sqrt{L-1}$ for maximal turnover of the entering/leaving transition, the pitch value at $L=6$ and its broad plateau are not, by themselves, strong evidence. Their cross-dimensional alignment is consistent with boundary sensitivity, while the broad plateaus preclude treating either curve alone as a unique automatic segmenter. Five-hundred-draw re-synthesis experiments quantify departure from the source in both dimensions and expose an exact-copy degeneracy at $L=2$.

cs.CL↗
arXiv2026

One Timeline, Many Renderings: A Wolfram Language Paclet for heterogeneous musical output

One algorithmic composition may require a Csound score, engraved notation, real-time control, and a rehearsal click. Authored separately, their timelines drift. Temporal System is a Wolfram Language paclet that instead compiles one immutable store of typed entities on a rational beat timeline through backend-specific contracts. It emits Csound synthesis, beta MusicXML 4.0, OSC control, and click artifacts that remain synchronized because they share that store. Conversion to seconds, samples, or hertz occurs only at render time. Csound notes use stable named instruments in external .orc files; curves become k-rate signals declared against score p-fields. The click backend derives rehearsal audio from the same meter and tempo and reuses the Csound serializer. We describe the temporal, semantic, and rendering-contract layers, their practical trade-offs, and the limits of this proprietary authoring environment within an otherwise open-source ecosystem. The archived supplement exposes the reported outputs pending paclet release.

cs.CL↗
arXiv2026

Arbitrary Polygon Oscillator: Generalizing Polygonal Synthesis to Arbitrary Shapes, Morphing, and Three-Dimensional Polyhedra

Polygonal synthesis generates audio by traversing the perimeter of a polygon with a phasor; prior work uses a constant angular velocity, whereas the proposed system adopts constant arc-length (perimeter) velocity. Existing formulations operate on regular, parametrically defined polygons, producing smooth timbral transitions within a single family of shapes. This paper generalizes polygonal synthesis around a unified arc-length engine: vertex data of any origin feed the same DSP pipeline. First, we adapt the oscillator to accept arbitrary vertex configurations from an external buffer, opening the possibility for a broad class of closed polygons -- regular, irregular, or star-shaped -- to function as a waveform generator. Second, a hybrid interpolation algorithm enables smooth morphing between polygons with unequal vertex counts, passing through intermediate shapes that have no parametric description. Third, we extend the paradigm to three dimensions: a convex polyhedron rotated about three axes is sliced by a fixed horizontal plane, and the resulting cross-section yields a continuously variable polygon controlled by the solid's orientation. The system runs in RNBO (Cycling~'74) with a geometry caching strategy that avoids per-sample recomputation. Antialiasing combines a four-point polyBLAMP correction derived from runtime Bézier tangents with adaptive oversampling, adapting the correction geometrically to general vertex configurations without per-shape analytical derivation.

cs.SD↗
arXiv2026

Effectiveness of IoT and Deep Learning for Detection and Severity Assessment of Postelectrotermes militaris in Tea Plantations

Tea plantations are vulnerable to Postelectrotermes militaris, commonly known as the Upcountry Live Wood Termite (ULWT), which can cause substantial damage when infestations remain undetected. This study proposes an IoT-enabled acoustic monitoring framework integrated with deep learning for early detection and severity assessment of ULWT infestations in tea plantations. Research Method: Audio signals were captured non-invasively from tea trunks using a high-sensitivity microphone connected to a Raspberry Pi-based IoT device, with geographic coordinates recorded for spatial tracking. After trimming, resampling, and segmentation, 2,000 ten-second samples were obtained, comprising 1,000 healthy and 1,000 infested samples, and divided into 1,600 training, 200 validation, and 200 test samples. The dataset used in this study is publicly available on Kaggle (Senevirathna et al. 2026). Fourier-derived spectrograms trained a CNN for infestation classification and probability estimation. A weighted severity model combined CNN probability, mean acoustic amplitude, and nearby infested plants within 5 m, with geospatial mapping used to visualize infestation distribution. Findings and Values: Field trials in a ULWT-affected tea plantation in Pundaluoya demonstrated feasibility under realistic environmental noise. On the held-out test set, the CNN achieved 81.5% accuracy, 80.6% precision, 83.0% recall, 81.8% F1-score, and 0.819 ROC-AUC. Beyond binary infestation detection, the framework introduced quantitative severity assessment using infestation probability, acoustic amplitude, and nearby infested plants. The resulting severity and geospatial outputs can support plantation managers in identifying high-risk areas, prioritizing field inspections, and implementing more timely and targeted control measures.

cs.AI↗
arXiv2026

Neural Multichannel Distant Speaker Diarization and Source Separation with Beta Speaker Activity Prior

Distant speaker diarization remains challenging due to adverse acoustic conditions, varying numbers of speakers and overlapping speech. While data-driven approaches have shown strong performance, model-driven methods offer a compelling alternative by leveraging spatial information from multichannel recordings. This paper is motivated to propose a Bayesian diarization model for a model-driven method called neural FCASA to enhance its robustness. Specifically, we propose a beta prior over speaker activity and hence a variational lower bound objective that can be seen as a regularized continuous speaker activity score in place of the original cross-entropy loss to train the diarization model. Our experiments show significant improvements in terms of Diarization Error Rate by at least 3% (16% relatively) and Jaccard Error Rate by at least 4% (20% relatively) on the AMI dataset compared to the baseline.

cs.SD↗
arXiv2026

Grounded Decoding for Autoregressive Speech Enhancement via Adaptive Code-Space Grounding and Local LLM Refinement

Large language model (LLM)-based autoregressive speech enhancement (SE) produces natural speech using learned clean-speech priors, but may hallucinate content unsupported by the input. Deterministic SE better preserves observation-coupled evidence, yet often retains residual noise or local distortion. We propose an evidence-grounded generative SE framework that uses a deterministic estimate as imperfect evidence. A Whisper-guided DPRNN produces an enhanced waveform, which is blended with the observation and tokenized into a discrete evidence sequence. The evidence conditions an autoregressive clean-speech token generator and is reused during decoding through Code-Space Grounding (CSG), which penalizes candidates according to their Hamming distance in the factorized finite-scalar-quantized (FSQ) space. Because the appropriate grounding strength depends on acoustic difficulty, we introduce SNR-Conditioned CSG (SNR-CSG), which maps a calibrated residual-SNR estimate to an utterance-level strength and constructs an adaptive grounded anchor. Although grounding improves content fidelity, the anchor may retain local acoustic defects inherited from the evidence. Since such defects are predominantly local in the FSQ space, nearby tokens may provide better acoustic realizations without large departures from the observation-supported trajectory. We therefore propose Grounded Neighborhood Refinement with LLM ranking (GNR-LLM). It performs one additional teacher-forced pass conditioned on the grounded-anchor history, intersects the LLM top-$K$ candidates with a local FSQ Hamming neighborhood. Experiments on in-domain, controlled-SNR, and DNS no-reverb conditions show that SNR-CSG provides robust automatic grounding, while GNR-LLM substantially improves low-SNR perceptual quality without sacrificing content fidelity.

cs.SD↗