arXiv subjects

Vipul Arora

Publications and source records attributed to Vipul Arora.

At least 37 records · Page 2Linked to original sources

arXiv2026

Learning from Limited Labels: Transductive Graph Label Propagation for Indian Music Analysis

Supervised machine learning frameworks rely on extensive labeled datasets for robust performance on real-world tasks. However, there is a lack of large annotated datasets in audio and music domains, as annotating such recordings is resource-intensive, laborious, and often require expert domain knowledge. In this work, we explore the use of label propagation (LP), a graph-based semi-supervised learning technique, for automatically labeling the unlabeled set in an unsupervised manner. By constructing a similarity graph over audio embeddings, we propagate limited label information from a small annotated subset to a larger unlabeled corpus in a transductive, semi-supervised setting. We apply this method to two tasks in Indian Art Music (IAM): Raga identification and Instrument classification. For both these tasks, we integrate multiple public datasets along with additional recordings we acquire from Prasar Bharati Archives to perform LP. Our experiments demonstrate that LP significantly reduces labeling overhead and produces higher-quality annotations compared to conventional baseline methods, including those based on pretrained inductive models. These results highlight the potential of graph-based semi-supervised learning to democratize data annotation and accelerate progress in music information retrieval.

eess.AS↗

arXiv2025

Continual Learning for Singing Voice Separation with Human in the Loop Adaptation

Deep learning-based works for singing voice separation have performed exceptionally well in the recent past. However, most of these works do not focus on allowing users to interact with the model to improve performance. This can be crucial when deploying the model in real-world scenarios where music tracks can vary from the original training data in both genre and instruments. In this paper, we present a deep learning-based interactive continual learning framework for singing voice separation that allows users to fine-tune the vocal separation model to conform it to new target songs. We use a U-Net-based base model architecture that produces a mask for separating vocals from the spectrogram, followed by a human-in-the-loop task where the user provides feedback by marking a few false positives, i.e., regions in the extracted vocals that should have been silence. We propose two continual learning algorithms. Experiments substantiate the improvement in singing voice separation performance by the proposed algorithms over the base model in intra-dataset and inter-dataset settings.

cs.SD↗

arXiv2025

Uncertainty Quantification in Melody Estimation using Histogram Representation

Confidence estimation can improve the reliability of melody estimation by indicating which predictions are likely incorrect. The existing classification-based approach provides confidence for predicted pitch classes but fails to capture the magnitude of deviation from the ground truth. To address this limitation, we reformulate melody estimation as a regression problem and propose a novel approach to estimate uncertainty directly from the histogram representation of the pitch values, which correlates well with the deviation between the prediction and the ground-truth. We design three methods to model pitch on a continuous support range of histogram, which introduces the challenge of handling the discontinuity of unvoiced from the voiced pitch values. The first two methods address the abrupt discontinuity by mapping the pitch values to a continuous range, while the third adopts a fully Bayesian formulation, which models voicing detection as a classification and voiced pitch estimation as a regression task. Experimental results demonstrate that regression-based formulations yield more reliable uncertainty estimates compared to classification-based approaches in identifying incorrect pitch predictions. Comparing the proposed methods with a state-of-the-art regression model, it is observed that the Bayesian method performs the best at estimating both the melody and its associated uncertainty.

eess.AS↗

arXiv2025

AudioNet: Supervised Deep Hashing for Retrieval of Similar Audio Events

This work presents a supervised deep hashing method for retrieving similar audio events. The proposed method, named AudioNet, is a deep-learning-based system for efficient hashing and retrieval of similar audio events using an audio example as a query. AudioNet achieves high retrieval performance on multiple standard datasets by generating binary hash codes for similar audio events, setting new benchmarks in the field, and highlighting its efficacy and effectiveness compare to other hashing methods. Through comprehensive experiments on standard datasets, our research represents a pioneering effort in evaluating the retrieval performance of similar audio events. A novel loss function is proposed which incorporates weighted contrastive and weighted pairwise loss along with hashcode balancing to improve the efficiency of audio event retrieval. The method adopts discrete gradient propagation, which allows gradients to be propagated through discrete variables during backpropagation. This enables the network to optimize the discrete hash codes using standard gradient-based optimization algorithms, which are typically used for continuous variables. The proposed method showcases promising retrieval performance, as evidenced by the experimental results, even when dealing with imbalanced datasets. The systematic analysis conducted in this study further supports the significant benefits of the proposed method in retrieval performance across multiple datasets. The findings presented in this work establish a baseline for future studies on the efficient retrieval of similar audio events using deep audio embeddings.

eess.AS↗

arXiv2025

Improving Active Learning for Melody Estimation by Disentangling Uncertainties

Estimating the fundamental frequency, or melody, is a core task in Music Information Retrieval (MIR). Various studies have explored signal processing, machine learning, and deep-learning-based approaches, with a very recent focus on utilizing uncertainty in active learning settings for melody estimation. However, these approaches do not investigate the relative effectiveness of different uncertainties. In this work, we follow a framework that disentangles aleatoric and epistemic uncertainties to guide active learning for melody estimation. Trained on a source dataset, our model adapts to new domains using only a small number of labeled samples. Experimental results demonstrate that epistemic uncertainty is more reliable for domain adaptation with reduced labeling effort as compared to aleatoric uncertainty.

eess.AS↗

arXiv2025

SyncNet: correlating objective for time delay estimation in audio signals

This study addresses the task of performing robust and reliable time-delay estimation in signals in noisy and reverberating environments. In contrast to the popular signal processing based methods, this paper proposes to transform the input signals using a deep neural network into another pair of sequences which show high cross correlation at the actual time delay. This is achieved with the help of a novel correlation function based objective function for training the network. The proposed approach is also intrinsically interpretable as it does not lose temporal information. Experimental evaluations are performed for estimating mutual time delays for different types of audio signals such as pulse, speech and musical beats. SyncNet outperforms other classical approaches, such as GCC-PHAT, and some other learning based approaches.

eess.AS↗

arXiv2025

Dementia Detection using Multi-modal Methods on Audio Data

Dementia is a neurodegenerative disease that causes gradual cognitive impairment, which is very common in the world and undergoes a lot of research every year to prevent and cure it. It severely impacts the patient's ability to remember events and communicate clearly, where most variations of it have no known cure, but early detection can help alleviate symptoms before they become worse. One of the main symptoms of dementia is difficulty in expressing ideas through speech. This paper attempts to talk about a model developed to predict the onset of the disease using audio recordings from patients. An ASR-based model was developed that generates transcripts from the audio files using Whisper model and then applies RoBERTa regression model to generate an MMSE score for the patient. This score can be used to predict the extent to which the cognitive ability of a patient has been affected. We use the PROCESS_V1 dataset for this task, which is introduced through the PROCESS Grand Challenge 2025. The model achieved an RMSE score of 2.6911 which is around 10 percent lower than the described baseline.

cs.LG↗

arXiv2025

Identification and Clustering of Unseen Ragas in Indian Art Music

Raga classification in Indian Art Music is an open-set problem where unseen classes may appear during testing. However, traditional approaches often treat it as a closed set problem, rejecting the possibility of encountering unseen classes. In this work, we try to tackle this problem by first employing an Uncertainty-based Out-Of-Distribution (OOD) detection, given a set containing known and unknown classes. Next, for the audio samples identified as OOD, we employ Novel Class Discovery (NCD) approach to cluster them into distinct unseen Raga classes. We achieve this by harnessing information from labelled data and further applying contrastive learning on unlabelled data. With thorough analysis, we demonstrate the influence of different components of the loss function on clustering performance and examine how varying openness affects the NCD task in hand.

eess.AS↗

arXiv2025

Meta-learning-based percussion transcription and $t\bar{a}la$ identification from low-resource audio

This study introduces a meta-learning-based approach for low-resource Tabla Stroke Transcription (TST) and $t\bar{a}la$ identification in Hindustani classical music. Using Model-Agnostic Meta-Learning (MAML), we address the challenges of limited annotated datasets and label heterogeneity, enabling rapid adaptation to new tasks with minimal data. The method is validated across various datasets, including tabla solo and concert recordings, demonstrating robustness in polyphonic audio scenarios. We propose two novel $t\bar{a}la$ identification techniques based on stroke sequences and rhythmic patterns. Additionally, the approach proves effective for Automatic Drum Transcription (ADT), showcasing its flexibility for Indian and Western percussion music. Experimental results show that the proposed method outperforms existing techniques in low-resource settings, significantly contributing to music transcription and studying musical traditions through computational tools.

eess.AS↗

arXiv2025

H-QuEST: Accelerating Query-by-Example Spoken Term Detection with Hierarchical Indexing

Query-by-example spoken term detection (QbE-STD) searches for matching words or phrases in an audio dataset using a sample spoken query. When annotated data is limited or unavailable, QbE-STD is often done using template matching methods like dynamic time warping (DTW), which are computationally expensive and do not scale well. To address this, we propose H-QuEST (Hierarchical Query-by-Example Spoken Term Detection), a novel framework that accelerates spoken term retrieval by utilizing Term Frequency and Inverse Document Frequency (TF-IDF)-based sparse representations obtained through advanced audio representation learning techniques and Hierarchical Navigable Small World (HNSW) indexing with further refinement. Experimental results show that H-QuEST delivers substantial improvements in retrieval speed without sacrificing accuracy compared to existing methods.

eess.AS↗

arXiv2025

Recognizing Ornaments in Vocal Indian Art Music with Active Annotation

Ornamentations, embellishments, or microtonal inflections are essential to melodic expression across many musical traditions, adding depth, nuance, and emotional impact to performances. Recognizing ornamentations in singing voices is key to MIR, with potential applications in music pedagogy, singer identification, genre classification, and controlled singing voice generation. However, the lack of annotated datasets and specialized modeling approaches remains a major obstacle for progress in this research area. In this work, we introduce Rāga Ornamentation Detection (ROD), a novel dataset comprising Indian classical music recordings curated by expert musicians. The dataset is annotated using a custom Human-in-the-Loop tool for six vocal ornaments marked as event-based labels. Using this dataset, we develop an ornamentation detection model based on deep time-series analysis, preserving ornament boundaries during the chunking of long audio recordings. We conduct experiments using different train-test configurations within the ROD dataset and also evaluate our approach on a separate, manually annotated dataset of Indian classical concert recordings. Our experimental results support the superior performance of our proposed approach over the baseline CRNN.

eess.AS↗

arXiv2025

Improvement of Heatbath Algorithm in LFT using Generative models

The Heatbath Algorithm is commonly used for sampling in local lattice field theories, but performing exact updates or sampling from the local density is challenging when dealing with continuous variables. Heatbath methods rely on rejection-based sampling at each site, which can suffer from low acceptance rates if the proposal distribution is not optimally chosen a nontrivial task. In this work, we propose a novel, straightforward approach for generating proposals at each lattice site for the phi4 and XY models using generative AI models. This method learns a conditional local distribution, without requiring training samples from the target, conditioned on both neighboring sites and action parameter values.

physics.comp-ph↗

arXiv2025

$T\bar{a}laGen:$ A System for Automatic $T\bar{a}la$ Identification and Generation

In Hindustani classical music, the tabla plays an important role as a rhythmic backbone and accompaniment. In applications like computer-based music analysis, learning singing, and learning musical instruments, tabla stroke transcription, $t\bar{a}la$ identification, and generation are crucial. This paper proposes a comprehensive system aimed at addressing these challenges. For tabla stroke transcription, we propose a novel approach based on model-agnostic meta-learning (MAML) that facilitates the accurate identification of tabla strokes using minimal data. Leveraging these transcriptions, the system introduces two novel $t\bar{a}la$ identification methods based on the sequence analysis of tabla strokes. \par Furthermore, the paper proposes a framework for $t\bar{a}la$ generation to bridge traditional and modern learning methods. This framework utilizes finite state transducers (FST) and linear time-invariant (LTI) filters to generate $t\bar{a}las$ with real-time tempo control through user interaction, enhancing practice sessions and musical education. Experimental evaluations on tabla solo and concert datasets demonstrate the system's exceptional performance on real-world data and its ability to outperform existing methods. Additionally, the proposed $t\bar{a}la$ identification methods surpass state-of-the-art techniques. The contributions of this paper include a combined approach to tabla stroke transcription, innovative $t\bar{a}la$ identification techniques, and a robust framework for $t\bar{a}la$ generation that handles the rhythmic complexities of Hindustani music.

eess.AS↗

arXiv2024

BEST-STD: Bidirectional Mamba-Enhanced Speech Tokenization for Spoken Term Detection

Spoken term detection (STD) is often hindered by reliance on frame-level features and the computationally intensive DTW-based template matching, limiting its practicality. To address these challenges, we propose a novel approach that encodes speech into discrete, speaker-agnostic semantic tokens. This facilitates fast retrieval using text-based search algorithms and effectively handles out-of-vocabulary terms. Our approach focuses on generating consistent token sequences across varying utterances of the same term. We also propose a bidirectional state space modeling within the Mamba encoder, trained in a self-supervised learning framework, to learn contextual frame-level features that are further encoded into discrete tokens. Our analysis shows that our speech tokens exhibit greater speaker invariance than those from existing tokenizers, making them more suitable for STD tasks. Empirical evaluation on LibriSpeech and TIMIT databases indicates that our method outperforms existing STD baselines while being more efficient.

eess.AS↗

arXiv2024

Explainable Deep Learning Analysis for Raga Identification in Indian Art Music

Raga identification is an important problem within the domain of Indian Art music, as Ragas are fundamental to its composition and performance, playing a crucial role in music retrieval, preservation, and education. Few studies that have explored this task employ approaches such as signal processing, Machine Learning (ML), and more recently, Deep Learning (DL) based methods. However, a key question remains unanswered in all these works: do these ML/DL methods learn and interpret Ragas in a manner similar to human experts? Besides, a significant roadblock in this research is the unavailability of an ample supply of rich, labeled datasets, which drives these ML/DL-based methods. In this paper, firstly we curate a dataset comprising 191 hours of Hindustani Classical Music (HCM) recordings, annotate it for Raga and tonic labels, and train a CNN-LSTM model for the task of Automatic Raga Identification (ARI). We achieve a chunk-wise f1-measure of 0.89 for a subset of 12 Raga classes. Following this, we make one of the first attempts to employ model explainability techniques: SoundLIME and GradCAM++ for Raga identification, to evaluate whether the classifier's predictions align with human understanding of Ragas. We compare the generated explanations with human expert annotations and further analyze individual test examples to understand the role of regions highlighted by explanations in making correct or incorrect predictions made by the model. Our results demonstrate a significant alignment of the model's understanding with human understanding, and the thorough analysis validates the effectiveness of our approach.

eess.AS↗

arXiv2024

On Optimal Testing of Linearity

Linearity testing has been a focal problem in property testing of functions. We combine different known techniques and observations about linearity testing in order to resolve two recent versions of this task. First, we focus on the online manipulations model introduced by Kalemaj, Raskhodnikova and Varma (ITCS 2022 \& Theory of Computing 2023). In this model, up to $t$ data entries are adversarially manipulated after each query is answered. Ben-Eliezer, Kelman, Meir, and Raskhodnikova (ITCS 2024) showed an asymptotically optimal linearity tester that is resilient to $t$ manipulations per query, but their approach fails if $t$ is too large. We extend this result, showing an optimal tester for almost any possible value of $t$. First, we simplify their result when $t$ is small, and for larger values of $t$ we instead use sample-based testers, as defined by Goldreich and Ron (ACM Transactions on Computation Theory 2016). A key observation is that sample-based testing is resilient to online manipulations, but still achieves optimal query complexity for linearity when $t$ is large. We complement our result by showing that when $t$ is \emph{very} large, any reasonable property, and in particular linearity, cannot be tested at all. Second, we consider linearity over the reals with proximity parameter $\varepsilon$. Fleming and Yoshida (ITCS 2020) gave a tester using $O(1/\varepsilon\ \cdot log(1/\varepsilon))$ queries. We simplify their algorithms and modify the analysis accordingly, showing an optimal tester that only uses $O(1/\varepsilon)$ queries. This modification works for the low-degree testers presented in Arora, Bhattacharyya, Fleming, Kelman, and Yoshida (SODA 2023) as well, resulting in optimal testers for degree-$d$ polynomials, for any constant degree $d$.

cs.CC↗

arXiv2024

Attention-Based Audio Embeddings for Query-by-Example

An ideal audio retrieval system efficiently and robustly recognizes a short query snippet from an extensive database. However, the performance of well-known audio fingerprinting systems falls short at high signal distortion levels. This paper presents an audio retrieval system that generates noise and reverberation robust audio fingerprints using the contrastive learning framework. Using these fingerprints, the method performs a comprehensive search to identify the query audio and precisely estimate its timestamp in the reference audio. Our framework involves training a CNN to maximize the similarity between pairs of embeddings extracted from clean audio and its corresponding distorted and time-shifted version. We employ a channel-wise spectral-temporal attention mechanism to better discriminate the audio by giving more weight to the salient spectral-temporal patches in the signal. Experimental results indicate that our system is efficient in computation and memory usage while being more accurate, particularly at higher distortion levels, than competing state-of-the-art systems and scalable to a larger database.

eess.AS↗

arXiv2024

AdvNF: Reducing Mode Collapse in Conditional Normalising Flows using Adversarial Learning

Deep generative models complement Markov-chain-Monte-Carlo methods for efficiently sampling from high-dimensional distributions. Among these methods, explicit generators, such as Normalising Flows (NFs), in combination with the Metropolis Hastings algorithm have been extensively applied to get unbiased samples from target distributions. We systematically study central problems in conditional NFs, such as high variance, mode collapse and data efficiency. We propose adversarial training for NFs to ameliorate these problems. Experiments are conducted with low-dimensional synthetic datasets and XY spin models in two spatial dimensions.

cs.LG↗