Searcharxiv⌕ Search

arXiv subjects

Sourav Bhattacharya

Publications and source records attributed to Sourav Bhattacharya.

At least 37 records · Page 2Linked to original sources
arXiv2025

Benchmarking Rotary Position Embeddings for Automatic Speech Recognition

Self-attention relies on positional embeddings to encode input order. Relative Position (RelPos) embeddings are widely used in Automatic Speech Recognition (ASR). However, RelPos has quadratic time complexity to input length and is often incompatible with fast GPU implementations of attention. In contrast, Rotary Positional Embedding (RoPE) rotates each input vector based on its absolute position, taking linear time to sequence length, implicitly encoding relative distances through self-attention dot products. Thus, it is usually compatible with efficient attention. However, its use in ASR remains underexplored. This work evaluates RoPE across diverse ASR tasks with training data ranging from 100 to 50,000 hours, covering various speech types (read, spontaneous, clean, noisy) and different accents in both streaming and non-streaming settings. ASR error rates are similar or better than RelPos, while training time is reduced by up to 21%. Code is available via the SpeechBrain toolkit.

cs.CL↗
arXiv2025

Robust Unsupervised Adaptation of a Speech Recogniser Using Entropy Minimisation and Speaker Codes

Speech recognisers usually perform optimally only in a specific environment and need to be adapted to work well in another. For adaptation to a new speaker, there is often too little data for fine-tuning to be robust, and that data is usually unlabelled. This paper proposes a combination of approaches to make adaptation to a single minute of data robust. First, instead of estimating the adaptation parameters with cross-entropy on a single error-prone hypothesis or "pseudo-label", this paper proposes a novel loss function, the conditional entropy over complete hypotheses. Using multiple hypotheses makes adaptation more robust to errors in the initial recognition. Second, a "speaker code" characterises a speaker in a vector short enough that it requires little data to estimate. On a far-field noise-augmented version of Common Voice, the proposed scheme yields a 20% relative improvement in word error rate on one minute of adaptation data, increasing on 10 minutes to 29%.

eess.AS↗
arXiv2025

Evaluation of LLMs in Speech is Often Flawed: Test Set Contamination in Large Language Models for Speech Recognition

Recent work suggests that large language models (LLMs) can improve performance of speech tasks compared to existing systems. To support their claims, results on LibriSpeech and Common Voice are often quoted. However, this work finds that a substantial amount of the LibriSpeech and Common Voice evaluation sets appear in public LLM pretraining corpora. This calls into question the reliability of findings drawn from these two datasets. To measure contamination impact, LLMs trained with/without contamination are compared. A contaminated LLM is more likely to generate test sentences it has seen during training. Then, speech recognisers based on LLMs are compared. They show only subtle error rate differences if the LLM is contaminated, but assign significantly higher probabilities to transcriptions seen during LLM training. Results show that LLM outputs can be biased by tiny amounts of data contamination, highlighting the importance of evaluating LLM-based speech systems with held-out data.

eess.AS↗
arXiv2025

Fast Sampling Through The Reuse Of Attention Maps In Diffusion Models

Text-to-image diffusion models have demonstrated unprecedented capabilities for flexible and realistic image synthesis. Nevertheless, these models rely on a time-consuming sampling procedure, which has motivated attempts to reduce their latency. When improving efficiency, researchers often use the original diffusion model to train an additional network designed specifically for fast image generation. In contrast, our approach seeks to reduce latency directly, without any retraining, fine-tuning, or knowledge distillation. In particular, we find the repeated calculation of attention maps to be costly yet redundant, and instead suggest reusing them during sampling. Our specific reuse strategies are based on ODE theory, which implies that the later a map is reused, the smaller the distortion in the final image. We empirically compare our reuse strategies with few-step sampling procedures of comparable latency, finding that reuse generates images that are closer to those produced by the original high-latency diffusion model.

cs.CV↗
arXiv2025

Guidance Free Image Editing via Explicit Conditioning

Current sampling mechanisms for conditional diffusion models rely mainly on Classifier Free Guidance (CFG) to generate high-quality images. However, CFG requires several denoising passes in each time step, e.g., up to three passes in image editing tasks, resulting in excessive computational costs. This paper introduces a novel conditioning technique to ease the computational burden of the well-established guidance techniques, thereby significantly improving the inference time of diffusion models. We present Explicit Conditioning (EC) of the noise distribution on the input modalities to achieve this. Intuitively, we model the noise to guide the conditional diffusion model during the diffusion process. We present evaluations on image editing tasks and demonstrate that EC outperforms CFG in generating diverse high-quality images with significantly reduced computations.

cs.CV↗
arXiv2025

Upcycling Text-to-Image Diffusion Models for Multi-Task Capabilities

Text-to-image synthesis has witnessed remarkable advancements in recent years. Many attempts have been made to adopt text-to-image models to support multiple tasks. However, existing approaches typically require resource-intensive re-training or additional parameters to accommodate for the new tasks, which makes the model inefficient for on-device deployment. We propose Multi-Task Upcycling (MTU), a simple yet effective recipe that extends the capabilities of a pre-trained text-to-image diffusion model to support a variety of image-to-image generation tasks. MTU replaces Feed-Forward Network (FFN) layers in the diffusion model with smaller FFNs, referred to as experts, and combines them with a dynamic routing mechanism. To the best of our knowledge, MTU is the first multi-task diffusion modeling approach that seamlessly blends multi-tasking with on-device compatibility, by mitigating the issue of parameter inflation. We show that the performance of MTU is on par with the single-task fine-tuned diffusion models across several tasks including image editing, super-resolution, and inpainting, while maintaining similar latency and computational load (GFLOPs) as the single-task fine-tuned models.

cs.CV↗
arXiv2024

Twist like behavior in non-twist patterns of triods

We prove a sufficient condition for a \emph{pattern} $π$ on a \emph{triod} $T$ to have \emph{rotation number} $ρ_π$ coincide with an end-point of its \emph{forced rotation interval} $I_π$. Then, we demonstrate the existence of peculiar \emph{patterns} on \emph{triods} that are neither \emph{triod twists} nor possess a \emph{block structure} over a \emph{triod twist pattern}, but their \emph{rotation numbers} are an end point of their respective \emph{forced rotation intervals}, mimicking the behavior of \emph{triod twist patterns}. These \emph{patterns}, absent in circle maps (see \cite{almBB}), highlight a key difference between the rotation theories for \emph{triods} (introduced in \cite{BMR}) and that of circle maps. We name these \emph{patterns}: ``\emph{strangely ordered}" and show that they are semi-conjugate to circle rotations via a piece-wise monotone map. We conclude by providing an algorithm to construct unimodal \emph{strangely ordered patterns} with arbitrary \emph{rotation pairs}.

math.DS↗
arXiv2024

MobileQuant: Mobile-friendly Quantization for On-device Language Models

Large language models (LLMs) have revolutionized language processing, delivering outstanding results across multiple applications. However, deploying LLMs on edge devices poses several challenges with respect to memory, energy, and compute costs, limiting their widespread use in devices such as mobile phones. A promising solution is to reduce the number of bits used to represent weights and activations. While existing works have found partial success at quantizing LLMs to lower bitwidths, e.g. 4-bit weights, quantizing activations beyond 16 bits often leads to large computational overheads due to poor on-device quantization support, or a considerable accuracy drop. Yet, 8-bit activations are very attractive for on-device deployment as they would enable LLMs to fully exploit mobile-friendly hardware, e.g. Neural Processing Units (NPUs). In this work, we make a first attempt to facilitate the on-device deployment of LLMs using integer-only quantization. We first investigate the limitations of existing quantization methods for on-device deployment, with a special focus on activation quantization. We then address these limitations by introducing a simple post-training quantization method, named MobileQuant, that extends previous weight equivalent transformation works by jointly optimizing the weight transformation and activation range parameters in an end-to-end manner. MobileQuant demonstrates superior capabilities over existing methods by 1) achieving near-lossless quantization on a wide range of LLM benchmarks, 2) reducing latency and energy consumption by 20\%-50\% compared to current on-device quantization strategies, 3) requiring limited compute budget, 4) being compatible with mobile-friendly compute units, e.g. NPU.

cs.CL↗
arXiv2024

Forcing Minimal Interval Patterns as Interval Exchange Transformations

We prove that any over-twist pattern is conjugate to an interval exchange transformation with bounded number of segments of isometry, restricted on one of its cycles. The bound is independent of the period and over-rotation number of the over-twist pattern and depends only on its modality.

math.DS↗
arXiv2024

Resummation of local and non-local scalar self energies via the Schwinger-Dyson equation in de Sitter spacetime

We consider a massless and minimally coupled self interacting quantum scalar field in the inflationary de Sitter spacetime. The scalar potential is taken to be a hybrid, $V(ϕ)= λϕ^4/4!+βϕ^3/3!$ ($λ>0$). Compared to the earlier well studied $β=0$ case, the present potential has a rolling down effect due to the $ϕ^3$ term, along with the usual bounding effect due to the $ϕ^4$ term. We begin by constructing the Schwinger-Dyson equation for the scalar Feynman propagator up to two loop, at ${\cal O}(λ)$, ${\cal O}(β^2)$, ${\cal O}(λ^2)$ and ${\cal O}(λβ^2)$. We consider first the local part of the scalar self energy and compute the rest mass squared of the scalar field, dynamically generated via the late time non-perturbative secular logarithms, by resumming the daisy-like graphs. The logarithms associated here are sub-leading, compared to those associated with the non-local, leading terms. We also argue that unlike the quartic case, considering merely the one loop results for the purpose of resummation does not give us any sensible result here. We next construct the non-perturbative two particle irreducible effective action up to three loop and derive from it the Schwinger-Dyson equation once again. This equation is satisfied by the non-perturbative Feynman propagator. By series expanding this propagator, the resummed local part of the self energy is shown to yield the same dynamical mass as that of the above. We next use this equation to resum the effect of the non-local part of the scalar self energy, and show that even though the perturbatively corrected propagator shows secular growth at late times, there exists one resummed solution which is vanishing for large spatial separations, in qualitative agreement with that of the stochastic formalism.

hep-th↗
arXiv2024

Loop correction and resummation of vertex functions for a self interacting scalar field in the de Sitter spacetime

We consider a massless and minimally coupled self interacting quantum scalar field theory in the inflationary de Sitter background of dimension four. The self interaction potential is taken to be either quartic, $λϕ^4/4!$, or quartic plus cubic, $λϕ^4/4!+βϕ^3/3!$ ($λ\,{\ensuremath >}\,0$). We compute the four and three point vertex functions up to two loop. The purely local or partly local part of these renormalised loop corrected vertex functions grow unboundedly after sufficient number of de Sitter $e$-foldings, due to the appearances of secular logarithms. We focus on the purely local part of the vertex functions and attempt a resummation of them in terms of the dynamically generated mass of the scalar field at late times. Such local logarithms have sub-leading powers compared to the non-local leading ones which can be resummed via the stochastic formalism. The variation of these vertex functions are investigated with respect to the tree level couplings numerically. Since neither the secular effect, nor the dynamical generation of field mass is possible in the Minkowski spacetime, the above phenomenon has no flat spacetime analogue. We have also compared our result with the ones that could be found via the recently proposed renormalisation group techniques. All these results suggest that at late times the value of the non-perturbative vertex function should be less than the tree level coupling.

hep-th↗
arXiv2024

Linear-Complexity Self-Supervised Learning for Speech Processing

Self-supervised learning (SSL) models usually require weeks of pre-training with dozens of high-end GPUs. These models typically have a multi-headed self-attention (MHSA) context encoder. However, MHSA takes quadratic time and space in the input length, contributing to the high pre-training cost. Linear-complexity alternatives to MHSA have been proposed. For instance, in supervised training, the SummaryMixing model is the first to outperform MHSA across multiple speech processing tasks. However, these cheaper alternatives have not been explored for SSL yet. This paper studies a linear-complexity context encoder for SSL for the first time. With better or equivalent performance for the downstream tasks of the MP3S benchmark, SummaryMixing reduces the pre-training time and peak VRAM of wav2vec 2.0 model by 18% and by 23%, respectively, leading to the pre-training of a 155M wav2vec 2.0 model finished within one week with 4 Tesla A100 GPUs. Code is available at https://github.com/SamsungLabs/SummaryMixing.

cs.CL↗
arXiv2024

SummaryMixing: A Linear-Complexity Alternative to Self-Attention for Speech Recognition and Understanding

Modern speech processing systems rely on self-attention. Unfortunately, token mixing with self-attention takes quadratic time in the length of the speech utterance, slowing down inference and training and increasing memory consumption. Cheaper alternatives to self-attention for ASR have been developed, but they fail to consistently reach the same level of accuracy. This paper, therefore, proposes a novel linear-time alternative to self-attention. It summarises an utterance with the mean over vectors for all time steps. This single summary is then combined with time-specific information. We call this method "SummaryMixing". Introducing SummaryMixing in state-of-the-art ASR models makes it feasible to preserve or exceed previous speech recognition performance while making training and inference up to 28% faster and reducing memory use by half.

cs.CL↗
arXiv2024

Decoherence and entropy generation at one loop in the inflationary de Sitter spacetime for Yukawa interaction

The decoherence mechanism is believed to be possibly connected to the quantum to classical transition of the primordial cosmological perturbations in the early universe. In this paper, we extend our previous analysis on decoherence in a fermion and scalar quantum field theory coupled via the Yukawa interaction in the Minkowski spacetime, to the inflationary de Sitter background. We treat the scalar field as the system and the fermions as the environment, and both the fields are taken to be massless. We utilise a non-equilibrium effective field theory formalism, suitable for open quantum systems such as this. We assume that an observer measures only the Gaussian 2-point correlator for the scalar field, as the simplest realistic scenario. In order to compute the von Neumann entropy generated at late times as a measure of the decoherence, we construct the one loop renormalised Kadanoff-Baym equation, which is the equation of motion satisfied by the 2-point correlators in the closed time path Schwinger-Keldysh formalism. These equations account to the self energy corrections. Using this, we next construct the one loop corrected statistical propagator for the scalar, which is related to its phase space area, to compute the von Neumann entropy. We also compute the variation of the von Neumann entropy with respect to relevant parameters. We note the qualitative similarity between our findings and the scenario where both the system and the environment are scalars. Our result is also qualitatively similar to an earlier one found by using the influence functional technique for a massive Yukawa theory.

hep-th↗
arXiv2024

Non-perturbative $\langle ϕ\rangle$, $\langle ϕ^2 \rangle$ and the dynamically generated scalar mass with Yukawa interaction in the inflationary de Sitter spacetime

We consider a massless minimally coupled self interacting quantum scalar field coupled to fermion via the Yukawa interaction, in the inflationary de Sitter background. The fermion is also taken to be massless and the scalar potential is taken to be a hybrid, $V(ϕ)= λϕ^4/4!+ βϕ^3/3!$ ($λ>0$). The chief physical motivation behind this choice of $V(ϕ)$ corresponds to, apart from its boundedness from below property, the fact that shape wise $V(ϕ)$ has qualitative similarity with standard inflationary classical slow roll potentials. Also, its vacuum expectation value can be negative, suggesting some screening of the inflationary cosmological constant. We choose that $\langle ϕ\rangle\sim 0$ at early times with respect to the Bunch-Davies vacuum, so that perturbation theory is valid initially. We consider the equations satisfied by $\langle ϕ(t) \rangle$ and $\langle ϕ^2(t) \rangle$, constructed from the coarse grained equation of motion for the slowly rolling $ϕ$. We then compute the vacuum diagrammes of various relevant operators using the in-in formalism up to three loop, in terms of the leading powers of the secular logarithms. For a closed fermion loop, we have restricted ourselves here to only the local contribution. These large temporal logarithms are then resummed by constructing suitable non-perturbative equations to compute $\langle ϕ\rangle$ and $\langle ϕ^2 \rangle$. $\langle ϕ\rangle$ turns out to be at least approximately an order of magnitude less compared to the minimum of the classical potential, $-3β/λ$, owing to the strong quantum fluctuations. For $\langle ϕ^2 \rangle$, we have computed the dynamically generated scalar mass at late times, by taking the appropriate purely local contributions. Variations of these quantities with respect to different couplings have also been presented.

hep-th↗
arXiv2023

Forcing minimal patterns of triods

\emph{Rotation numbers} for some maps of \emph{triods} was introduced in \cite{BMR}. The goal of this paper is to study \emph{patterns} of \emph{triods} which don't force other \emph{patterns} with the same \emph{rotation number} which we name \emph{triod twists}. We obtain their complete characterization and show that these \emph{patterns} can be conjugated to \emph{circle rotation} by a \emph{piecewise monotone} map. We also describe the dynamics of \emph{unimodal triod twist patterns} with a given rational \emph{rotation number}.

math.DS↗
arXiv2023

Non-perturbative analysis for a massless minimal quantum scalar with $V(ϕ)=λϕ^4/4!+βϕ^3/3!$ in the inflationary de Sitter spacetime

We consider a massless, minimally coupled quantum scalar field theory with an asymmetric self interaction, $V (ϕ) = λϕ^4/4!+βϕ^3/3!$ ($λ>0$) in the inflationary de Sitter spacetime. The potential is bounded from below. While the $β=0$ case has been much well studied, the motivation behind taking such a hybrid potential corresponds to the fact that it might generate finite negative vacuum expectation values of $V(ϕ)$ as well of $ϕ$, leading to some dynamical screening of the inflationary cosmological constant $Λ$, at late times, with the initial conditions, $\langle ϕ\rangle=0=\langle V(ϕ) \rangle $. In this work we first compute the vacuum expectation values of $ϕ,\, ϕ^2$ and $V(ϕ)$, using the late time, non-perturbative and infrared effective stochastic formalism. The backreactions to the inflationary $Λ$ are estimated. We also compute the dynamically generated mass of the scalar field using $\langle ϕ^2 \rangle$. We next compute $\langleϕ^2\rangle$ using quantum field theory with respect to the initial Bunch-Davies vacuum at one and two loops, using the Schwinger-Keldysh formalism. These results show non-perturbative secular logarithms, growing with the cosmological time. Using next a recently proposed renormalisation group inspired formalism, we attempt to find out a resummed $\langleϕ^2\rangle$. We have been able to resum some part of the same which contains contributions only from the local self energy. The corresponding dynamically generated mass is computed. Comparison of the stochastic and the quantum field theory results shows that they differ numerically, although they have similar qualitative behaviour. Possible reasons for such quantitative mismatch is discussed. The manifestation of strong non-classical effects in the results found via both the formalisms has been emphasised.

hep-th↗