arXiv subjects

Kartheek Kumar Reddy Nareddy

Publications and source records attributed to Kartheek Kumar Reddy Nareddy.

3 recordsLinked to original sources

arXiv2025

Analyzing and Fine-Tuning Whisper Models for Multilingual Pilot Speech Transcription in the Cockpit

The developments in transformer encoder-decoder architectures have led to significant breakthroughs in machine translation, Automatic Speech Recognition (ASR), and instruction-based chat machines, among other applications. The pre-trained models were trained on vast amounts of generic data over a few epochs (fewer than five in most cases), resulting in their strong generalization capabilities. Nevertheless, the performance of these models does suffer when applied to niche domains like transcribing pilot speech in the cockpit, which involves a lot of specific vocabulary and multilingual conversations. This paper investigates and improves the transcription accuracy of cockpit conversations with Whisper models. We have collected around 85 minutes of cockpit simulator recordings and 130 minutes of interview recordings with pilots and manually labeled them. The speakers are middle aged men speaking both German and English. To improve the accuracy of transcriptions, we propose multiple normalization schemes to refine the transcripts and improve Word Error Rate (WER). We then employ fine-tuning to enhance ASR performance, utilizing performance-efficient fine-tuning with Low-Rank Adaptation (LoRA). Hereby, WER decreased from 68.49 \% (pretrained whisper Large model without normalization baseline) to 26.26\% (finetuned whisper Large model with the proposed normalization scheme).

cs.CL↗

arXiv2023

Tight-frame-like Analysis-Sparse Recovery Using Non-tight Sensing Matrices

The choice of the sensing matrix is crucial in compressed sensing. Random Gaussian sensing matrices satisfy the restricted isometry property, which is crucial for solving the sparse recovery problem using convex optimization techniques. However, tight-frame sensing matrices result in minimum mean-squared-error recovery given oracle knowledge of the support of the sparse vector. If the sensing matrix is not tight, could one achieve the recovery performance assured by a tight frame by suitably designing the recovery strategy? -- This is the key question addressed in this paper. We consider the analysis-sparse l1-minimization problem with a generalized l2-norm-based data-fidelity and show that it effectively corresponds to using a tight-frame sensing matrix. The new formulation offers improved performance bounds when the number of non-zeros is large. One could develop a tight-frame variant of a known sparse recovery algorithm using the proposed formalism. We solve the analysis-sparse recovery problem in an unconstrained setting using proximal methods. Within the tight-frame sensing framework, we rescale the gradients of the data-fidelity loss in the iterative updates to further improve the accuracy of analysis-sparse recovery. Experimental results show that the proposed algorithms offer superior analysis-sparse recovery performance. Proceeding further, we also develop deep-unfolded variants, with a convolutional neural network as the sparsifying operator. On the application front, we consider compressed sensing image recovery. Experimental results on Set11, BSD68, Urban100, and DIV2K datasets show that the proposed techniques outperform the state-of-the-art techniques, with performance measured in terms of peak signal-to-noise ratio and structural similarity index metric.

eess.SP↗

arXiv2021

Quantized Proximal Averaging Network for Analysis Sparse Coding

We solve the analysis sparse coding problem considering a combination of convex and non-convex sparsity promoting penalties. The multi-penalty formulation results in an iterative algorithm involving proximal-averaging. We then unfold the iterative algorithm into a trainable network that facilitates learning the sparsity prior. We also consider quantization of the network weights. Quantization makes neural networks efficient both in terms of memory and computation during inference, and also renders them compatible for low-precision hardware deployment. Our learning algorithm is based on a variant of the ADAM optimizer in which the quantizer is part of the forward pass and the gradients of the loss function are evaluated corresponding to the quantized weights while doing a book-keeping of the high-precision weights. We demonstrate applications to compressed image recovery and magnetic resonance image reconstruction. The proposed approach offers superior reconstruction accuracy and quality than state-of-the-art unfolding techniques and the performance degradation is minimal even when the weights are subjected to extreme quantization.

cs.LG↗