arXiv subjects

Abhay Kumar

Publications and source records attributed to Abhay Kumar.

At least 37 records · Page 2Linked to original sources

arXiv2025

Physics-Informed Neural Network-Based Control for Grid-Forming Converter's Stability Under Overload Conditions

Grid-forming converters (GFCs) are crucial for frequency and voltage stability in modern power systems. However, their performance under overload conditions remains a challenge. This paper highlights the limitations of existing approaches in managing DC source saturation and AC current limits, emphasizing the need for improved control strategies to ensure system stability. This paper proposes a control strategy based on a physics-informed neural network (PINN) to improve GFC performance under overloaded conditions, effectively preventing switch failures and mitigating DC source saturation. This approach outperforms conventional methods by maintaining stable voltage and frequency, even under significant load increase where traditional droop control alone proves inadequate. The post-disturbance operating point of GFCs remains unchanged using PINN-based control with an improvement of 0.245 Hz in frequency and 0.03 p.u. in active power when compared to an already existing current limitation strategy. Additionally, it reduces peak voltage deviations during transients by 24.14\%, lowers the rate of change of frequency (ROCOF) from 0.02 Hz/s to 0.005 Hz/s, and improves the rate of change of voltage (ROCOV), keeping both within acceptable limits. These improvements significantly enhance system resilience, especially in inertia-less power networks.

eess.SY↗

arXiv2025

ZClip: Adaptive Spike Mitigation for LLM Pre-Training

Training large language models (LLMs) presents numerous challenges, including gradient instability and loss spikes. These phenomena can lead to catastrophic divergence, requiring costly checkpoint restoration and data batch skipping. Traditional gradient clipping techniques, such as constant or norm-based methods, fail to address these issues effectively due to their reliance on fixed thresholds or heuristics, leading to inefficient learning and requiring frequent manual intervention. In this work, we propose ZClip, an adaptive gradient clipping algorithm that dynamically adjusts the clipping threshold based on statistical properties of gradient norms over time. Unlike prior reactive strategies, ZClip proactively adapts to training dynamics without making any prior assumptions on the scale and the temporal evolution of gradient norms. At its core, it leverages z-score-based anomaly detection to identify and mitigate large gradient spikes, preventing malignant loss spikes while not interfering with convergence otherwise. Our code is available at: https://github.com/bluorion-com/ZClip.

cs.LG↗

arXiv2025

A Refined Analysis of Massive Activations in LLMs

Motivated in part by their relevance for low-precision training and quantization, massive activations in large language models (LLMs) have recently emerged as a topic of interest. However, existing analyses are limited in scope, and generalizability across architectures is unclear. This paper helps address some of these gaps by conducting an analysis of massive activations across a broad range of LLMs, including both GLU-based and non-GLU-based architectures. Our findings challenge several prior assumptions, most importantly: (1) not all massive activations are detrimental, i.e. suppressing them does not lead to an explosion of perplexity or a collapse in downstream task performance; (2) proposed mitigation strategies such as Attention KV bias are model-specific and ineffective in certain cases. We consequently investigate novel hybrid mitigation strategies; in particular pairing Target Variance Rescaling (TVR) with Attention KV bias or Dynamic Tanh (DyT) successfully balances the mitigation of massive activations with preserved downstream model performance in the scenarios we investigated. Our code is available at: https://github.com/bluorion-com/refine_massive_activations.

cs.CL↗

arXiv2025

Variance Control via Weight Rescaling in LLM Pre-training

The outcome of Large Language Model (LLM) pre-training strongly depends on weight initialization and variance control strategies. Although the importance of initial variance control has been well documented in neural networks in general, the literature on initialization and management of its growth during LLM pre-training, specifically, is somewhat sparse. In this paper, we introduce the Layer Index Rescaling (LIR) weight initialization scheme, and the Target Variance Rescaling (TVR) variance control strategy. Experiments on a 1B parameter LLaMA model demonstrate that better variance management using these techniques yields substantial improvements in downstream task performance (up to 4.6% on common pre-training benchmarks) and reduces extreme activation values, thus mitigating challenges associated with quantization and low-precision training. Our code is available at: https://github.com/bluorion-com/weight_rescaling.

cs.LG↗

arXiv2024

Streamlining Video Analysis for Efficient Violence Detection

This paper addresses the challenge of automated violence detection in video frames captured by surveillance cameras, specifically focusing on classifying scenes as "fight" or "non-fight." This task is critical for enhancing unmanned security systems, online content filtering, and related applications. We propose an approach using a 3D Convolutional Neural Network (3D CNN)-based model named X3D to tackle this problem. Our approach incorporates pre-processing steps such as tube extraction, volume cropping, and frame aggregation, combined with clustering techniques, to accurately localize and classify fight scenes. Extensive experimentation demonstrates the effectiveness of our method in distinguishing violent from non-violent events, providing valuable insights for advancing practical violence detection systems.

cs.CV↗

arXiv2024

ProvocationProbe: Instigating Hate Speech Dataset from Twitter

In the recent years online social media platforms has been flooded with hateful remarks such as racism, sexism, homophobia etc. As a result, there have been many measures taken by various social media platforms to mitigate the spread of hate-speech over the internet. One particular concept within the domain of hate speech is instigating hate, which involves provoking hatred against a particular community, race, colour, gender, religion or ethnicity. In this work, we introduce \textit{ProvocationProbe} - a dataset designed to explore what distinguishes instigating hate speech from general hate speech. For this study, we collected around twenty thousand tweets from Twitter, encompassing a total of nine global controversies. These controversies span various themes including racism, politics, and religion. In this paper, i) we present an annotated dataset after comprehensive examination of all the controversies, ii) we also highlight the difference between hate speech and instigating hate speech by identifying distinguishing features, such as targeted identity attacks and reasons for hate.

cs.CL↗

arXiv2024

Leveraging User-Generated Reviews for Recommender Systems with Dynamic Headers

E-commerce platforms have a vast catalog of items to cater to their customers' shopping interests. Most of these platforms assist their customers in the shopping process by offering optimized recommendation carousels, designed to help customers quickly locate their desired items. Many models have been proposed in academic literature to generate and enhance the ranking and recall set of items in these carousels. Conventionally, the accompanying carousel title text (header) of these carousels remains static. In most instances, a generic text such as "Items similar to your current viewing" is utilized. Fixed variations such as the inclusion of specific attributes "Other items from a similar seller" or "Items from a similar brand" in addition to "frequently bought together" or "considered together" are observed as well. This work proposes a novel approach to customize the header generation process of these carousels. Our work leverages user-generated reviews that lay focus on specific attributes (aspects) of an item that were favorably perceived by users during their interaction with the given item. We extract these aspects from reviews and train a graph neural network-based model under the framework of a conditional ranking task. We refer to our innovative methodology as Dynamic Text Snippets (DTS) which generates multiple header texts for an anchor item and its recall set. Our approach demonstrates the potential of utilizing user-generated reviews and presents a unique paradigm for exploring increasingly context-aware recommendation systems.

cs.IR↗

arXiv2024

Komodo: A Linguistic Expedition into Indonesia's Regional Languages

The recent breakthroughs in Large Language Models (LLMs) have mostly focused on languages with easily available and sufficient resources, such as English. However, there remains a significant gap for languages that lack sufficient linguistic resources in the public domain. Our work introduces Komodo-7B, 7-billion-parameter Large Language Models designed to address this gap by seamlessly operating across Indonesian, English, and 11 regional languages in Indonesia. Komodo-7B is a family of LLMs that consist of Komodo-7B-Base and Komodo-7B-Instruct. Komodo-7B-Instruct stands out by achieving state-of-the-art performance in various tasks and languages, outperforming the benchmarks set by OpenAI's GPT-3.5, Cohere's Aya-101, Llama-2-Chat-13B, Mixtral-8x7B-Instruct-v0.1, Gemma-7B-it , and many more. This model not only demonstrates superior performance in both language-specific and overall assessments but also highlights its capability to excel in linguistic diversity. Our commitment to advancing language models extends beyond well-resourced languages, aiming to bridge the gap for those with limited linguistic assets. Additionally, Komodo-7B-Instruct's better cross-language understanding contributes to addressing educational disparities in Indonesia, offering direct translations from English to 11 regional languages, a significant improvement compared to existing language translation services. Komodo-7B represents a crucial step towards inclusivity and effectiveness in language models, providing to the linguistic needs of diverse communities.

cs.CL↗

arXiv2024

BED: Bi-Encoder-Based Detectors for Out-of-Distribution Detection

This paper introduces a novel method leveraging bi-encoder-based detectors along with a comprehensive study comparing different out-of-distribution (OOD) detection methods in NLP using different feature extractors. The feature extraction stage employs popular methods such as Universal Sentence Encoder (USE), BERT, MPNET, and GLOVE to extract informative representations from textual data. The evaluation is conducted on several datasets, including CLINC150, ROSTD-Coarse, SNIPS, and YELLOW. Performance is assessed using metrics such as F1-Score, MCC, FPR@90, FPR@95, AUPR, an AUROC. The experimental results demonstrate that the proposed bi-encoder-based detectors outperform other methods, both those that require OOD labels in training and those that do not, across all datasets, showing great potential for OOD detection in NLP. The simplicity of the training process and the superior detection performance make them applicable to real-world scenarios. The presented methods and benchmarking metrics serve as a valuable resource for future research in OOD detection, enabling further advancements in this field. The code and implementation details can be found on our GitHub repository: https://github.com/yellowmessenger/ood-detection.

cs.CL↗

arXiv2023

High hard X-ray polarization in Cygnus X-1 confined to the intermediate hard state: evidence for a variable jet component

Cygnus X-1, the well-known accreting black hole system, exhibits several observational features hinting at an intricate interplay between the accretion disk, its atmosphere known as the corona and the putative relativistic jet. It has been extensively studied using all available observational methods, including using the newly available technique of sensitive X-ray polarimetry. X-ray polarization characteristics are distinct for coronal and jet emissions. The low X-ray polarization measured below $\sim$100 keV is understood as arising from the corona. In contrast, the high polarization measurements reported above $\sim$400 keV required a separate jet-dominated spectral component, which spectroscopy does not demonstrate conclusively. Here we report precise polarization measurements in the 100-380 keV region made during three different sub-classes of spectral states of the source using the CZTI instrument onboard {\em AstroSat}. A high polarization (23$\pm$4 \%) is found mainly in the Intermediate Hard State of the source, and the energy-resolved measurements smoothly connect the coronal and the jet regimes. When high polarization is observed, the simultaneous spectral data hints at a separate power law component above 100 keV. We examine the possible sources of this energy-dependent high polarization in Cygnus X-1.

astro-ph.HE↗

arXiv2023

[Re] Double Sampling Randomized Smoothing

This paper is a contribution to the reproducibility challenge in the field of machine learning, specifically addressing the issue of certifying the robustness of neural networks (NNs) against adversarial perturbations. The proposed Double Sampling Randomized Smoothing (DSRS) framework overcomes the limitations of existing methods by using an additional smoothing distribution to improve the robustness certification. The paper provides a clear manifestation of DSRS for a generalized family of Gaussian smoothing and a computationally efficient method for implementation. The experiments on MNIST and CIFAR-10 demonstrate the effectiveness of DSRS, consistently certifying larger robust radii compared to other methods. Also various ablations studies are conducted to further analyze the hyperparameters and effect of adversarial training methods on the certified radius by the proposed framework.

cs.LG↗

arXiv2022

Extending the energy range of AstroSat-CZTI up to 380 keV with Compton Spectroscopy

The CZTI (Cadmium Zinc Telluride Imager) onboard AstroSat is a high energy coded mask imager and spectrometer in the energy range of 20 - 100 keV. Above 100 keV, the dominance of Compton scattering cross-section in CZTI results in a significant number of 2-pixel Compton events and these have been successfully utilized for polarization analysis of Crab pulsar and nebula (and transients like Gamma-ray bursts) in 100 - 380 keV. These 2-pixel Compton events can also be used to extend the spectroscopic energy range of CZTI up to 380 keV for bright sources. However, unlike the spectroscopy in primary energy range, where simultaneous background measurement is available from masked pixels, Compton spectroscopy requires blank sky observation for background measurement. Background subtraction, in this case, is non-trivial because of the presence of both short-term and long-term temporal variations in the data, which depend on multiple factors like earth rotation and the effect of South Atlantic Anomaly (SAA) regions etc. We have developed a methodology of background selection and subtraction that takes into account for these effects. Here, we describe these background selection and subtraction techniques and validate them using spectroscopy of Crab in the extended energy range of 30 - 380 keV region, and compare the obtained spectral parameters with the INTEGRAL results. This new capability allows for the extension of the energy range of AstroSat spectroscopy and will also enable the simultaneous spectro-polarimetric study of other bright sources like Cygnus X-1.

astro-ph.IM↗

arXiv2022

Hard X-ray polarization catalog for a 5-year sample of Gamma-Ray Bursts using AstroSat CZT-Imager

Cadmium Zinc Telluride Imager (CZTI) aboard AstroSat has been regularly detecting Gamma-Ray Bursts (GRBs) since its launch in 2015. Its sensitivity to polarization measurements at energies above 100 keV allows CZTI to attempt spectro-polarimetric studies of GRBs. Here, we present the first catalog of GRB polarization measurements made by CZTI during its first five years of operation. This presents the time integrated polarization measurements of the prompt emission of 20 GRBs in the energy range 100-600 keV. The sample includes the bright GRBs which were detected within an angle range of 0-60 degree and 120-180 degree where the instrument has useful polarization sensitivity and is less prone to systematics. We implement a few new modifications in the analysis to enhance polarimetric sensitivity of the instrument. Majority of the GRBs in the sample are found to possess less / null polarization across the total bursts' duration in contrast to a small fraction of five GRBs exhibiting high polarization. The low polarization across the bursts can be speculated to be either due to the burst being intrinsically weakly polarized or due to varying polarization angle within the burst even when it is highly polarized. In comparison to POLAR measurements, CZTI has detected a larger number of cases with high polarization. This may be a consequence of the higher energy window of CZTI observations which results in the sampling of smaller duration of burst emissions in contrast to POLAR, thereby, probing emissions of less temporal variations of polarization properties.

astro-ph.HE↗

arXiv2021

Exploring Sub-MeV Sensitivity of AstroSat-CZTI for ON-axis Bright Sources

The Cadmium Zinc Telluride Imager (CZTI) onboard AstroSat is designed for hard X-ray imaging and spectroscopy in the energy range of 20 - 100 keV. The CZT detectors are of 5 mm thickness and hence have good efficiency for Compton interactions beyond 100 keV. The polarisation analysis using CZTI relies on such Compton events and have been verified experimentally. The same Compton events can also be used to extend the spectroscopy up to 380 keV. Further, it has been observed that about 20% pixels of the CZTI detector plane have low gain, and they are excluded from the primary spectroscopy. If these pixels are included, then the spectroscopic capability of CZTI can be extended up to 500 keV and further up to 700 keV with a better gain calibration in the future. Here we explore the possibility of using the Compton events as well as the low gain pixels to extend the spectroscopic energy range of CZTI for ON-axis bright X-ray sources. We demonstrate this technique using Crab observations and explore its sensitivity.

astro-ph.IM↗

arXiv2021

Sub-MeV spectroscopy with AstroSat-CZT Imager for Gamma Ray Bursts

Cadmium Zinc Telluride Imager (CZTI) onboard AstroSat has been a prolific Gamma-Ray Burst (GRB) monitor. While the 2-pixel Compton scattered events (100 - 300 keV) are used to extract sensitive spectroscopic information, the inclusion of the low-gain pixels (around 20% of the detector plane) after careful calibration extends the energy range of Compton energy spectra to 600 keV. The new feature also allows single-pixel spectroscopy of the GRBs to the sub-MeV range which is otherwise limited to 150 keV. We also introduced a new noise rejection algorithm in the analysis ('Compton noise'). These new additions not only enhances the spectroscopic sensitivity of CZTI, but the sub-MeV spectroscopy will also allow proper characterization of the GRBs not detected by Fermi. This article describes the methodology of single, Compton event and veto spectroscopy in 100 - 600 keV for the GRBs detected in the first year of operation. CZTI in last five years has detected around 20 bright GRBs. The new methodologies, when applied on the spectral analysis for this large sample of GRBs, has the potential to improve the results significantly and help in better understanding the prompt emission mechanism.

astro-ph.HE↗

arXiv2019

Learning Discriminative features using Center Loss and Reconstruction as Regularizer for Speech Emotion Recognition

This paper proposes a Convolutional Neural Network (CNN) inspired by Multitask Learning (MTL) and based on speech features trained under the joint supervision of softmax loss and center loss, a powerful metric learning strategy, for the recognition of emotion in speech. Speech features such as Spectrograms and Mel-frequency Cepstral Coefficient s (MFCCs) help retain emotion-related low-level characteristics in speech. We experimented with several Deep Neural Network (DNN) architectures that take in speech features as input and trained them under both softmax and center loss, which resulted in highly discriminative features ideal for Speech Emotion Recognition (SER). Our networks also employ a regularizing effect by simultaneously performing the auxiliary task of reconstructing the input speech features. This sharing of representations among related tasks enables our network to better generalize the original task of SER. Some of our proposed networks contain far fewer parameters when compared to state-of-the-art architectures.

cs.SD↗

arXiv2019

MTCNET: Multi-task Learning Paradigm for Crowd Count Estimation

We propose a Multi-Task Learning (MTL) paradigm based deep neural network architecture, called MTCNet (Multi-Task Crowd Network) for crowd density and count estimation. Crowd count estimation is challenging due to the non-uniform scale variations and the arbitrary perspective of an individual image. The proposed model has two related tasks, with Crowd Density Estimation as the main task and Crowd-Count Group Classification as the auxiliary task. The auxiliary task helps in capturing the relevant scale-related information to improve the performance of the main task. The main task model comprises two blocks: VGG-16 front-end for feature extraction and a dilated Convolutional Neural Network for density map generation. The auxiliary task model shares the same front-end as the main task, followed by a CNN classifier. Our proposed network achieves 5.8% and 14.9% lower Mean Absolute Error (MAE) than the state-of-the-art methods on ShanghaiTech dataset without using any data augmentation. Our model also outperforms with 10.5% lower MAE on UCF_CC_50 dataset.

cs.LG↗

arXiv2019

Visual Context-aware Convolution Filters for Transformation-invariant Neural Network

We propose a novel visual context-aware filter generation module which incorporates contextual information present in images into Convolutional Neural Networks (CNNs). In contrast to traditional CNNs, we do not employ the same set of learned convolution filters for all input image instances. Our proposed input-conditioned convolution filters when combined with techniques inspired by Multi-instance learning and max-pooling, results in a transformation-invariant neural network. We investigated the performance of our proposed framework on three MNIST variations, which covers both rotation and scaling variance, and achieved 1.13% error on MNIST-rot-12k, 1.12% error on Half-rotated MNIST and 0.68% error on Scaling MNIST, which is significantly better than the state-of-the-art results. We make use of visualization to further prove the effectiveness of our visual context-aware convolution filters. Our proposed visual context-aware convolution filter generation framework can also serve as a plugin for any CNN based architecture and enhance its modeling capacity.

cs.CV↗