arXiv subjects

Subhabrata Majumdar

Publications and source records attributed to Subhabrata Majumdar.

At least 37 records · Page 2Linked to original sources

arXiv2025

Consistency in Language Models: Current Landscape, Challenges, and Future Directions

The hallmark of effective language use lies in consistency: expressing similar meanings in similar contexts and avoiding contradictions. While human communication naturally demonstrates this principle, state-of-the-art language models (LMs) struggle to maintain reliable consistency across task- and domain-specific applications. Here we examine the landscape of consistency research in LMs, analyze current approaches to measure aspects of consistency, and identify critical research gaps. Our findings point to an urgent need for quality benchmarks to measure and interdisciplinary approaches to ensure consistency while preserving utility.

cs.CL↗

arXiv2025

Semantic Consistency for Assuring Reliability of Large Language Models

Large Language Models (LLMs) exhibit remarkable fluency and competence across various natural language tasks. However, recent research has highlighted their sensitivity to variations in input prompts. To deploy LLMs in a safe and reliable manner, it is crucial for their outputs to be consistent when prompted with expressions that carry the same meaning or intent. While some existing work has explored how state-of-the-art LLMs address this issue, their evaluations have been confined to assessing lexical equality of single- or multi-word answers, overlooking the consistency of generative text sequences. For a more comprehensive understanding of the consistency of LLMs in open-ended text generation scenarios, we introduce a general measure of semantic consistency, and formulate multiple versions of this metric to evaluate the performance of various LLMs. Our proposal demonstrates significantly higher consistency and stronger correlation with human evaluations of output consistency than traditional metrics based on lexical consistency. Finally, we propose a novel prompting strategy, called Ask-to-Choose (A2C), to enhance semantic consistency. When evaluated for closed-book question answering based on answer variations from the TruthfulQA benchmark, A2C increases accuracy metrics for pretrained and finetuned LLMs by up to 47%, and semantic consistency metrics for instruction-tuned models by up to 7-fold.

cs.CL↗

arXiv2025

Evaluating Defences against Unsafe Feedback in RLHF

While there has been progress towards aligning Large Language Models (LLMs) with human values and ensuring safe behaviour at inference time, safety guards can easily be removed when fine tuned on unsafe and harmful datasets. While this setting has been treated extensively, another popular training paradigm, learning from unsafe feedback with reinforcement learning, has previously been unexplored. This is concerning due to the widespread deployment of feedback collection systems. We address this gap by providing an analysis of learning settings where feedback is harmful, i.e. that unsafe samples are preferred over safe ones despite model developers goal to maintain safety. We find that safety-aligned LLMs easily explore unsafe action spaces via generating harmful text and optimize for reward that violates safety constraints indicating that current safety guards are not enough to prevent learning from unsafe feedback. In order to protect against this vulnerability, we adapt a number of both "implict" and "explicit" harmful fine-tuning defences to evaluate whether they are effective as learning constraints in an RLHF setting finding that no method is generally effective pointing to the need for more defence research. We end the paper with the observation that some defences work by performing "harmless reward hacking" for which we provide a theoretical explanation drawn from the theory of Constrained Markov Decision Processes and provide some direction for future defence development.

cs.LG↗

arXiv2025

Improving Consistency in Large Language Models through Chain of Guidance

Consistency is a fundamental dimension of trustworthiness in Large Language Models (LLMs). For humans to be able to trust LLM-based applications, their outputs should be consistent when prompted with inputs that carry the same meaning or intent. Despite this need, there is no known mechanism to control and guide LLMs to be more consistent at inference time. In this paper, we introduce a novel alignment strategy to maximize semantic consistency in LLM outputs. Our proposal is based on Chain of Guidance (CoG), a multistep prompting technique that generates highly consistent outputs from LLMs. For closed-book question-answering (Q&A) tasks, when compared to direct prompting, the outputs generated using CoG show improved consistency. While other approaches like template-based responses and majority voting may offer alternative paths to consistency, our work focuses on exploring the potential of guided prompting. We use synthetic data sets comprised of consistent input-output pairs to fine-tune LLMs to produce consistent and correct outputs. Our fine-tuned models are more than twice as consistent compared to base models and show strong generalization capabilities by producing consistent outputs over datasets not used in the fine-tuning process.

cs.CL↗

arXiv2024

Representation Noising: A Defence Mechanism Against Harmful Finetuning

Releasing open-source large language models (LLMs) presents a dual-use risk since bad actors can easily fine-tune these models for harmful purposes. Even without the open release of weights, weight stealing and fine-tuning APIs make closed models vulnerable to harmful fine-tuning attacks (HFAs). While safety measures like preventing jailbreaks and improving safety guardrails are important, such measures can easily be reversed through fine-tuning. In this work, we propose Representation Noising (RepNoise), a defence mechanism that operates even when attackers have access to the weights. RepNoise works by removing information about harmful representations such that it is difficult to recover them during fine-tuning. Importantly, our defence is also able to generalize across different subsets of harm that have not been seen during the defence process as long as they are drawn from the same distribution of the attack set. Our method does not degrade the general capability of LLMs and retains the ability to train the model on harmless tasks. We provide empirical evidence that the efficacy of our defence lies in its ``depth'': the degree to which information about harmful representations is removed across all layers of the LLM. We also find areas where RepNoise still remains ineffective and highlight how those limitations can inform future research.

cs.CL↗

arXiv2024

Embedding-based classifiers can detect prompt injection attacks

Large Language Models (LLMs) are seeing significant adoption in every type of organization due to their exceptional generative capabilities. However, LLMs are found to be vulnerable to various adversarial attacks, particularly prompt injection attacks, which trick them into producing harmful or inappropriate content. Adversaries execute such attacks by crafting malicious prompts to deceive the LLMs. In this paper, we propose a novel approach based on embedding-based Machine Learning (ML) classifiers to protect LLM-based applications against this severe threat. We leverage three commonly used embedding models to generate embeddings of malicious and benign prompts and utilize ML classifiers to predict whether an input prompt is malicious. Out of several traditional ML methods, we achieve the best performance with classifiers built using Random Forest and XGBoost. Our classifiers outperform state-of-the-art prompt injection classifiers available in open-source implementations, which use encoder-only neural networks.

cs.CR↗

arXiv2023

Beyond Best-Fits and Model Selection -- Introducing "Reliability" of cusp-core inference of dark matter halos

We introduce the notion of a Bayesian analysis motivated `reliability' that gives a truer distinction of cusp-core and other halo-parameters (like mass-concentration) in an ensemble of observed galaxies. Our approach goes beyond the standard statistical techniques of parameter estimation and model fitting. We create hundreds of thousands of realistic mock SPARC RCs, with both cuspy and cored DM density profiles as model inputs. These RCs carefully incorporate the details of SPARC data such as the nature of observed uncertainties and different sources of scatters arising from observation, presence of baryons, DM mass-concentration, etc. Bayesian analysis of these mock RCs enables us to reconstruct and identify the parameter space in galaxy observable and theory where one can venture beyond best-fits to a preferred DM halo model or model selections between different density models. We find that it is imperative to choose low stellar surface density ($Σ_{\star}$) galaxies for reliable cusp-vs-core distinction; for example, RC data for galaxies with $Σ_{\star} \leq 2.5$ is needed for a 75\% confidence in distinguishing cusps from cores. Similarly, we also find that for correct estimations of the halo masses and concentrations, the RCs need to be measured to at least a radial distance $\geq 0.8r_s$ where $r_s$ is the scale radii of the corresponding DM halo density profiles. Out of the total $\sim$ 135 SPARC galaxies, using our reliability criteria, we find that only 21 RCs clear the bar to be used for any unbiased cusp-core distinction as well as DM halo mass-concentration estimates at $\geq$75\% reliability confidence level. With $\geq$66\% ( $\geq$50\%) reliability settings, the sample size increases to 44 (59). Interestingly, in the $\geq 75$\% reliable subsample, there are 5 times more galaxies that are reliably cored than cuspy. [Abridged]

astro-ph.CO↗

arXiv2023

Intrinsic Sliced Wasserstein Distances for Comparing Collections of Probability Distributions on Manifolds and Graphs

Collections of probability distributions arise in a variety of applications ranging from user activity pattern analysis to brain connectomics. In practice these distributions can be defined over diverse domain types including finite intervals, circles, cylinders, spheres, other manifolds, and graphs. This paper introduces an approach for detecting differences between two collections of distributions over such general domains. To this end, we propose the intrinsic slicing construction that yields a novel class of Wasserstein distances on manifolds and graphs. These distances are Hilbert embeddable, allowing us to reduce the distribution collection comparison problem to a more familiar mean testing problem in a Hilbert space. We provide two testing procedures one based on resampling and another on combining p-values from coordinate-wise tests. Our experiments in various synthetic and real data settings show that the resulting tests are powerful and the p-values are well-calibrated.

stat.ME↗

arXiv2023

Simultaneous Selection of Multiple Important Single Nucleotide Polymorphisms in Familial Genome Wide Association Studies Data

We propose a resampling-based fast variable selection technique for detecting relevant single nucleotide polymorphisms (SNP) in a multi-marker mixed effect model. Due to computational complexity, current practice primarily involves testing the effect of one SNP at a time, commonly termed as `single SNP association analysis'. Joint modeling of genetic variants within a gene or pathway may have better power to detect associated genetic variants, especially the ones with weak effects. In this paper, we propose a computationally efficient model selection approach -- based on the e-values framework -- for single SNP detection in families while utilizing information on multiple SNPs simultaneously. To overcome computational bottleneck of traditional model selection methods, our method trains one single model, and utilizes a fast and scalable bootstrap procedure. We illustrate through numerical studies that our proposed method is more effective in detecting SNPs associated with a trait than either single-marker analysis using family data or model selection methods that ignore the familial dependency structure. Further, we perform gene-level analysis in Minnesota Center for Twin and Family Research (MCTFR) dataset using our method to detect several SNPs using this that have been implicated to be associated with alcohol consumption.

stat.AP↗

arXiv2023

Measuring Reliability of Large Language Models through Semantic Consistency

While large pretrained language models (PLMs) demonstrate incredible fluency and performance on many natural language tasks, recent work has shown that well-performing PLMs are very sensitive to what prompts are feed into them. Even when prompts are semantically identical, language models may give very different answers. When considering safe and trustworthy deployments of PLMs we would like their outputs to be consistent under prompts that mean the same thing or convey the same intent. While some work has looked into how state-of-the-art PLMs address this need, they have been limited to only evaluating lexical equality of single- or multi-word answers and do not address consistency of generative text sequences. In order to understand consistency of PLMs under text generation settings, we develop a measure of semantic consistency that allows the comparison of open-ended text outputs. We implement several versions of this consistency metric to evaluate the performance of a number of PLMs on paraphrased versions of questions in the TruthfulQA dataset, we find that our proposed metrics are considerably more consistent than traditional metrics embodying lexical consistency, and also correlate with human evaluation of output consistency to a higher degree.

cs.CL↗

arXiv2023

Revisiting secondary CMB distortions due to kinetic Sunyaev-Zel'dovich effect from quasar bubbles before reionization

We discuss the secondary cosmic microwave background (CMB) anisotropy due to kinetic Sunyaev-Zel'dovich (kSZ) effect from ionized bubbles around individual quasars prior to the reionization of the Universe. The bubbles create local ionization modulations which move with the large scale structure linear bulk flow and act as sources for kSZ. We improve upon previous calculations of this effect, using a halo model based description of quasar abundance, and find that the kSZ distortion power spectrum, $C_\ell$, from the bubbles to be sub-dominant to kSZ from patchy reionization driven by galaxies. However, the shape of the two $C_\ell$'s are very different with the quasar bubble $C_\ell$ having a peak at $\ell \approx 500-700$ whereas the $C_\ell$ due to patchy reionization flattening out at $\ell > 1000$ thus making it plausible to separate the two using $C_\ell$ template-fitting in a future survey like CMB-HD. Next, we look at the imprint of a single quasar bubble on the CMB and show that it can be detected in a high-resolution, ambitious effort like CMB-HD. A detection of a high redshift quasar bubble in the CMB would carry complimentary information to its detection in HI or Lyman-$α$ and a joint analysis can be used to break parameter degeneracies.

astro-ph.CO↗

arXiv2023

Constraints on cosmic curvature from cosmic chronometer and quasar observations

We consider cosmic chronometer (CC) data for the Hubble parameter, quasar (QSO) luminosities data of X-rays and ultraviolet rays emission, and the latest measurements of the present value of the Hubble parameter from 2018 Planck mission (PL18), and SH0ES observations (SHOES) to constrain the present value of cosmic curvature density parameter. We consider three kinds of dark energy models: the $Λ$CDM model, the wCDM model, and the CPL parametrization. In all these three models, we find higher values of the matter-energy density parameter, $Ω_{\rm m0}$ compared to the one obtained from the Planck 2018 mission of CMB observation. Also, we find evidence for a nonflat and closed Universe at 0.5$σ$ to 3$σ$ confidence levels. The flat Universe is almost 2 to 3$σ$, 1 to 1.5$σ$, and 0.5 to 1$σ$ away from the corresponding mean values, obtained in $Λ$CDM model, wCDM model, and CPL parametrization respectively obtained from different combinations of datasets. The evidence for nonzero cosmic curvature is lesser in dynamical dark energy models compared to the $Λ$CDM model. That means the evidence of nonzero cosmic curvature depends on the behavior of the equation of state of the dark energy. Since the values of the cosmic curvature are degenerate to the equation of state of the dark energy, we also consider a model independent analysis to constrain the cosmic curvature using the combination of Gaussian process regression analysis and artificial neural networks analysis. In the model independent analysis, we also find evidence for a closed Universe, and the flat Universe is almost 1$σ$ away. So, both the model dependent and independent analyses favor a closed Universe from the combinations of CC, QSO, and $H_0$ observations.

astro-ph.CO↗

arXiv2022

Network Security Modelling with Distributional Data

We investigate the detection of botnet command and control (C2) hosts in massive IP traffic using machine learning methods. To this end, we use NetFlow data -- the industry standard for monitoring of IP traffic -- and ML models using two sets of features: conventional NetFlow variables and distributional features based on NetFlow variables. In addition to using static summaries of NetFlow features, we use quantiles of their IP-level distributions as input features in predictive models to predict whether an IP belongs to known botnet families. These models are used to develop intrusion detection systems to predict traffic traces identified with malicious attacks. The results are validated by matching predictions to existing denylists of published malicious IP addresses and deep packet inspection. The usage of our proposed novel distributional features, combined with techniques that enable modelling complex input feature spaces result in highly accurate predictions by our trained models.

cs.CR↗

arXiv2022

Towards Algorithmic Fairness in Space-Time: Filling in Black Holes

New technologies and the availability of geospatial data have drawn attention to spatio-temporal biases present in society. For example: the COVID-19 pandemic highlighted disparities in the availability of broadband service and its role in the digital divide; the environmental justice movement in the United States has raised awareness to health implications for minority populations stemming from historical redlining practices; and studies have found varying quality and coverage in the collection and sharing of open-source geospatial data. Despite the extensive literature on machine learning (ML) fairness, few algorithmic strategies have been proposed to mitigate such biases. In this paper we highlight the unique challenges for quantifying and addressing spatio-temporal biases, through the lens of use cases presented in the scientific literature and media. We envision a roadmap of ML strategies that need to be developed or adapted to quantify and overcome these challenges -- including transfer learning, active learning, and reinforcement learning techniques. Further, we discuss the potential role of ML in providing guidance to policy makers on issues related to spatial fairness.

stat.AP↗

arXiv2022

The non-thermal secondary CMB anisotropies from a cosmic distribution of radio galaxy lobes

Current and upcoming high angular resolution and multi-frequency experiments are well poised to explore the rich landscape of secondary CMB anisotropies. In this context, we compute for the first time, the power spectrum of CMB fluctuations from a cosmological distribution of evolving lobes of giant radio galaxies. We, also, explicitly take into account the non-thermal electron distribution, which has important implications for the inference of the CMB angular power spectrum. We calculate the mean global non-thermal y-distortion, \ynt. For observationally reasonable distribution of the jet luminosities in the range of $10^{45}-10^{47}$ ergs$^{-1}$, we find \ynt to be less than $10^{-5}$, and hence not violating the COBE limit as previously claimed. Using the unique spectral dependence of the non-thermal SZ, we show that a detection of \ynt can be within reach at the level of $\gtrsim 5σ$ from a future PIXIE-like experiment provided we understand the foregrounds precisely. The total non-thermal SZ power spectrum, $C^{NT}_\ell$, from the radio lobes peaks at $\ell \sim 3000$ with an amplitude $\sim 1\%$ of thermal SZ power spectrum from galaxy clusters. A a detection of the $C^{NT}_\ell$, with a PIXIE-like sensitivity experiment, can lead to $\sim 5σ$ constraint on the mass dependence of the jet luminosity with the constraint becoming, at least, ten times better for the proposed more ambitious CMB-HD survey. This will, further, lead to the tightest constrain on the central black hole mass -to- host halo mass scaling relations.

astro-ph.CO↗

arXiv2022

Heating of the intracluster medium by buoyant bubbles and sound waves

Active galactic nuclei (AGN) powered by the central Super-Massive Black Holes (SMBHs) play a major role in modifying the thermal properties of the intracluster medium (ICM). In this work, we implement two AGN heating models: (i) by buoyant cavities rising through stratified ICM (effervescent model) and, (ii) by viscous and conductive dissipation of sound waves (acoustic model). Our aim is to determine whether these heating models are consistent with ICM observables and if one is preferred over the other. We assume an initial entropy profile of ICM that is expected from the purely gravitational infall of the gas in the potential of the dark matter halo. We then incorporate heating, radiative cooling, and thermal conduction to study the evolution of ICM over the age of the clusters. Our results are: (i) Both the heating processes can produce comparable thermal profiles of the ICM with some tuning of relevant parameters. (ii) Thermal conduction is crucially important, even at the level of 10\% of the Spitzer values, in transferring the injected energy beyond the central regions, and without which the temperature/entropy profiles are unrealistically high. (iii) The required injected AGN power scales with cluster mass as $M_{\rm vir}^{1.5}$ for both models. (iv) The required AGN luminosity is comparable with the observed radio jet power, reinforcing the idea that AGNs are the dominant heating source in clusters. (v) Finally, we estimate that the fraction of the total AGN luminosity available as the AGN mechanical luminosity at $0.02r_{500}$ is less than 0.05\%.

astro-ph.CO↗

arXiv2022

Exploring diffuse radio emission in galaxy clusters and groups with the uGMRT and the SKA

Diffuse radio emission has been detected in a considerable number of galaxy clusters and groups, revealing the presence of pervasive cosmic magnetic fields, and of relativistic particles in the large-scale structure (LSS) of the Universe. Since cluster radio emission is faint and steep spectrum, its observations are largely limited by the instrument sensitivity and frequency of observation, leading to a dearth of information, more so for lower-mass systems. The unprecedented sensitivity of recently commissioned low-frequency radio telescope arrays, aided by the development of advanced calibration and imaging techniques, have helped in achieving unparalleled image quality. At the same time, the development of sophisticated numerical simulations and the availability of supercomputing facilities have paved the way for high-resolution numerical modeling of radio emission, and the structure of the cosmic magnetic fields in LSS, leading to predictions matching the capabilities of observational facilities. In view of these rapidly-evolving scenerio in modeling and observations, in this review, we summarise the role of the new telescope arrays and the development of advanced imaging techniques and discuss the detections of various kinds of cluster radio sources. In particular, we discuss observations of the cosmic web in the form of supercluster filaments, studies of emission in poor clusters and groups of galaxies, and of ultra-steep spectrum sources. We also review the current theoretical understanding of various diffuse cluster radio sources and the associated magnetic field and polarization. As the statistics of detections improve along with our theoretical understanding, we update the source classification schemes based on their intrinsic properties. We conclude by summarising the role of the upgraded GMRT and our expectations from the upcoming Square Kilometre Array (SKA) observatories.

astro-ph.CO↗

arXiv2022

Feature Selection using e-values

In the context of supervised parametric models, we introduce the concept of e-values. An e-value is a scalar quantity that represents the proximity of the sampling distribution of parameter estimates in a model trained on a subset of features to that of the model trained on all features (i.e. the full model). Under general conditions, a rank ordering of e-values separates models that contain all essential features from those that do not. The e-values are applicable to a wide range of parametric models. We use data depths and a fast resampling-based algorithm to implement a feature selection procedure using e-values, providing consistency results. For a $p$-dimensional feature space, this procedure requires fitting only the full model and evaluating $p+1$ models, as opposed to the traditional requirement of fitting and evaluating $2^p$ models. Through experiments across several model settings and synthetic and real datasets, we establish that the e-values method as a promising general alternative to existing model-specific methods of feature selection.

stat.ML↗