arXiv subjects

Anirban Roy

Publications and source records attributed to Anirban Roy.

At least 19 recordsLinked to original sources

arXiv2026

Consilience: Conformally Calibrated Communication Control for Hidden-Profile Multi-Agent Reasoning

Multi-agent LLM systems can improve reasoning by pooling diverse perspectives, but their effectiveness depends on coordinating communication, particularly in hidden-profile settings where each agent holds only part of the evidence required for a correct decision. Existing protocols, including fixed schedules, round-robin exchange, and unstructured debate, provide no guarantee that a conversational action is appropriate. We propose Consilience, an inference-time orchestration framework that both steers and certifies multi-agent communication under distributed private information. At each turn, Consilience summarizes the discussion using a compact state capturing uncertainty, disagreement, evidence gain, redundancy, and premature consensus, then selects both a communication intervention (challenge, clarify, seek evidence, or route) and an appropriate speaker. Its central contribution is a round-wise conformal calibration procedure that provides a distribution-free, finite-sample guarantee: at each discussion round, conditional on reaching that round, the one-step regret of a controller's proposed action is bounded by a calibrated threshold with marginal probability at least 1 - alpha; an acceptance mechanism enforces the same guarantee for the executed action by replacing inadmissible proposals. On HiddenBench-style hidden-profile tasks spanning 12 open and closed weight language models, Consilience improves decision accuracy and communication efficiency over fixed and unstructured discussion protocols, sometimes surpassing a full-information baseline where every agent observes all evidence. These results demonstrate that certified adaptive communication control can be more valuable than increasing information availability, providing a practical mechanism for reliable multi-agent LLM coordination.

cs.AI↗

arXiv2026

Beyond the Transcript: Detecting Covert Co ordination in Latent Multi-Agent Communication

Language-model agents can communicate through continuous hidden states that are invisible in public transcripts, creating opportunities for covert harmful coordination. We introduce Verifiable Latent Alignments (VLA), an activation-aware framework for monitoring and steering these private communication channels. For every monitored decision, VLA links the private latent-state record and channel status to the resulting public action using a shared event identifier, enabling matched causal analysis. Our first contribution is a neutral-only three-layer monitor combining representation anomaly detection, counterfactual action-distribution influence, and sparse-autoencoder interpretation support. Our second contribution is a steerability framework spanning black-box behavioral instructions and white-box matched-neutral counterfactuals. Our third contribution is an evaluation on a controlled multi-agent auction benchmark covering homogeneous and heterogeneous model pairs, many-agent scalability, and intervention effectiveness. The sequential monitor achieves mean area under the receiver operating characteristic curve (AUROC) of 0.993 for homogeneous agents and 0.854 for heterogeneous pairs when text- and latent-collusion rows are pooled as positives. In Qwen3-0.6B auctions with 25-100 bidders, monitoring requires only a small normalized load relative to all possible directed pairs, while full white-box steering achieves 100% bid-distribution recovery and reduces collusive low-bid behavior by 47.3 percentage points. Because full white-box steering replays the matched neutral counterfactual, its exact recovery is a sanity check by construction. Overall, the controlled study shows that the evaluated private channel attacks can be monitored without training the primary monitor on attack examples and mitigated when matched counterfactual access is available.

cs.AI↗

arXiv2026

Interlopers as Signal in Line Intensity Mapping

Line intensity mapping measures the combined emission of atomic and molecular lines from unresolved galaxies, offering a way to map cosmic structure across enormous volumes. A single observed frequency, however, contains emission from several spectral lines at different redshifts. These interlopers are usually removed before cosmological inference. We show that this removal is not always optimal. Once the possible emitting lines and their redshifts are included in the model, interlopers become additional tracers of large-scale structure at other epochs, projected into the coordinate system of the target line. We formulate interloper treatment as a Fisher decision problem, requiring both improved precision and parameter bias below a chosen tolerance. In SPHEREx forecasts for H$\alpha$ with [O III], H$\beta$, and [O II] interlopers, and in FYST forecasts for [C II] with CO interlopers, $\Omega_m h^2$ and baryon acoustic oscillation (BAO) distance measurements are comparatively robust to astrophysical calibration errors, whereas the clustering amplitude $\sigma_8$ and the dark energy parameters $w_0$ and $w_a$ are more sensitive to the adopted line model and priors. The BAO distance measurement gives a simple physical picture: modeling interlopers turns two observed bands into a transverse BAO distance ladder over $0.7\lesssim z\lesssim5.8$. Interlopers are therefore not only contaminants to remove, but a calibrated component of the cosmological signal model for future LIM surveys.

astro-ph.CO↗

arXiv2026

Reading Between the Lines: Forward Modeling Dust, Continuum, and Spectral Cleaning for Multi-Line Intensity Mapping

Line intensity mapping (LIM) offers a tomographic view of galaxy evolution by measuring the aggregate emission from unresolved galaxies. In the optical and near-infrared, the line emission is accompanied by much brighter continuum emission that must be removed to recover line auto- and cross-power spectra. We develop a forward-modeling framework using a $\sim2$deg$^2$ lightcone drawn from a cosmological $N$-body simulation populated with galaxies using a physics-based semi-analytic model (SAM). We construct intensity maps for the stellar continuum and for the strongest optical lines, H$\alpha$, H$\beta$, [O III] $\lambda5007$, and [O II] $\lambda3727$, including nebular dust attenuation tied to galaxy properties. From these cubes, we measure cross-channel angular power spectra, $C_\ell(\lambda_i,\lambda_j)$, and the normalized correlation matrix $r_{ij}$. In line-only maps, the correlation matrices show same-redshift ridges between emission lines, demonstrating how multi-line intensity mapping (MLIM) can isolate large-scale structure and probe dust attenuation. We show that dust suppresses the line cross-power by an amount that depends on galaxy properties, wavelength, and line pair, so a single overall amplitude cannot capture its effect. In total maps, however, the continuum dominates the raw correlations and hides much of the line-ridge structure. We therefore apply principal component analysis (PCA)-based spectral cleaning and quantify the line-transfer function using the simulation truth. Removing 20 PCA modes gives the best trade-off between line recovery and continuum suppression in our mock maps. Our results demonstrate the promise and challenges of extracting dust-sensitive LIM observables from SPHEREx-like observations, and highlight the need to model continuum cleaning and its transfer function in quantitative inference pipelines.

astro-ph.GA↗

arXiv2026

Bayesian Adaptation Gym: A Benchmark for the Bayesian Low-Rank Adaptation of Multi-Modal Language Models

Large multi-modal language models are increasingly deployed in high-stakes domains, making well-calibrated uncertainty essential. Traditional Bayesian methods approximate posteriors over all model weights, which becomes intractable for modern large models. For this reason, recent work instead considers Bayesian low-rank adaptation to enable tractable posterior approximation. Due to a lack of a standardized benchmark to evaluate these approaches, it remains unclear where these methods provide meaningful benefits. To fill this gap, we introduce Bayesian Adaptation Gym (BAG), a benchmark for the Bayesian adaptation of multi-modal language models. BAG provides reference implementations of classic Bayesian baselines and state-of-the-art adaptation methods, along with a multi-modal dataset and task suite designed to probe calibration, robustness under distribution shift, and decision-making under uncertainty via active learning. Using BAG, we conduct and report extensive experiments across model sizes, datasets, and tasks to highlight the successes and failures of current Bayesian adaptation approaches. To enable further research, BAG is fully open source: https://github.com/SRI-CSL/BayesAdapt.

cs.LG↗

arXiv2026

Closed-Loop Neural Activation Control in Vision-Language-Action Models

Vision-Language-Action (VLA) models can be steered at test time by intervening on semantically meaningful internal directions, but existing methods use a fixed steering coefficient, effectively operating in open loop. This is poorly suited to embodied control, where task state and concept error evolve over time, often causing overcorrection, oscillation, and reduced task success, especially for temporal behaviors such as speed and smoothness. We propose CTRL-STEER, a closed-loop framework that replaces static intervention strength with adaptive, time-varying control signals. The key idea is to decouple representation from regulation: rather than assuming temporal concepts are directly controlled by individual neurons, we steer along motion-aligned residual directions while a feedback controller adjusts intervention magnitude online. We instantiate this framework with both PID and reinforcement learning based controllers. Experiments with a fine-tuned OpenVLA policy on four LIBERO task suites show that CTRL-STEER achieves more stable concept regulation and a better steering-task success trade-off than fixed-coefficient baselines, without modifying or retraining the base model.

cs.AI↗

arXiv2026

Breaking Bad: Interpretability-Based Safety Audits of State-of-the-Art LLMs

Effective safety auditing of large language models (LLMs) demands tools that go beyond black-box probing and systematically uncover vulnerabilities rooted in model internals. We present a comprehensive, interpretability-driven jailbreaking audit of eight SOTA open-source LLMs: Llama-3.1-8B, Llama-3.3-70B-4bt, GPT-oss- 20B, GPT-oss-120B, Qwen3-0.6B, Qwen3-32B, Phi4-3.8B, and Phi4-14B. Leveraging interpretability-based approaches -- Universal Steering (US) and Representation Engineering (RepE) -- we introduce an adaptive two-stage grid search algorithm to identify optimal activation-steering coefficients for unsafe behavioral concepts. Our evaluation, conducted on a curated set of harmful queries and a standardized LLM-based judging protocol, reveals stark contrasts in model robustness. The Llama-3 models are highly vulnerable, with up to 91\% (US) and 83\% (RepE) jailbroken responses on Llama-3.3-70B-4bt, while GPT-oss-120B remains robust to attacks via both interpretability approaches. Qwen and Phi models show mixed results, with the smaller Qwen3-0.6B and Phi4-3.8B mostly exhibiting lower jailbreaking rates, while their larger counterparts are more susceptible. Our results establish interpretability-based steering as a powerful tool for systematic safety audits, but also highlight its dual-use risks and the need for better internal defenses in LLM deployment.

cs.CR↗

arXiv2026

From Actions to Understanding: Conformal Interpretability of Temporal Concepts in LLM Agents

Large Language Models (LLMs) are increasingly deployed as autonomous agents capable of reasoning, planning, and acting within interactive environments. Despite their growing capability to perform multi-step reasoning and decision-making tasks, internal mechanisms guiding their sequential behavior remain opaque. This paper presents a framework for interpreting the temporal evolution of concepts in LLM agents through a step-wise conformal lens. We introduce the conformal interpretability framework for temporal tasks, which combines step-wise reward modeling with conformal prediction to statistically label model's internal representation at each step as successful or failing. Linear probes are then trained on these representations to identify directions of temporal concepts - latent directions in the model's activation space that correspond to consistent notions of success, failure or reasoning drift. Experimental results on two simulated interactive environments, namely ScienceWorld and AlfWorld, demonstrate that these temporal concepts are linearly separable, revealing interpretable structures aligned with task success. We further show preliminary results on improving an LLM agent's performance by leveraging the proposed framework for steering the identified successful directions inside the model. The proposed approach, thus, offers a principled method for early failure detection as well as intervention in LLM-based agents, paving the path towards trustworthy autonomous language models in complex interactive settings.

cs.AI↗

arXiv2026

Do Diffusion Models Dream of Electric Planes? Discrete and Continuous Simulation-Based Inference for Aircraft Design

In this paper, we generate conceptual engineering designs of electric vertical take-off and landing (eVTOL) aircraft. We follow the paradigm of simulation-based inference (SBI), whereby we look to learn a posterior distribution over the full eVTOL design space. To learn this distribution, we sample over discrete aircraft configurations (topologies) and their corresponding set of continuous parameters. Therefore, we introduce a hierarchical probabilistic model consisting of two diffusion models. The first model leverages recent work on Riemannian Diffusion Language Modeling (RDLM) and Unified World Models (UWMs) to enable us to sample topologies from a discrete and continuous space. For the second model we introduce a masked diffusion approach to sample the corresponding parameters conditioned on the topology. Our approach rediscovers known trends and governing physical laws in aircraft design, while significantly accelerating design generation.

cs.LG↗

arXiv2025

Investigating the Reionization Epoch through 21\,cm and Line Intensity Mapping Experiments

The epoch of reionization (EoR), marking the Universe's transition from a neutral to ionized state, represents a pivotal phase for understanding the formation of the first stars and galaxies. Intensity mapping of atomic and molecular lines, such as $[\mathrm{CII}]$ and CO J-ladder transitions, across a broad redshift range is a powerful tool for investigating star formation history, metallicity, the distribution of gas and dust, and the physical conditions within galaxies. Additionally, 21\,cm line intensity mapping directly probes the neutral hydrogen content in the intergalactic medium, offering a unique window into the timing and morphology of reionization. In this study, we explore the cross-correlation between the 21\,cm signal and multi-line intensity mapping (LIM) to forecast their detectability for next-generation experiments. Our analysis emphasizes the complementary potential of these techniques to constrain parameters such as the minimum mass of ionizing sources and the ionization fraction $x_e$. Cross-correlations with LIM also enable constraints on physical properties like metal enrichment and the relationship between star formation rates and multi-line luminosities. Using mock observations from Square Kilometre Array (SKA)-low 21\,cm and Fred Young Submillimeter Telescope (FYST)-like LIM experiments, we find that the $[\mathrm{CII}]$--21\,cm cross-correlation can constrain reionization history by measuring $x_e$ across multiple redshift bins with significance levels ranging from 9 to 40$\sigma$. We extend our analysis to CO transitions, showing that the CO(1-0)--21\,cm cross-correlation provides competitive constraints on reionization parameters. The synergies explored here will enable robust constraints on both reionization and LIM parameters, maximizing the scientific return of current and next-generation intensity mapping experiments.

astro-ph.GA↗

arXiv2025

Class Prototypes based Contrastive Learning for Classifying Multi-Label and Fine-Grained Educational Videos

The recent growth in the consumption of online media by children during early childhood necessitates data-driven tools enabling educators to filter out appropriate educational content for young learners. This paper presents an approach for detecting educational content in online videos. We focus on two widely used educational content classes: literacy and math. For each class, we choose prominent codes (sub-classes) based on the Common Core Standards. For example, literacy codes include `letter names', `letter sounds', and math codes include `counting', `sorting'. We pose this as a fine-grained multilabel classification problem as videos can contain multiple types of educational content and the content classes can get visually similar (e.g., `letter names' vs `letter sounds'). We propose a novel class prototypes based supervised contrastive learning approach that can handle fine-grained samples associated with multiple labels. We learn a class prototype for each class and a loss function is employed to minimize the distances between a class prototype and the samples from the class. Similarly, distances between a class prototype and the samples from other classes are maximized. As the alignment between visual and audio cues are crucial for effective comprehension, we consider a multimodal transformer network to capture the interaction between visual and audio cues in videos while learning the embedding for videos. For evaluation, we present a dataset, APPROVE, employing educational videos from YouTube labeled with fine-grained education classes by education researchers. APPROVE consists of 193 hours of expert-annotated videos with 19 classes. The proposed approach outperforms strong baselines on APPROVE and other benchmarks such as Youtube-8M, and COIN. The dataset is available at https://github.com/rohit-gupta/MMContrast/tree/main/APPROVE

cs.CV↗

arXiv2025

Privacy Preserving In-Context-Learning Framework for Large Language Models

Large language models (LLMs) have significantly transformed natural language understanding and generation, but they raise privacy concerns due to potential exposure of sensitive information. Studies have highlighted the risk of information leakage, where adversaries can extract sensitive information embedded in the prompts. In this work, we introduce a novel private prediction framework for generating high-quality synthetic text with strong privacy guarantees. Our approach leverages the Differential Privacy (DP) framework to ensure worst-case theoretical bounds on information leakage without requiring any fine-tuning of the underlying models. The proposed method performs inference on private records and aggregates the resulting per-token output distributions. This enables the generation of longer and coherent synthetic text while maintaining privacy guarantees. Additionally, we propose a simple blending operation that combines private and public inference to further enhance utility. Empirical evaluations demonstrate that our approach outperforms previous state-of-the-art methods on in-context-learning (ICL) tasks, making it a promising direction for privacy-preserving text generation while maintaining high utility. Our code is available at https://github.com/bhusalb/privacy-preserving-icl.

cs.LG↗

arXiv2025

Polysemantic Dropout: Conformal OOD Detection for Specialized LLMs

We propose a novel inference-time out-of-domain (OOD) detection algorithm for specialized large language models (LLMs). Despite achieving state-of-the-art performance on in-domain tasks through fine-tuning, specialized LLMs remain vulnerable to incorrect or unreliable outputs when presented with OOD inputs, posing risks in critical applications. Our method leverages the Inductive Conformal Anomaly Detection (ICAD) framework, using a new non-conformity measure based on the model's dropout tolerance. Motivated by recent findings on polysemanticity and redundancy in LLMs, we hypothesize that in-domain inputs exhibit higher dropout tolerance than OOD inputs. We aggregate dropout tolerance across multiple layers via a valid ensemble approach, improving detection while maintaining theoretical false alarm bounds from ICAD. Experiments with medical-specialized LLMs show that our approach detects OOD inputs better than baseline methods, with AUROC improvements of $2\%$ to $37\%$ when treating OOD datapoints as positives and in-domain test datapoints as negatives.

cs.CL↗

arXiv2025

The Modeling Landscape of Extragalactic CO in CMB Surveys

Extragalactic carbon monoxide (CO) line emission will likely be an important signal in current and future Cosmic Microwave Background (CMB) surveys on small scales. However, great uncertainty surrounds our current understanding of CO emission. We investigate the implications of this modeling uncertainty on CMB surveys. Using a range of star formation rate and luminosity relations, we generate a suite of CO simulations across cosmic time, together with the broadband cosmic infrared background (CIB). From these, we quantify the power spectrum signatures of CO that we would observe in a CMB experiment at 90, 150, and 220 GHz. We find that the resulting range of CO auto-spectra spans up to two orders of magnitude and that while CO on its own is unlikely to be detectable in current CMB experiments, its cross-correlation with the CIB will be a significant CMB foreground in future surveys. We then forecast the bias on CMB foregrounds that would result if CO were neglected in a CMB power spectrum analysis, finding shifts that can be comparable to some of the uncertainties on CMB foreground constraints from recent surveys, particularly for the thermal and kinetic Sunyaev-Zel'dovich effects and radio sources, and many times greater than the expected uncertainties expected for future data. Finally, we assess how the broad range of multifrequency CO$\times$CIB spectra we obtain is captured by a reduced parameter set by performing a principal component analysis, finding that three amplitude parameters suffice for a CMB-S4-like survey. Our results demonstrate the importance for future CMB experiments to account for a wide range of CO modeling, and that high-precision CMB experiments may help constrain extragalactic CO models.

astro-ph.CO↗

arXiv2025

TOGA: Temporally Grounded Open-Ended Video QA with Weak Supervision

We address the problem of video question answering (video QA) with temporal grounding in a weakly supervised setup, without any temporal annotations. Given a video and a question, we generate an open-ended answer grounded with the start and end time. For this task, we propose TOGA: a vision-language model for Temporally Grounded Open-Ended Video QA with Weak Supervision. We instruct-tune TOGA to jointly generate the answer and the temporal grounding. We operate in a weakly supervised setup where the temporal grounding annotations are not available. We generate pseudo labels for temporal grounding and ensure the validity of these labels by imposing a consistency constraint between the question of a grounding response and the response generated by a question referring to the same temporal segment. We notice that jointly generating the answers with the grounding improves performance on question answering as well as grounding. We evaluate TOGA on grounded QA and open-ended QA tasks. For grounded QA, we consider the NExT-GQA benchmark which is designed to evaluate weakly supervised grounded question answering. For open-ended QA, we consider the MSVD-QA and ActivityNet-QA benchmarks. We achieve state-of-the-art performance for both tasks on these benchmarks.

cs.CV↗

arXiv2025

Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding

We introduce a novel approach for calibrating uncertainty quantification (UQ) tailored for multi-modal large language models (LLMs). Existing state-of-the-art UQ methods rely on consistency among multiple responses generated by the LLM on an input query under diverse settings. However, these approaches often report higher confidence in scenarios where the LLM is consistently incorrect. This leads to a poorly calibrated confidence with respect to accuracy. To address this, we leverage cross-modal consistency in addition to self-consistency to improve the calibration of the multi-modal models. Specifically, we ground the textual responses to the visual inputs. The confidence from the grounding model is used to calibrate the overall confidence. Given that using a grounding model adds its own uncertainty in the pipeline, we apply temperature scaling - a widely accepted parametric calibration technique - to calibrate the grounding model's confidence in the accuracy of generated responses. We evaluate the proposed approach across multiple multi-modal tasks, such as medical question answering (Slake) and visual question answering (VQAv2), considering multi-modal models such as LLaVA-Med and LLaVA. The experiments demonstrate that the proposed framework achieves significantly improved calibration on both tasks.

cs.CL↗

arXiv2025

Safety Monitoring for Learning-Enabled Cyber-Physical Systems in Out-of-Distribution Scenarios

The safety of learning-enabled cyber-physical systems is compromised by the well-known vulnerabilities of deep neural networks to out-of-distribution (OOD) inputs. Existing literature has sought to monitor the safety of such systems by detecting OOD data. However, such approaches have limited utility, as the presence of an OOD input does not necessarily imply the violation of a desired safety property. We instead propose to directly monitor safety in a manner that is itself robust to OOD data. To this end, we predict violations of signal temporal logic safety specifications based on predicted future trajectories. Our safety monitor additionally uses a novel combination of adaptive conformal prediction and incremental learning. The former obtains probabilistic prediction guarantees even on OOD data, and the latter prevents overly conservative predictions. We evaluate the efficacy of the proposed approach in two case studies on safety monitoring: 1) predicting collisions of an F1Tenth car with static obstacles, and 2) predicting collisions of a race car with multiple dynamic obstacles. We find that adaptive conformal prediction obtains theoretical guarantees where other uncertainty quantification methods fail to do so. Additionally, combining adaptive conformal prediction and incremental learning for safety monitoring achieves high recall and timeliness while reducing loss in precision. We achieve these results even in OOD settings and outperform alternative methods.

cs.LG↗

arXiv2025

TeleLoRA: Teleporting Model-Specific Alignment Across LLMs

Mitigating Trojans in Large Language Models (LLMs) is one of many tasks where alignment data is LLM specific, as different LLMs have different Trojan triggers and trigger behaviors to be removed. In this paper, we introduce TeleLoRA (Teleporting Low-Rank Adaptation), a novel framework that synergizes model-specific alignment data across multiple LLMs to enable zero-shot Trojan mitigation on unseen LLMs without alignment data. TeleLoRA learns a unified generator of LoRA adapter weights by leveraging local activation information across multiple LLMs. This generator is designed to be permutation symmetric to generalize across models with different architectures and sizes. We optimize the model design for memory efficiency, making it feasible to learn with large-scale LLMs with minimal computational resources. Experiments on LLM Trojan mitigation benchmarks demonstrate that TeleLoRA effectively reduces attack success rates while preserving the benign performance of the models.

cs.LG↗