arXiv subjects

Pragya Srivastava

Publications and source records attributed to Pragya Srivastava.

15 recordsLinked to original sources

arXiv2026

Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs

Many safety and alignment failures of large language models (LLMs) occur due to out-of-distribution (OOD) situations: unusual prompt or response patterns that are unforeseen by model developers. We systematically study whether LLM monitoring pipelines can detect these OOD alignment failures by introducing a benchmark called Misalignment Out Of Distribution (MOOD). It is difficult to find failures that are truly OOD for off-the-shelf models trained on vast safety datasets. We sidestep this by including a restricted training set in MOOD that we use to train our own monitors, as well as seven test sets with diverse alignment failures that are outside the training distribution. Using MOOD, we find that guard models (safety classifiers) often fail to generalize OOD. To fix this, we propose combining guard models with OOD detectors. We test four types of OOD detectors and find that a combination of a guard model with Mahalanobis distance and perplexity-based OOD detectors can improve recall from 39% to 45%. We also establish positive scaling trends across model scales for monitors that combine a guard model and OOD detector; we find that incorporating OOD detection into monitoring achieves a higher recall gain than using a guard model with 20 times more parameters. Our work suggests that OOD detection should be a crucial component of LLM monitoring and provides a foundation for further work on this important problem. We release the code and data for our experiments publicly, and you can find the relevant links here: https://github.com/Dylan102938/mood-bench.

cs.AI↗

arXiv2025

Evaluating LLMs' Mathematical Reasoning in Financial Document Question Answering

Large Language Models (LLMs), excel in natural language understanding, but their capability for complex mathematical reasoning with an amalgamation of structured tables and unstructured text is uncertain. This study explores LLMs' mathematical reasoning on four financial tabular question-answering datasets: TATQA, FinQA, ConvFinQA, and Multihiertt. Through extensive experiments with various models and prompting techniques, we assess how LLMs adapt to complex tables and mathematical tasks. We focus on sensitivity to table complexity and performance variations with an increasing number of arithmetic reasoning steps. The results provide insights into LLMs' capabilities and limitations in handling complex mathematical scenarios for semi-structured tables. Ultimately, we introduce a novel prompting technique tailored to semi-structured documents, matching or outperforming other baselines in performance while providing a nuanced understanding of LLMs abilities for such a task.

cs.CL↗

arXiv2025

Robust Reward Modeling via Causal Rubrics

Reward models (RMs) are fundamental to aligning Large Language Models (LLMs) via human feedback, yet they often suffer from reward hacking. They tend to latch on to superficial or spurious attributes, such as response length or formatting, mistaking these cues learned from correlations in training data for the true causal drivers of quality (e.g., factuality, relevance). This occurs because standard training objectives struggle to disentangle these factors, leading to brittle RMs and misaligned policies. We introduce Crome (Causally Robust Reward Modeling), a novel framework grounded in an explicit causal model designed to mitigate reward hacking. Crome employs the following synthetic targeted augmentations during training: (1) Causal Augmentations, which are pairs that differ along specific causal attributes, to enforce sensitivity along each causal attribute individually, and (2) Neutral Augmentations, which are tie-label pairs varying primarily in spurious attributes, to enforce invariance along spurious attributes. Notably, our augmentations are produced without any knowledge of spurious factors, via answer interventions only along causal rubrics, that are identified by querying an oracle LLM. Empirically, Crome significantly outperforms standard baselines on RewardBench, improving average accuracy by up to 5.4% and achieving gains of up to 13.2% and 7.2% in specific categories. The robustness of Crome is further testified by the consistent gains obtained in a Best-of-N inference setting across increasing N, across various benchmarks, including the popular RewardBench (covering chat, chat-hard, safety, and reasoning tasks), the safety-focused WildGuardTest, and the reasoning-specific GSM8k.

cs.LG↗

arXiv2025

Understanding and Mitigating Bottlenecks of State Space Models through the Lens of Recency and Over-smoothing

Structured State Space Models (SSMs) have emerged as alternatives to transformers. While SSMs are often regarded as effective in capturing long-sequence dependencies, we rigorously demonstrate that they are inherently limited by strong recency bias. Our empirical studies also reveal that this bias impairs the models' ability to recall distant information and introduces robustness issues. Our scaling experiments then discovered that deeper structures in SSMs can facilitate the learning of long contexts. However, subsequent theoretical analysis reveals that as SSMs increase in depth, they exhibit another inevitable tendency toward over-smoothing, e.g., token representations becoming increasingly indistinguishable. This fundamental dilemma between recency and over-smoothing hinders the scalability of existing SSMs. Inspired by our theoretical findings, we propose to polarize two channels of the state transition matrices in SSMs, setting them to zero and one, respectively, simultaneously addressing recency bias and over-smoothing. Experiments demonstrate that our polarization technique consistently enhances the associative recall accuracy of long-range tokens and unlocks SSMs to benefit further from deeper architectures. All source codes are released at https://github.com/VITA-Group/SSM-Bottleneck.

cs.LG↗

arXiv2024

NICE: To Optimize In-Context Examples or Not?

Recent work shows that in-context learning and optimization of in-context examples (ICE) can significantly improve the accuracy of large language models (LLMs) on a wide range of tasks, leading to an apparent consensus that ICE optimization is crucial for better performance. However, most of these studies assume a fixed or no instruction provided in the prompt. We challenge this consensus by investigating the necessity of optimizing ICE when task-specific instructions are provided and find that there are many tasks for which it yields diminishing returns. In particular, using a diverse set of tasks and a systematically created instruction set with gradually added details, we find that as the prompt instruction becomes more detailed, the returns on ICE optimization diminish. To characterize this behavior, we introduce a task-specific metric called Normalized Invariability to Choice of Examples (NICE) that quantifies the learnability of tasks from a given instruction, and provides a heuristic to help decide whether to optimize instructions or ICE for a new task. Given a task, the proposed metric can reliably predict the utility of optimizing ICE compared to using random ICE. Our code is available at https://github.com/microsoft/nice-icl.

cs.CL↗

arXiv2023

The information content of brain states is explained by structural constraints on state energetics

Signal propagation along the structural connectome of the brain induces changes in the patterns of activity. These activity patterns define global brain states and contain information in accordance with their expected probability of occurrence. The structural connectome, in conjunction with the dynamics, determines the set of possible brain states and constrains the transition between accessible states. Yet, precisely how these structural constraints on state-transitions relate to their information content remains unexplored. To address this gap in knowledge, we defined the information content as a function of the activation distribution, where statistically rare values of activation correspond to high information content. With this numerical definition in hand, we studied the spatiotemporal distribution of information content in fMRI data from the Human Connectome Project during different tasks, and report four key findings. First, information content strongly depends on the cognitive task. Second, while information content shows similarities to other measures of brain activity, it is distinct from both Neurosynth maps and task contrast maps generated by a general linear model applied to the fMRI data. Third, the brain's structural wiring constrains the cost to control its state, where the cost to transition into high information content states is larger than that to transition into low information content states. Finally, all state transitions - especially those to high information content states - are less costly than expected from random network null models, thereby indicating the brain's marked efficiency. Taken together, our findings establish an explanatory link between the information contained in a brain state and the energetic cost of attaining that state, thereby laying important groundwork for our understanding of large-scale cognitive computations.

q-bio.NC↗

arXiv2022

Towards Zero-Shot and Few-Shot Table Question Answering using GPT-3

We present very early results on using GPT-3 to perform question answering on tabular data. We find that stock pre-trained GPT-3 is able to zero-shot learn the table structure from a serialized JSON array-of-arrays representation, and able to answer lookup queries and simple comparison questions in natural language without any fine-tuning. We further find that simple prompt engineering to include few-shot static Q&A examples significantly improves accuracy. Lastly, we find that intermixing passage text improves accuracy even further on heterogeneous data. We apply our approach on a novel dataset of simple tables in newspaper infographics with promising results. Overall, we find much cause for optimism in this basic approach.

cs.LG↗

arXiv2021

Citation inequity and gendered citation practices in contemporary physics

The historical and contemporary under-attribution of women's contributions to scientific scholarship is well-known and well-studied, with effects that are felt today in myriad ways by women scientists. One measure of this under-attribution is the so-called citation gap between men and women: the under-citation of papers authored by women relative to expected rates coupled with a corresponding over-citation of papers authored by men relative to expected rates. We explore the citation gap in contemporary physics, analyzing over one million articles published over the last 25 years in 35 physics journals that span a wide range of subfields. Using a model that predicts papers' expected citation rates according to a set of characteristics separate from author gender, we find a global bias wherein papers authored by women are significantly under-cited, and papers authored by men are significantly over-cited. Moreover, we find that citation behavior varies along several dimensions, such that imbalances differ according to who is citing, where they are citing, and how they are citing. Specifically, citation imbalance in favor of man-authored papers is highest for papers authored by men, papers published in general physics journals, and papers likely to be less familiar to citing authors. Our results suggest that, although deciding which papers to cite is an individual choice, the cumulative effects of these choices needlessly harm a subset of scholars. We discuss several strategies for the mitigation of these effects, including conscious behavioral changes at the individual, journal, and community levels.

physics.soc-ph↗

arXiv2021

Structural underpinnings of control in multiplex networks

To design control strategies that predictably manipulate a system's behavior, it is first necessary to understand how the system's structure relates to its response. Many complex systems can be represented as multilayer networks whose response and control can be studied in the framework of network control theory. It remains unknown how a system's layered architecture dictates its control properties, particularly when control signals can only access the system through a single input layer. Here, we use the framework of linear control theory to probe the control properties of a duplex network with directed interlayer links. We determine the manner in which the structural properties of layers and relative interlayer arrangement together dictate the system's response. For this purpose, we calculate the exact expression of optimal control energy in terms of layer spectra and the relative alignment between the eigenmodes of the input layer and the deeper target layer. For a range of numerically constructed duplex networks, we then calculate the control properties of the two layers as a function of target-layer densities and layer topology. The alignment of layer eigenmodes emerges as an important parameter that sets the cost and routing of the optimal energy for control. We understand these results in a simplified limit of a single-mode approximation, and we build metrics to characterize the routing of optimal control energy through the eigenmodes of each layer. Our analytical and numerical results together provide insights into the relationships between structure and control in duplex networks. More generally, they serve as a platform for future work designing optimal \emph{interlayer} control strategies.

q-bio.NC↗

arXiv2020

Models of communication and control for brain networks: distinctions, convergence, and future outlook

Recent advances in computational models of signal propagation and routing in the human brain have underscored the critical role of white matter structure. A complementary approach has utilized the framework of network control theory to better understand how white matter constrains the manner in which a region or set of regions can direct or control the activity of other regions. Despite the potential for both of these approaches to enhance our understanding of the role of network structure in brain function, little work has sought to understand the relations between them. Here, we seek to explicitly bridge computational models of communication and principles of network control in a conceptual review of the current literature. By drawing comparisons between communication and control models in terms of the level of abstraction, the dynamical complexity, the dependence on network attributes, and the interplay of multiple spatiotemporal scales, we highlight the convergence of and distinctions between the two frameworks. Based on the understanding of the intertwined nature of communication and control in human brain networks, this work provides an integrative perspective for the field and outlines exciting directions for future work.

q-bio.NC↗

arXiv2019

Swimmer suspensions on substrates: anomalous stability and long-range order

We present a comprehensive theory of the dynamics and fluctuations of a two-dimensional suspension of polar active particles in an incompressible fluid confined to a substrate. We show that, depending on the sign of a single parameter, a state with polar orientational order is anomalously stable (or anomalously unstable), with a nonzero relaxation (or growth) rate for angular fluctuations at zero wavenumber. This screening of the broken-symmetry mode in the stable state does lead to conventional rather than giant number fluctuations as argued by Bricard et al., Nature ${\bf 503}$, 95 (2013), but their bend instability in a splay-stable flock does not exist and the polar phase has long-range order in two dimensions. Our theory also describes confined three-dimensional thin-film suspensions of active polar particles as well as dense compressible active polar rods, and predicts a flocking transition without a banding instability

cond-mat.soft↗

arXiv2017

Stability from activity

Suspensions of actively driven anisotropic objects exhibit distinctively nonequilibrium behaviors, and current theories predict that they are incapable of sustaining orientational order at high activity. By contrast, here we show that nematic suspensions on a substrate can display order at arbitrarily high activity due to a previously unreported, potentially stabilizing active force. The resulting nonequilibrium ordered phase displays robust giant number fluctuations that cannot be suppressed even by an incompressible solvent. Our results apply to virtually all experimental assays used to investigate the active nematic ordering of self-propelled colloids, bacterial suspensions and the cytoskeleton, and have testable implications in interpreting their nonequilibrium behaviors

cond-mat.soft↗

arXiv2016

Negative stiffness and modulated states in active nematics

We examine the dynamics of a compressible active nematic liquid crystal on a frictional substrate. When frictional damping dominates over viscous dissipation, we eliminate flow in favor of active stresses to obtain a minimal dynamical model for the nematic order parameter, with elastic constants renormalized by activity. The renormalized elastic constants can become negative at large activity, leading to the selection of spatially inhomogeneous patterns via a mechanism analogous to that responsible for modulated phases arising at an equilibrium Lifshitz point. Tuning activity and the degree of nematic order in the passive system, we obtain a linear stability phase diagram that exhibits a nonequilibrium tricritical point where ordered, modulated and disordered phases meet. Numerical solution of the nonlinear equations yields a succession of spatial structures of increasing complexity with increasing activity, including kink walls and active turbulence, as observed in experiments on microtubule bundles confined at an oil-water interface. Our work provides a minimal model for an overdamped active nematic that reproduces all the nonequilibrium structures seen in simulations of the full active nematic hydrodynamics and provides a framework for understanding some of the mechanisms for selection of the nonequilibrium patterns in the language of equilibrium critical phenomena.

cond-mat.soft↗

arXiv2014

Activating membranes

We present a general dynamical theory of a membrane coupled to an actin cortex containing polymerizing filaments with active stresses and currents, and demonstrate that active membrane dynamics [Phys. Rev. Lett \textbf{84}, 3494 (2000)] and spontaneous shape oscillations emerge from this description. We also consider membrane instabilities and patterns induced by the presence of filaments with polar orientational correlations in the tangent plane of the membrane. The dynamical features we predict should be seen in a variety of cellular contexts involving the dynamics of the membrane-cytoskeleton composite and cytoskeletal extracts coupled to synthetic vesicles.

physics.bio-ph↗

arXiv2012

Patterning of polar active filaments on a tense cylindrical membrane

We study the dynamics and patterning of polar contractile filaments on the surface of a cylindrical cell using active hydrodynamic equations that incorporate couplings between curvature and filament orientation. Cables and rings spontaneously emerge as steady state configurations on the cylinder, and can be stationary or moving, helical or segments moving along helical trajectories. Contractility induces coalescence of proximal rings. We observe phase transitions in the steady state patterns upon changing cell diameter and make several testable predictions. Our results are relevant to the dynamics and patterning of a variety of active biopolymers in cylindrical cells.

cond-mat.soft↗