arXiv subjects

Isha Gupta

Publications and source records attributed to Isha Gupta.

11 recordsLinked to original sources

arXiv2026

AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors

We introduce AuditBench, an alignment auditing benchmark. AuditBench consists of 56 language models with implanted hidden behaviors. Each model has one of 14 concerning behaviors--such as sycophantic deference, opposition to AI regulation, or secret geopolitical loyalties--which it does not confess to when directly asked. AuditBench models are highly diverse--some are subtle, while others are overt, and we use varying training techniques both for implanting behaviors and training models not to confess. To demonstrate AuditBench's utility, we develop an investigator agent that autonomously employs a configurable set of auditing tools. By measuring investigator agent success using different tools, we can evaluate their efficacy. Notably, we observe a tool-to-agent gap, where tools that perform well in standalone non-agentic evaluations fail to translate into improved performance when used with our investigator agent. We find that our most effective tools involve scaffolded calls to auxiliary models that generate diverse prompts for the target. White-box interpretability tools can be helpful, but the agent performs best with black-box tools. We also find that audit success varies greatly across training techniques: models trained on synthetic documents are easier to audit than models trained on demonstrations, with better adversarial training further increasing auditing difficulty. We release our models, agent, and evaluation framework to support future quantitative, iterative science on alignment auditing.

cs.CL↗

arXiv2025

Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed

The field of adversarial robustness has long established that adversarial examples can successfully transfer between image classifiers and that text jailbreaks can successfully transfer between language models (LMs). However, a pair of recent studies reported being unable to successfully transfer image jailbreaks between vision-language models (VLMs). To explain this striking difference, we propose a fundamental distinction regarding the transferability of attacks against machine learning models: attacks in the input data-space can transfer, whereas attacks in model representation space do not, at least not without geometric alignment of representations. We then provide theoretical and empirical evidence of this hypothesis in four different settings. First, we mathematically prove this distinction in a simple setting where two networks compute the same input-output map but via different representations. Second, we construct representation-space attacks against image classifiers that are as successful as well-known data-space attacks, but fail to transfer. Third, we construct representation-space attacks against LMs that successfully jailbreak the attacked models but again fail to transfer. Fourth, we construct data-space attacks against VLMs that successfully transfer to new VLMs, and we show that representation space attacks can transfer when VLMs' latent geometries are sufficiently aligned in post-projector space. Our work reveals that adversarial transfer is not an inherent property of all attacks but contingent on their operational domain - the shared data-space versus models' unique representation spaces - a critical insight for building more robust models.

cs.LG↗

arXiv2025

Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track

Science progresses by iteratively advancing and correcting humanity's understanding of the world. In machine learning (ML) research, rapid advancements have led to an explosion of publications, but have also led to misleading, incorrect, flawed or perhaps even fraudulent studies being accepted and sometimes highlighted at ML conferences due to the fallibility of peer review. While such mistakes are understandable, ML conferences do not offer robust processes to help the field systematically correct when such errors are made. This position paper argues that ML conferences should establish a dedicated "Refutations and Critiques" (R&C) Track. This R&C Track would provide a high-profile, reputable platform to support vital research that critically challenges prior research, thereby fostering a dynamic self-correcting research ecosystem. We discuss key considerations including track design, review principles, potential pitfalls, and provide an illustrative example submission concerning a recent ICLR 2025 Oral. We conclude that ML conferences should create official, reputable mechanisms to help ML research self-correct.

cs.LG↗

arXiv2025

"I am bad": Interpreting Stealthy, Universal and Robust Audio Jailbreaks in Audio-Language Models

The rise of multimodal large language models has introduced innovative human-machine interaction paradigms but also significant challenges in machine learning safety. Audio-Language Models (ALMs) are especially relevant due to the intuitive nature of spoken communication, yet little is known about their failure modes. This paper explores audio jailbreaks targeting ALMs, focusing on their ability to bypass alignment mechanisms. We construct adversarial perturbations that generalize across prompts, tasks, and even base audio samples, demonstrating the first universal jailbreaks in the audio modality, and show that these remain effective in simulated real-world conditions. Beyond demonstrating attack feasibility, we analyze how ALMs interpret these audio adversarial examples and reveal them to encode imperceptible first-person toxic speech - suggesting that the most effective perturbations for eliciting toxic outputs specifically embed linguistic features within the audio signal. These results have important implications for understanding the interactions between different modalities in multimodal models, and offer actionable insights for enhancing defenses against adversarial audio attacks.

cs.LG↗

arXiv2024

Fragile Giants: Understanding the Susceptibility of Models to Subpopulation Attacks

As machine learning models become increasingly complex, concerns about their robustness and trustworthiness have become more pressing. A critical vulnerability of these models is data poisoning attacks, where adversaries deliberately alter training data to degrade model performance. One particularly stealthy form of these attacks is subpopulation poisoning, which targets distinct subgroups within a dataset while leaving overall performance largely intact. The ability of these attacks to generalize within subpopulations poses a significant risk in real-world settings, as they can be exploited to harm marginalized or underrepresented groups within the dataset. In this work, we investigate how model complexity influences susceptibility to subpopulation poisoning attacks. We introduce a theoretical framework that explains how overparameterized models, due to their large capacity, can inadvertently memorize and misclassify targeted subpopulations. To validate our theory, we conduct extensive experiments on large-scale image and text datasets using popular model architectures. Our results show a clear trend: models with more parameters are significantly more vulnerable to subpopulation poisoning. Moreover, we find that attacks on smaller, human-interpretable subgroups often go undetected by these models. These results highlight the need to develop defenses that specifically address subpopulation vulnerabilities.

cs.LG↗

arXiv2024

iToT: An Interactive System for Customized Tree-of-Thought Generation

As language models have become increasingly successful at a wide array of tasks, different prompt engineering methods have been developed alongside them in order to adapt these models to new tasks. One of them is Tree-of-Thoughts (ToT), a prompting strategy and framework for language model inference and problem-solving. It allows the model to explore multiple solution paths and select the best course of action, producing a tree-like structure of intermediate steps (i.e., thoughts). This method was shown to be effective for several problem types. However, the official implementation has a high barrier to usage as it requires setup overhead and incorporates task-specific problem templates which are difficult to generalize to new problem types. It also does not allow user interaction to improve or suggest new thoughts. We introduce iToT (interactive Tree-of-Thoughts), a generalized and interactive Tree of Thought prompting system. iToT allows users to explore each step of the model's problem-solving process as well as to correct and extend the model's thoughts. iToT revolves around a visual interface that facilitates simple and generic ToT usage and transparentizes the problem-solving process to users. This facilitates a better understanding of which thoughts and considerations lead to the model's final decision. Through three case studies, we demonstrate the usefulness of iToT in different human-LLM co-writing tasks.

cs.HC↗

arXiv2017

Spike sorting using non-volatile metal-oxide memristors

Electrophysiological techniques have improved substantially over the past years to the point that neuroprosthetics applications are becoming viable. This evolution has been fuelled by the advancement of implantable microelectrode technologies that have followed their own version of Moore's scaling law. Similarly to electronics, however, excessive data-rates and strained power budgets require the development of more efficient computation paradigms for handling neural data in-situ, in particular the computationally heavy task of events classification. Here, we demonstrate how the intrinsic analogue programmability of memristive devices can be exploited to perform spike-sorting. We then show how combining memristors with standard logic enables efficient in-silico template matching. Leveraging the physical properties of nanoscale memristors allows us to implement ultra-compact analogue circuits for neural signal processing at the power cost of digital.

cs.ET↗

arXiv2016

Sub 100nW volatile nano-metal-oxide memristor as synaptic-like encoder of neuronal spikes

Advanced neural interfaces mediate a bio-electronic link between the nervous system and microelectronic devices, bearing great potential as innovative therapy for various diseases. Spikes from a large number of neurons are recorded leading to creation of big data that require on-line processing under most stringent conditions, such as minimal power dissipation and on-chip space occupancy. Here, we present a new concept where the inherent volatile properties of a nano-scale memristive device are used to detect and compress information on neural spikes as recorded by a multi-electrode array. Simultaneously, and similarly to a biological synapse, information on spike amplitude and frequency is transduced in metastable resistive state transitions of the device, which is inherently capable of self-resetting and of continuous encoding of spiking activity. Furthermore, operating the memristor in a very high resistive state range reduces its average in-operando power dissipation to less than 100 nW, demonstrating the potential to build highly scalable, yet energy-efficient on-node processors for advanced neural interfaces.

cs.ET↗

arXiv2015

Memristive integrative sensors for neuronal activity

The advent of advanced neuronal interfaces offers great promise for linking brain functions to electronics. A major bottleneck in achieving this is real-time processing of big data that imposes excessive requirements on bandwidth, energy and computation capacity; limiting the overall number of bio-electronic links. Here, we present a novel monitoring system concept that exploits the intrinsic properties of memristors for processing neural information in real time. We demonstrate that the inherent voltage thresholds of solid-state TiOx memristors can be useful for discriminating significant neural activity, i.e. spiking events, from noise. When compared with a multi-dimensional, principal component feature space threshold detector, our system is capable of recording the majority of significant events, without resorting to computationally heavy off-line processing. We also show a memristive integrating sensing array that discriminates neuronal activity recorded in-vitro. We prove that information on spiking event amplitude is simultaneously transduced and stored as non-volatile resistive state transitions, allowing for more efficient data compression, demonstrating the memristors' potential for building scalable, yet energy efficient on-node processors for big data.

cs.ET↗

arXiv2015

Spatially resolved TiOx phases in RRAM conductive nanofilaments using soft X-ray spectromicroscopy

Reduction in metal-oxide thin films has been suggested as the key mechanism responsible for forming conductive nanofilaments within solid-state memory devices, enabling their resistive switching capacity. The quantitative spatial identification of such filaments is a daunting task, particularly for metal-oxides capable of exhibiting multiple phases as in the case of TiOx. Here, we spatially resolve and chemically characterize distinct TiOx phases in localized regions of a TiOx-based memristive device by combining full-field transmission X-ray microscopy with soft X-ray spectroscopic analysis that is performed on lamella samples. We particularly show that electrically pre-switched devices in low-resistive states comprise reduced disordered phases with O/Ti ratios close to Ti2O3 stoichiometry that aggregate in a ~ 100 nm filamentary region electrically conducting the top and bottom electrodes of the devices. We have also identified crystalline rutile and orthorhombic-like TiO2 phases in the region adjacent to the filament, suggesting that the temperature increases locally up to 1000 K, validating the role of Joule heating in resistive switching. Contrary to previous studies, our approach enables to simultaneously investigate morphological and chemical changes in a quantitative manner without incurring difficulties imposed by interpretation of electron diffraction patterns acquired via conventional electron microscopy techniques.

cond-mat.mtrl-sci↗

arXiv2012

BIGP- a new single protocol that can work as an igp (interior gateway protocol) as well as egp (exterior gateway protocol)

EGP and IGP are the key components of the present internet infrastructure. Routers in a domain forward IP packet within and between domains. Each domain uses an intra-domain routing protocol known as Interior Gateway Protocol (IGP) like IS-IS, OSPF, RIP etc to populate the routing tables of its routers. Routing information must also be exchanged between domains to ensure that a host in one domain can reach another host in remote domain. This role is performed by inter-domain routing protocol called Exterior Gateway Protocol (EGP). Basically EGP used these days is Border Gateway Protocol (BGP). Basic difference between the both is that BGP has smaller convergence as compared to the IGP's. And IGP's on the other hand have lesser scalability as compared to the BGP. So in this paper a proposal to create a new protocol is given which can act as an IGP when we consider inter-domain transfer of traffic and acts as BGP when we consider intra-domain transfer of traffic.

cs.NI↗