arXiv subjects

Aditi Jain

Publications and source records attributed to Aditi Jain.

5 recordsLinked to original sources

arXiv2026

From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents

Memory is a core component of AI agents, enabling them to accumulate knowledge across interactions and improve performance. However, persistent memory introduces the risk of memory poisoning, where a single adversarial memory write can exert long-term influence over agent behavior. We present a systematic study of memory poisoning in LLM-based agents. We identify four memory write channels and nine structural vulnerabilities in model capabilities, system prompt design, and agent system architecture that make these channels exploitable. Based on these vulnerabilities, we develop a taxonomy of six classes of memory poisoning attacks. Furthermore, we design MPBench -- a benchmark for evaluating memory poisoning attacks, and show that agents designed to write and retrieve memory more aggressively are more exploitable. We also show that existing prompt injection defenses fail to cover memory poisoning attacks. Our findings provide a foundation for understanding and mitigating memory poisoning attacks against AI agents.

cs.CR↗

arXiv2025

Should AI Mimic People? Understanding AI-Supported Writing Technology Among Black Users

AI-supported writing technologies (AISWT) that provide grammatical suggestions, autocomplete sentences, or generate and rewrite text are now a regular feature integrated into many people's workflows. However, little is known about how people perceive the suggestions these tools provide. In this paper, we investigate how Black American users perceive AISWT, motivated by prior findings in natural language processing that highlight how the underlying large language models can contain racial biases. Using interviews and observational user studies with 13 Black American users of AISWT, we found a strong tradeoff between the perceived benefits of using AISWT to enhance their writing style and feeling like "it wasn't built for us". Specifically, participants reported AISWT's failure to recognize commonly used names and expressions in African American Vernacular English, experiencing its corrections as hurtful and alienating and fearing it might further minoritize their culture. We end with a reflection on the tension between AISWT that fail to include Black American culture and language, and AISWT that attempt to mimic it, with attention to accuracy, authenticity, and the production of social difference.

cs.HC↗

arXiv2024

Investigating the Quality of DermaMNIST and Fitzpatrick17k Dermatological Image Datasets

The remarkable progress of deep learning in dermatological tasks has brought us closer to achieving diagnostic accuracies comparable to those of human experts. However, while large datasets play a crucial role in the development of reliable deep neural network models, the quality of data therein and their correct usage are of paramount importance. Several factors can impact data quality, such as the presence of duplicates, data leakage across train-test partitions, mislabeled images, and the absence of a well-defined test partition. In this paper, we conduct meticulous analyses of three popular dermatological image datasets: DermaMNIST, its source HAM10000, and Fitzpatrick17k, uncovering these data quality issues, measure the effects of these problems on the benchmark results, and propose corrections to the datasets. Besides ensuring the reproducibility of our analysis, by making our analysis pipeline and the accompanying code publicly available, we aim to encourage similar explorations and to facilitate the identification and addressing of potential data quality issues in other large datasets.

cs.CV↗

arXiv2019

Hipsters and the Cool: A Game Theoretic Analysis of Social Identity, Trends and Fads

Cultural trends and popularity cycles can be observed all around us, yet our theories of social influence and identity expression do not explain what perpetuates these complex, often unpredictable social dynamics. We propose a theory of social identity expression based on the opposing, but not mutually exclusive, motives to conform and to be unique among one's neighbors in a social network. We then model the social dynamics that arise from these motives. We find that the dynamics typically enter random walks or stochastic limit cycles rather than converging to a static equilibrium. We also prove that without social network structure or, alternatively, without the uniqueness motive, reasonable adaptive dynamics would necessarily converge to equilibrium. Thus, we show that nuanced psychological assumptions (recognizing preferences for uniqueness along with conformity) and realistic social network structure are both necessary for explaining how complex, unpredictable cultural trends emerge.

econ.TH↗

arXiv2019

Change Detection with Sparse Signals using Quantum Designs

We consider the change detection problem where the pre-change observation vectors are purely noise and the post-change observation vectors are noise-corrupted compressive measurements of sparse signals with a common support, measured using a sensing matrix. In general, post-change distribution of the observations depends on parameters such as the support and variances of the sparse signal. When these parameters are unknown, we propose two approaches. In the first approach, we approximate the post-change pdf based on the known parameters such as mutual coherence of the sensing matrix and bounds on the signal variances. In the second approach, we parameterize the post-change pdf with an unknown parameter and try to adaptively estimate this parameter using a stochastic gradient descent method. In both these approaches, we employ CUSUM algorithm with various decision statistics such as the energy of the observations, correlation values with columns of the sensing matrix and the maximum value of such correlations. We study the performance of these approaches and offer insights on the relevance of different decision statistics in different SNR regimes. We also address the problem of designing sensing matrices with small coherence by using designs from quantum information theory. One such design, called SIC POVM, also has an additional structure which allows exact computation of the post-change pdfs of some decision statistics even when the support set of the sparse signal is unknown. We apply our detection algorithms with SIC POVM based sequences to a massive random access problem and show their superior performance over conventional Gold codes.

eess.SP↗