arXiv subjects

Rachel Xu

Publications and source records attributed to Rachel Xu.

4 recordsLinked to original sources

arXiv2026

AI usage patterns are shaped by perceived gains in human agency

As conversational AI systems become more deeply integrated into daily life, the implications for human agency are increasingly urgent to understand. AI's potential to amplify capability sits alongside risks of individual and collective disempowerment, yet empirical, ecologically-valid evidence about cumulative usage is scarce. We analyze deep ethnographic data from a study of daily AI chatbot users (n = 51) in the United States, Germany, and Singapore to illuminate conversational AI usage in situated context as a sociotechnical practice. We show that people consistently link sustained AI usage to perceived gains in individual agency. Crucially, these perceived gains often outweigh concerns about accuracy, reliability, and consistency to shape usage patterns. Our findings challenge prevailing assumptions about how and why humans use AI systems over time, suggesting that traditional trust-based models are not sufficient for explaining human behavior with conversational AI. Finally, we expose a critical tension: immediate psychological boosts to perceived agency may not necessarily translate into material effects, structural empowerment, or long-term capacity. Our results help establish a new foundation for novel behavioral frameworks, measurement tools, and AI benchmarks to ensure conversational AI strengthens human agency in substantial, sustained ways.

cs.CY↗

arXiv2024

Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data

Generative, multimodal artificial intelligence (GenAI) offers transformative potential across industries, but its misuse poses significant risks. Prior research has shed light on the potential of advanced AI systems to be exploited for malicious purposes. However, we still lack a concrete understanding of how GenAI models are specifically exploited or abused in practice, including the tactics employed to inflict harm. In this paper, we present a taxonomy of GenAI misuse tactics, informed by existing academic literature and a qualitative analysis of approximately 200 observed incidents of misuse reported between January 2023 and March 2024. Through this analysis, we illuminate key and novel patterns in misuse during this time period, including potential motivations, strategies, and how attackers leverage and abuse system capabilities across modalities (e.g. image, text, audio, video) in the wild.

cs.AI↗

arXiv2024

New contexts, old heuristics: How young people in India and the US trust online content in the age of generative AI

We conducted in-person ethnography in India and the US to investigate how young people (18-24) trusted online content, just as generative AI (genAI) became mainstream. We found that when online, how participants determined what content to trust was shaped by emotional states, which we term "information modes." Our participants reflexively shifted between modes to maintain "emotional equilibrium," and eschewed engaging literacy skills in the more passive modes in which they spent the most time. We found participants imported trust heuristics from established online contexts into emerging ones (i.e., genAI). This led them to use ill-fitting trust heuristics, and exposed them to the risk of trusting false and misleading information. While many had reservations about AI, prioritizing efficiency, they used genAI and habitual heuristics to quickly achieve goals at the expense of accuracy. We conclude that literacy interventions designed to match users' distinct information modes will be most effective.

cs.HC↗

arXiv2024

Like-minded, like-bodied: How users (18-26) trust online eating and health information

This paper investigates the relationship between social media and eating practices amongst 42 internet users aged 18-26. We conducted an ethnography in the US and India to observe how they navigated eating and health information online. We found that participants portrayed themselves online through a vocabulary we have labeled "the good life": performing holistic health by displaying a socially-ideal body. In doing so, participants unconsciously engaged in behaviors of disordered eating while actively eschewing them. They also valued personal testimonies, and readily tested tips from content creators who shared similar beliefs and bodies to them. In doing so, they discarded probabilistic thinking and opened themselves to harm. Our study found that their social media feeds did not unidirectionally influence participants - they also reflected participants' internalized views of health, in an intertwined, non-linear journey. Reducing the online spread of disordered eating practices requires addressing it within young people's social context.

cs.HC↗