arXiv subjects

Elissa M. Redmiles

Publications and source records attributed to Elissa M. Redmiles.

At least 19 recordsLinked to original sources

arXiv2026

"I Thought You Were The Uncensored Place": Norms, Rules, and Moderation in AI-Generated Sexual Content Communities

As AI-generated sexual content (AIG-SC) is increasingly produced, online communities have emerged to support creators' needs. To understand whether and how community governance attempts work to prevent abuse while supporting free expression, we interviewed 24 members and moderators of large AIG-SC online communities (10,000+ members) with stated rules against creating and sharing abusive content (e.g., AI-generated CSAM). Through in-depth interviews, we offer insight into: (1) how and why these communities form; (2) implicit community norms; (3) explicitly stated rules---and their operationalization via content moderation; and (4) tensions between community values and moderation that leave space for abusive behavior. Our findings reveal a complex picture: while many creators and communities have personal boundaries against abuse, advice and resources for creating any form of AI-generated sexual content are accessible to users regardless of their intentions. Further complicating community moderation are norms that center anti-censorship and non-judgment, which leave moderators to justify their actions using the limits of the law and terms of service. We end by reflecting on the ways in which technical, community, and legal governance may most effectively mitigate the production of abusive content.

cs.CR↗

arXiv2026

"Unlimited Realm of Exploration and Experimentation": Methods and Motivations of AI-Generated Sexual Content Creators

AI-generated media is radically changing the way content is both consumed and produced on the internet, and in no place is this potentially more visible than in sexual content. AI-generated sexual content (AIG-SC) is increasingly enabled by an ecosystem of individual AI developers, specialized third-party applications, and foundation model providers. AIG-SC raises a number of concerns from older debates about the line between pornography and obscenity to newer debates about fair use and labor displacement (in this case, of sex workers), and has spurred new regulations to curb the spread of non-consensual intimate imagery (NCII) created using the same technology used to create AIG-SC. However, despite the growing prevalence of AIG-SC, little is known about its creators, their motivations, and what types of content they produce. To inform effective governance in this space, we conducted an in-depth study to understand what AIG-SC creators make, along with how and why they make it. Interviews with 28 AIG-SC creators, ranging from hobbyists to entrepreneurs to those who moderate communities of hundreds of thousands of other creators, revealed a wide spectrum of motivations, including sexual exploration, creative expression, technical experimentation, and in a handful of cases, the creation of NCII.

cs.CY↗

arXiv2026

Evaluating Concept Filtering Defenses against Child Sexual Abuse Material Generation by Text-to-Image Models

We evaluate the effectiveness of filtering child images from training datasets of text-to-image models to prevent model misuse to create child sexual abuse material (CSAM). First, we capture the complexity of preventing CSAM generation using a game-based security definition. Second, we show that current detection methods cannot remove all children from a dataset. Third, using an ethical proxy for CSAM (a child wearing glasses), we show that even when only a small percentage of child images are left in the training dataset after filtering, there exist prompting strategies that generate a child wearing glasses using only a few more queries than when the model is trained on the unfiltered data. Fine-tuning the filtered model on child images further reduces the additional query overhead. We also show that re-introducing a concept is possible via fine-tuning even if filtering is perfect. Our results show that current child filtering methods offer limited protection to closed-weight models and no protection to open-weight models, while reducing the generality of the model by hindering the generation of child-related concepts or changing their representation. We conclude by outlining challenges in conducting evaluations that establish robust evidence on the impact of concept filtering defenses for CSAM.

cs.CR↗

arXiv2025

"Having Confidence in My Confidence Intervals": How Data Users Engage with Privacy-Protected Wikipedia Data

In response to calls for open data and growing privacy threats, organizations are increasingly adopting privacy-preserving techniques such as differential privacy (DP) that inject statistical noise when generating published datasets. These techniques are designed to protect privacy of data subjects while enabling useful analyses, but their reception by data users is under-explored. We developed documentation that presents the noise characteristics of two Wikipedia pageview datasets: one using rounding (heuristic privacy) and another using DP (formal privacy). After incorporating expert feedback (n=5), we used these documents to conduct a task-based contextual inquiry (n=15) exploring how data users--largely unfamiliar with these methods--perceive, interact with, and interpret privacy-preserving noise during data analysis. Participants readily used simple uncertainty metrics from the documentation, but struggled when asked to compute confidence intervals across multiple noisy estimates. They were better able to devise simulation-based approaches for computing uncertainty with DP data compared to rounded data. Surprisingly, several participants incorrectly believed DP's stronger utility implied weaker privacy protections. Based on our findings, we offer design recommendations for documentation and tools to better support data users working with privacy-noised data.

cs.HC↗

arXiv2025

The Role of Privacy Guarantees in Voluntary Donation of Private Health Data for Altruistic Goals

The voluntary donation of private health information for altruistic purposes, such as supporting research advancements, is a common practice. However, concerns about data misuse and leakage may deter people from donating their information. Privacy Enhancement Technologies (PETs) aim to alleviate these concerns and in turn allow for safe and private data sharing. This study conducts a vignette survey (N=494) with participants recruited from Prolific to examine the willingness of US-based people to donate medical data for developing new treatments under four general guarantees offered across PETs: data expiration, anonymization, purpose restriction, and access control. The study explores two mechanisms for verifying these guarantees: self-auditing and expert auditing, and controls for the impact of confounds including demographics and two types of data collectors: for-profit and non-profit institutions. Our findings reveal that respondents hold such high expectations of privacy from non-profit entities a priori that explicitly outlining privacy protections has little impact on their overall perceptions. In contrast, offering privacy guarantees elevates respondents' expectations of privacy for for-profit entities, bringing them nearly in line with those for non-profit organizations. Further, while the technical community has suggested audits as a mechanism to increase trust in PET guarantees, we observe limited effect from transparency about such audits. We emphasize the risks associated with these findings and underscore the critical need for future interdisciplinary research efforts to bridge the gap between the technical community's and end-users' perceptions regarding the effectiveness of auditing PETs.

cs.CR↗

arXiv2024

Analyzing the AI Nudification Application Ecosystem

Given a source image of a clothed person (an image subject), AI-based nudification applications can produce nude (undressed) images of that person. Moreover, not only do such applications exist, but there is ample evidence of the use of such applications in the real world and without the consent of an image subject. Still, despite the growing awareness of the existence of such applications and their potential to violate the rights of image subjects and cause downstream harms, there has been no systematic study of the nudification application ecosystem across multiple applications. We conduct such a study here, focusing on 20 popular and easy-to-find nudification websites. We study the positioning of these web applications (e.g., finding that most sites explicitly target the nudification of women, not all people), the features that they advertise (e.g., ranging from undressing-in-place to the rendering of image subjects in sexual positions, as well as differing user-privacy options), and their underlying monetization infrastructure (e.g., credit cards and cryptocurrencies). We believe this work will empower future, data-informed conversations -- within the scientific, technical, and policy communities -- on how to better protect individuals' rights and minimize harm in the face of modern (and future) AI-based nudification applications. Content warning: This paper includes descriptions of web applications that can be used to create synthetic non-consensual explicit AI-created imagery (SNEACI). This paper also includes an artistic rendering of a user interface for such an application.

cs.HC↗

arXiv2024

"Violation of my body:" Perceptions of AI-generated non-consensual (intimate) imagery

AI technology has enabled the creation of deepfakes: hyper-realistic synthetic media. We surveyed 315 individuals in the U.S. on their views regarding the hypothetical non-consensual creation of deepfakes depicting them, including deepfakes portraying sexual acts. Respondents indicated strong opposition to creating and, even more so, sharing non-consensually created synthetic content, especially if that content depicts a sexual act. However, seeking out such content appeared more acceptable to some respondents. Attitudes around acceptability varied further based on the hypothetical creator's relationship to the participant, the respondent's gender and their attitudes towards sexual consent. This study provides initial insight into public perspectives of a growing threat and highlights the need for further research to inform social norms as well as ongoing policy conversations and technical developments in generative AI.

cs.CY↗

arXiv2024

"Did They F***ing Consent to That?": Safer Digital Intimacy via Proactive Protection Against Image-Based Sexual Abuse

As many as 8 in 10 adults share intimate content such as nude or lewd images. Sharing such content has significant benefits for relationship intimacy and body image, and can offer employment. However, stigmatizing attitudes and a lack of technological mitigations put those sharing such content at risk of sexual violence. An estimated 1 in 3 people have been subjected to image-based sexual abuse (IBSA), a spectrum of violence that includes the nonconsensual distribution or threat of distribution of consensually-created intimate content (also called NDII). In this work, we conducted a rigorous empirical interview study of 52 European creators of intimate content to examine the threats they face and how they defend against them, situated in the context of their different use cases for intimate content sharing and their choice of technologies for storing and sharing such content. Synthesizing our results with the limited body of prior work on technological prevention of NDII, we offer concrete next steps for both platforms and security & privacy researchers to work toward safer intimate content sharing through proactive protection. Content Warning: This work discusses sexual violence, specifically, the harms of image-based sexual abuse (particularly in Sections 2 and 6).

cs.CR↗

arXiv2024

SoK (or SoLK?): On the Quantitative Study of Sociodemographic Factors and Computer Security Behaviors

Researchers are increasingly exploring how gender, culture, and other sociodemographic factors correlate with user computer security and privacy behaviors. To more holistically understand relationships between these factors and behaviors, we make two contributions. First, we broadly survey existing scholarship on sociodemographics and secure behavior (151 papers) before conducting a focused literature review of 47 papers to synthesize what is currently known and identify open questions for future research. Second, by incorporating contemporary social and critical theories, we establish guidelines for future studies of sociodemographic factors and security behaviors that address how to overcome common pitfalls. We present a case study to demonstrate our guidelines in action, at-scale, that conduct a measurement study of the relationships between sociodemographics and de-identified, aggregated log data of security and privacy behaviors among 16,829 users on Facebook across 16 countries. Through these contributions, we position our work as a systemization of a lack of knowledge (SoLK). Overall, we find contradictory results and vast unknowns about how identity shapes security behavior. Through our guidelines and discussion, we chart new directions to more deeply examine how and why sociodemographic factors affect security behaviors.

cs.CR↗

arXiv2024

Analyzing User Engagement with TikTok's Short Format Video Recommendations using Data Donations

Short-format videos have exploded on platforms like TikTok, Instagram, and YouTube. Despite this, the research community lacks large-scale empirical studies into how people engage with short-format videos and the role of recommendation systems that offer endless streams of such content. In this work, we analyze user engagement on TikTok using data we collect via a data donation system that allows TikTok users to donate their data. We recruited 347 TikTok users and collected 9.2M TikTok video recommendations they received. By analyzing user engagement, we find that the average daily usage time increases over the users' lifetime while the user attention remains stable at around 45%. We also find that users like more videos uploaded by people they follow than those recommended by people they do not follow. Our study offers valuable insights into how users engage with short-format videos on TikTok and lessons learned from designing a data donation system.

cs.SI↗

arXiv2024

Safer Digital Intimacy For Sex Workers And Beyond: A Technical Research Agenda

Many people engage in digital intimacy: sex workers, their clients, and people who create and share intimate content recreationally. With this intimacy comes significant security and privacy risk, exacerbated by stigma. In this article, we present a commercial digital intimacy threat model and 10 research directions for safer digital intimacy

cs.CY↗

arXiv2024

Likes and Fragments: Examining Perceptions of Time Spent on TikTok

Researchers use information about the amount of time people spend on digital media for numerous purposes. While social media platforms commonly do not allow external access to measure the use time directly, a usual alternative method is to use participants' self-estimation. However, doubts were raised about the self-estimation's accuracy, posing questions regarding the cognitive factors that underline people's perceptions of the time they spend on social media. In this work, we build on prior studies and explore a novel social media platform in the context of use time: TikTok. We conduct platform-independent measurements of people's self-reported and server-logged TikTok usage (n=255) to understand how users' demographics and platform engagement influence their perceptions of the time they spend on the platform and their estimation accuracy. Our work adds to the body of work seeking to understand time estimations in different digital contexts and identifies new influential engagement factors.

cs.CY↗

arXiv2023

Problematic Advertising and its Disparate Exposure on Facebook

Targeted advertising remains an important part of the free web browsing experience, where advertisers' targeting and personalization algorithms together find the most relevant audience for millions of ads every day. However, given the wide use of advertising, this also enables using ads as a vehicle for problematic content, such as scams or clickbait. Recent work that explores people's sentiments toward online ads, and the impacts of these ads on people's online experiences, has found evidence that online ads can indeed be problematic. Further, there is the potential for personalization to aid the delivery of such ads, even when the advertiser targets with low specificity. In this paper, we study Facebook -- one of the internet's largest ad platforms -- and investigate key gaps in our understanding of problematic online advertising: (a) What categories of ads do people find problematic? (b) Are there disparities in the distribution of problematic ads to viewers? and if so, (c) Who is responsible -- advertisers or advertising platforms? To answer these questions, we empirically measure a diverse sample of user experiences with Facebook ads via a 3-month longitudinal panel. We categorize over 32,000 ads collected from this panel ($n=132$); and survey participants' sentiments toward their own ads to identify four categories of problematic ads. Statistically modeling the distribution of problematic ads across demographics, we find that older people and minority groups are especially likely to be shown such ads. Further, given that 22% of problematic ads had no specific targeting from advertisers, we infer that ad delivery algorithms (advertising platforms themselves) played a significant role in the biased distribution of these ads.

cs.CY↗

arXiv2022

Examining the Landscape of Digital Safety and Privacy Assistance for Black Communities

Recent events have placed a renewed focus on the issue of racial justice in the United States and other countries. One dimension of this issue that has received considerable attention is the security and privacy threats and vulnerabilities faced by the communities of color. Our study focuses on community-level advocates who organize workshops, clinics, and other initiatives that inform Black communities about existing digital safety and privacy threats and ways to mitigate against them. Additionally, we aim to understand the online security and privacy needs and attitudes of participants who partake in these initiatives. We hope that by understanding how advocates work in different contexts and what teaching methods are effective, we can help other digital safety experts and activists become advocates within their communities.

cs.CY↗

arXiv2022

Dimensions of Diversity in Human Perceptions of Algorithmic Fairness

A growing number of oversight boards and regulatory bodies seek to monitor and govern algorithms that make decisions about people's lives. Prior work has explored how people believe algorithmic decisions should be made, but there is little understanding of how individual factors like sociodemographics or direct experience with a decision-making scenario may affect their ethical views. We take a step toward filling this gap by exploring how people's perceptions of one aspect of procedural algorithmic fairness (the fairness of using particular features in an algorithmic decision) relate to their (i) demographics (age, education, gender, race, political views) and (ii) personal experiences with the algorithmic decision-making scenario. We find that political views and personal experience with the algorithmic decision context significantly influence perceptions about the fairness of using different features for bail decision-making. Drawing on our results, we discuss the implications for stakeholder engagement and algorithmic oversight including the need to consider multiple dimensions of diversity in composing oversight and regulatory bodies.

cs.CY↗

arXiv2022

Risk, Resilience and Reward: Impacts of Shifting to Digital Sex Work

Workers from a variety of industries rapidly shifted to remote work at the onset of the COVID-19 pandemic. While existing work has examined the impact of this shift on office workers, little work has examined how shifting from in-person to online work affected workers in the informal labor sector. We examine the impact of shifting from in-person to online-only work on a particularly marginalized group of workers: sex workers. Through 34 qualitative interviews with sex workers from seven countries in the Global North, we examine how a shift to online-only sex work impacted: (1) working conditions, (2) risks and protective behaviors, and (3) labor rewards. We find that online work offers benefits to sex workers' financial and physical well-being. However, online-only work introduces new and greater digital and mental health risks as a result of the need to be publicly visible on more platforms and to share more explicit content. From our findings we propose design and platform governance suggestions for digital sex workers and for informal workers more broadly, particularly those who create and sell digital content.

cs.CY↗

arXiv2022

Ctrl-Shift: How Privacy Sentiment Changed from 2019 to 2021

People's privacy sentiments influence changes in legislation as well as technology design and use. While single-point-in-time investigations of privacy sentiment offer useful insight, study of people's privacy sentiments over time is also necessary to better understand and anticipate evolving privacy attitudes. In this work, we use repeated cross-sectional surveys (n=6,676) to model the sentiments of people in the U.S. toward collection and use of data for government- and health-related purposes from 2019-2021. After the onset of COVID-19, we observe significant decreases in respondent acceptance of government data use and significant increases in acceptance of health-related data uses. While differences in privacy attitudes between sociodemographic groups largely decreased over this time period, following the 2020 U.S. national elections, we observe some of the first evidence that privacy sentiments may change based on the alignment between a user's politics and the political party in power. Our results offer insight into how privacy attitudes may have been impacted by recent events and allow us to identify potential predictors of changes in privacy attitudes during times of geopolitical or national change.

cs.CY↗

arXiv2022

Ethics and Efficacy of Unsolicited Anti-Trafficking SMS Outreach

The sex industry exists on a continuum based on the degree of work autonomy present in labor conditions: a high degree exists on one side of the continuum where independent sex workers have a great deal of agency, while much less autonomy exists on the other side, where sex is traded under conditions of human trafficking. Organizations across North America perform outreach to sex industry workers to offer assistance in the form of services (e.g., healthcare, financial assistance, housing), prayer, and intervention. Increasingly, technology is used to look for trafficking victims or facilitate the provision of assistance or services, for example through scraping and parsing sex industry workers' advertisements into a database of contact information that can be used by outreach organizations. However, little is known about the efficacy of anti-trafficking outreach technology, nor the potential risks of using it to identify and contact the highly stigmatized and marginalized population of those working in the sex industry. In this work, we investigate the use, context, benefits, and harms of an anti-trafficking technology platform via qualitative interviews with multiple stakeholders: the technology developers (n=6), organizations that use the technology (n=17), and sex industry workers who have been contacted or wish to be contacted (n=24). Our findings illustrate misalignment between developers, users of the platform, and sex industry workers they are attempting to assist. In their current state, anti-trafficking outreach tools such as the one we investigate are ineffective and, at best, serve as a mechanism for spam and, at worst, scale and exacerbate harm against the population they aim to serve. We conclude with a discussion of best practices for technology-facilitated outreach efforts to minimize risk or harm to sex industry workers while efficiently providing needed services.

cs.CY↗