arXiv subjects

Sarita Schoenebeck

Publications and source records attributed to Sarita Schoenebeck.

12 recordsLinked to original sources

arXiv2026

Position: AI/ML Deepfake Research is Misaligned with AI-Generated Non-Consensual Intimate Imagery (AIG-NCII)

AI-generated non-consensual intimate imagery (AIG-NCII) is not adequately addressed in AI/ML literature regarding AI-generated media, commonly referred to as "deepfakes". While research on deepfakes currently focuses on its epistemic harms -- or harms relating to truth and authenticity -- this is misaligned with the dominant reality of generative AI abuse involving sexualized imagery. We conduct a landscape analysis of highly-cited works to demonstrate that technical interventions addressing deepfakes almost entirely ignore AIG-NCII, limiting the research ecosystem to authenticity detection tools. In this position paper, we argue that existing interventions address viewer-centric epistemic harms, such as fraud or scams, but ignore subject-centric dignity harms, such as AIG-NCII. We illustrate that knowing an image is synthetic does not mitigate harms to subjects and may, in some cases, even exacerbate them. We conclude by offering recommendations to realign the field, including updating threat models to consider subject-centric harms and addressing AIG-NCII in AI safety research. Finally, we caution that researchers should only engage in this high-risk domain if they implement safety guardrails for both subjects and researchers and establish partnerships with domain experts in sexual violence prevention.

cs.AI↗

arXiv2025

Platforms as Crime Scene, Judge, and Jury: How Victim-Survivors of Non-Consensual Intimate Imagery Report Abuse Online

Non-consensual intimate imagery (NCII), also known as image-based sexual abuse (IBSA), is mediated through online platforms. Victim-survivors must turn to platforms to collect evidence and request content removal. Platforms act as the crime scene, judge, and jury, determining whether perpetrators face consequences and if harmful material is removed. We present a study of NCII victim-survivors' online reporting experiences, drawing on trauma-informed interviews with 13 participants. We find that platform reporting processes are hostile, opaque, and ineffective, often forcing complex harms into narrow interfaces, responding inconsistently, and failing to result in meaningful action. Leveraging institutional betrayal theory, we show how platforms' structures and practices compound harm, and, in doing so, surface concrete intervention points for redesigning reporting systems and shaping policy to better support victim-survivors

cs.HC↗

arXiv2025

Large Language Models in Qualitative Research: Uses, Tensions, and Intentions

Qualitative researchers use tools to collect, sort, and analyze their data. Should qualitative researchers use large language models (LLMs) as part of their practice? LLMs could augment qualitative research, but it is unclear if their use is appropriate, ethical, or aligned with qualitative researchers' goals and values. We interviewed twenty qualitative researchers to investigate these tensions. Many participants see LLMs as promising interlocutors with attractive use cases across the stages of research, but wrestle with their performance and appropriateness. Participants surface concerns regarding the use of LLMs while protecting participant interests, and call attention to an urgent lack of norms and tooling to guide the ethical use of LLMs in research. We document the rapid and broad adoption of LLMs across surfaces, which can interfere with intentional use vital to qualitative research. We use the tensions surfaced by our participants to outline recommendations for researchers considering using LLMs in qualitative research and design principles for LLM-assisted qualitative research tools.

cs.HC↗

arXiv2025

A Law of One's Own: The Inefficacy of the DMCA for Non-Consensual Intimate Media

Non-consensual intimate media (NCIM) presents internet-scale harm to individuals who are depicted. One of the most powerful tools for requesting its removal is the Digital Millennium Copyright Act (DMCA). However, the DMCA was designed to protect copyright holders rather than to address the problem of NCIM. Using a dataset of more than 54,000 DMCA reports and over 85 million infringing URLs spanning over a decade, this paper evaluates the efficacy of the DMCA for NCIM takedown. Results show that for non-commercial requests, while more than half of URLs are deindexed from Google Search within 48 hours, the actual removal of content from website hosts is much slower. The median infringing URL takes more than 45 days to be removed from website hosts, and only 5.39% URLs are removed within the first 48 hours. Additionally, the most frequently reported domains for non-commercial NCIM are smaller websites, not large platforms. We stress the need for new laws that ensure a shorter time to takedown that are enforceable across big and small platforms alike.

cs.CY↗

arXiv2024

Reporting Non-Consensual Intimate Media: An Audit Study of Deepfakes

Non-consensual intimate media (NCIM) inflicts significant harm. Currently, victim-survivors can use two mechanisms to report NCIM - as a non-consensual nudity violation or as copyright infringement. We conducted an audit study of takedown speed of NCIM reported to X (formerly Twitter) of both mechanisms. We uploaded 50 AI-generated nude images and reported half under X's "non-consensual nudity" reporting mechanism and half under its "copyright infringement" mechanism. The copyright condition resulted in successful image removal within 25 hours for all images (100% removal rate), while non-consensual nudity reports resulted in no image removal for over three weeks (0% removal rate). We stress the need for targeted legislation to regulate NCIM removal online. We also discuss ethical considerations for auditing NCIM on social platforms.

cs.CY↗

arXiv2024

Misfitting With AI: How Blind People Verify and Contest AI Errors

Blind people use artificial intelligence-enabled visual assistance technologies (AI VAT) to gain visual access in their everyday lives, but these technologies are embedded with errors that may be difficult to verify non-visually. Previous studies have primarily explored sighted users' understanding of AI output and created vision-dependent explainable AI (XAI) features. We extend this body of literature by conducting an in-depth qualitative study with 26 blind people to understand their verification experiences and preferences. We begin by describing errors blind people encounter, highlighting how AI VAT fails to support complex document layouts, diverse languages, and cultural artifacts. We then illuminate how blind people make sense of AI through experimenting with AI VAT, employing non-visual skills, strategically including sighted people, and cross-referencing with other devices. Participants provided detailed opportunities for designing accessible XAI, such as affordances to support contestation. Informed by disability studies framework of misfitting and fitting, we unpacked harmful assumptions with AI VAT, underscoring the importance of celebrating disabled ways of knowing. Lastly, we offer practical takeaways for Responsible AI practice to push the field of accessible XAI forward.

cs.HC↗

arXiv2024

Feminist Interaction Techniques: Deterring Non-Consensual Screenshots with Interaction Techniques

Non-consensual Intimate Media (NCIM) refers to the distribution of sexual or intimate content without consent. NCIM is common and causes significant emotional, financial, and reputational harm. We developed Hands-Off, an interaction technique for messaging applications that deters non-consensual screenshots. Hands-Off requires recipients to perform a hand gesture in the air, above the device, to unlock media -- which makes simultaneous screenshotting difficult. A lab study shows that Hands-Off gestures are easy to perform and reduce non-consensual screenshots by 67 percent. We conclude by generalizing this approach and introduce the idea of Feminist Interaction Techniques (FIT), interaction techniques that encode feminist values and speak to societal problems, and reflect on FIT's opportunities and limitations.

cs.HC↗

arXiv2024

The Sociotechnical Stack: Opportunities for Social Computing Research in Non-consensual Intimate Media

Non-consensual intimate media (NCIM) involves sharing intimate content without the depicted person's consent, including "revenge porn" and sexually explicit deepfakes. While NCIM has received attention in legal, psychological, and communication fields over the past decade, it is not sufficiently addressed in computing scholarship. This paper addresses this gap by linking NCIM harms to the specific technological components that facilitate them. We introduce the sociotechnical stack, a conceptual framework designed to map the technical stack to its corresponding social impacts. The sociotechnical stack allows us to analyze sociotechnical problems like NCIM, and points toward opportunities for computing research. We propose a research roadmap for computing and social computing communities to deter NCIM perpetration and support victim-survivors through building and rebuilding technologies.

cs.CY↗

arXiv2024

The Future of Research on Social Technologies: CCC Workshop Visioning Report

Social technologies are the systems, interfaces, features, infrastructures, and architectures that allow people to interact with each other online. These technologies dramatically shape the fabric of our everyday lives, from the information we consume to the people we interact with to the foundations of our culture and politics. While the benefits of social technologies are well documented, the harms, too, have cast a long shadow. To address widespread problems like harassment, disinformation, information access, and mental health concerns, we need to rethink the foundations of how social technologies are designed, sustained, and governed. This report is based on discussions at the Computing Community Consortium Workshop, The Future of Research on Social Technologies, that was held November 2-3, 2023 in Washington, DC. The visioning workshop came together to focus on two questions. What should we know about social technologies, and what is needed to get there? The workshop brought together over 50 information and computer scientists, social scientists, communication and journalism scholars, and policy experts. We used a discussion format, with one day of guiding topics and a second day using an unconference model where participants created discussion topics. The interdisciplinary group of attendees discussed gaps in existing scholarship and the methods, resources, access, and collective effort needed to address those gaps. We also discussed approaches for translating scholarship for various audiences including citizens, funders, educators, industry professionals, and policymakers. This report presents a synthesis of major themes during our discussions. The themes presented are not a summary of what we know already, they are an exploration of what we do not know enough about, and what we should spend more effort and investment on in the coming years.

cs.SI↗

arXiv2023

Trauma-Informed Social Media: Towards Solutions for Reducing and Healing Online Harm

Social media platforms exacerbate trauma, and many users experience various forms of trauma unique to them (e.g., doxxing and swatting). Trauma is the psychological and physical response to experiencing a deeply disturbing event. Platforms' failures to address trauma threaten users' well-being globally, especially amongst minoritized groups. Platform policies also expose moderators and designers to trauma through content they must engage with as part of their jobs (e.g., child sexual abuse). We consider how a trauma-informed approach might help address or decrease the likelihood of (re)experiencing trauma online. A trauma-informed approach to social media recognizes that everyone likely has a trauma history and that trauma is experienced at the individual, secondary, collective, and cultural levels. This paper proceeds by detailing trauma and its impacts. We then describe how the six trauma-informed principles can be applied to social media design, content moderation, and companies. We conclude by offering recommendations that balance platform responsibility and accountability with well-being and healing for all.

cs.HC↗

arXiv2023

Online Harassment in Majority Contexts: Examining Harms and Remedies across Countries

Online harassment is a global problem. This article examines perceptions of harm and preferences for remedies associated with online harassment with nearly 4000 participants in 14 countries around the world. The countries in this work reflect a range of identities and values, with a focus on those outside of North American and European contexts. Results show that perceptions of harm are higher among participants from all countries studied compared to the United States. Non-consensual sharing of sexual photos is consistently rated as harmful in all countries, while insults and rumors are perceived as more harmful in non-U.S. countries, especially harm to family reputation. Lower trust in other people and lower trust in sense of safety in one's neighborhood correlate with increased perceptions of harm of online harassment. In terms of remedies, participants in most countries prefer monetary compensation, apologies, and publicly revealing offender's identities compared to the U.S. Social media platform design and policy must consider regional values and norms, which may depart from U.S. centric-approaches.

cs.HC↗

arXiv2023

Women's Perspectives on Harm and Justice after Online Harassment

Social media platforms aspire to create online experiences where users can participate safely and equitably. However, women around the world experience widespread online harassment, including insults, stalking, aggression, threats, and non-consensual sharing of sexual photos. This article describes women's perceptions of harm associated with online harassment and preferred platform responses to that harm. We conducted a survey in 14 geographic regions around the world (N = 3,993), focusing on regions whose perspectives have been insufficiently elevated in social media governance decisions (e.g. Mongolia, Cameroon). {Results show} that, on average, women perceive greater harm associated with online harassment than men, especially for non-consensual image sharing. Women also prefer most platform responses compared to men, especially removing content and banning users; however, women are less favorable towards payment as a response. Addressing global gender-based violence online requires understanding how women experience online harms and how they wish for it to be addressed. This is especially important given that the people who build and govern technology are not typically those who are most likely to experience online harms.

cs.HC↗