Searcharxiv⌕ Search

arXiv subjects

Aaron Shaw

Publications and source records attributed to Aaron Shaw.

17 recordsLinked to original sources
arXiv2026

Critical Acclaim Orientation in Large Language Models: Evidence from Film Preference Elicitation

Large language models (LLMs) are trained on corpora that contain expressions of human judgment about films, books, music, and more. Yet whether LLMs systematically reproduce evaluative hierarchies remains unclear. Prior research on cultural bias in LLMs suggests competing expectations: models may mirror the popularity signals of internet texts, or may reproduce forms of prestige embedded in critical discourse. We probe this question through a study of film evaluations with eight models from four families (Anthropic, OpenAI, Alibaba, and Mistral), using a 200-film benchmark partitioned into critically acclaimed, commercially successful, and dual-legitimacy (critical acclaim + commercial success) films. Across 20,000 pairwise forced-choice comparisons per model analyzed with Bradley--Terry estimation, we observe a consistent critical acclaim orientation with all models: critically acclaimed yet commercially obscure films are selected over commercially successful yet critically unrecognized ones. This pattern grows with model scale within each family. In addition, nested OLS regression analyses show that evaluative orientation, public visibility, and popular reception distinctly help explain preferences. Adjusting for public visibility reverses the models' preference for dual-legitimacy films over critical acclaim-only films, while additionally accounting for popular reception attenuates much of the disadvantage of films with commercial success only. Finally, evaluative and recommendation-oriented prompt framings produce divergent rankings, suggesting that critical acclaim orientation may manifest indirectly in real-world LLM deployments.

cs.AI↗
arXiv2026

Linguistic Similarity Within Centralized FLOSS Development

When free/libre and open source software (FLOSS) stewards centralize project development, they potentially undermine project sustainability and impact how contributors talk to each other. To study the relationship between steward-centralized development and contributor discussion, we compared the development of three Wikimedia platform features that the Wikimedia Foundation (WMF) built in MediaWiki. In a mixed-methods multi-case comparison, we used repository mining, linguistic style features, and principal component analysis to track MediaWiki feature development and issue discussions. Contrary to both our intuition and prior work, there were no identifiable differences in the linguistic style of WMF-affiliates and external contributors, even when feature development was guided by WMF contributions. From these results, we offer two provocations to the study of collaborative FLOSS development: (1) stewards dominate development according to their own use of specific project functionality; (2) centralized project development does not entail hierarchical language within project discussions.

cs.SE↗
arXiv2026

This human study did not involve human subjects: Validating LLM simulations as behavioral evidence

A growing literature uses large language models (LLMs) as synthetic participants to generate cost-effective and nearly instantaneous responses in social science experiments. However, there is limited guidance on when such simulations support valid inference about human behavior. We contrast two strategies for obtaining valid estimates of causal effects and clarify the assumptions under which each is suitable for exploratory versus confirmatory research. Heuristic approaches seek to establish that simulated and observed human behavior are interchangeable through prompt engineering, model fine-tuning, and other repair strategies designed to reduce LLM-induced inaccuracies. While useful for many exploratory tasks, heuristic approaches lack the formal statistical guarantees typically required for confirmatory research. In contrast, statistical calibration combines auxiliary human data with statistical adjustments to account for discrepancies between observed and simulated responses. Under explicit assumptions, statistical calibration preserves validity and provides more precise estimates of causal effects at lower cost than experiments that rely solely on human participants. Yet the potential of both approaches depends on how well LLMs approximate the relevant populations. We consider what opportunities are overlooked when researchers focus myopically on substituting LLMs for human participants in a study.

cs.AI↗
arXiv2026

SMART: A Social Movement Analysis & Reasoning Tool with Case Studies on #MeToo and #BlackLivesMatter

Social movements supporting the UN's Sustainable Development Goals (SDGs) play a vital role in improving human lives. If journalists were aware of the relationship between social movements and external events, they could provide more precise, time-sensitive reporting about movement issues and SDGs. Our SMART system achieves this goal by collecting data from multiple sources, extracting emotions on various themes, and then using a transformer-based forecasting engine (DEEP) to predict quantity and intensity of emotions in future posts. This paper demonstrates SMART's Retrospective capabilities required by journalists via case studies analyzing social media discussions of the #MeToo and #BlackLivesMatter before and after the 2024 U.S. election. We create a novel 1-year dataset which we will release upon publication. It contains over 2.7M Reddit posts and over 1M news articles. We show that SMART enables early detection of discourse shifts around key political events, providing journalists with actionable insights to inform editorial planning. SMART was developed through multiple interactions with a panel of over 20 journalists from a variety of news organizations over a 2-year period, including an author of this paper.

cs.SI↗
arXiv2025

DEEP: A Discourse Evolution Engine for Predictions about Social Movements

Numerous social movements (SMs) around the world help support the UN's Sustainable Development Goals (SDGs). Understanding how key events shape SMs is key to the achievement of the SDGs. We have developed SMART (Social Media Analysis & Reasoning Tool) to track social movements related to the SDGs. SMART was designed by a multidisciplinary team of AI researchers, journalists, communications scholars and legal experts. This paper describes SMART's transformer-based multivariate time series Discourse Evolution Engine for Predictions about Social Movements (DEEP) to predict the volume of future articles/posts and the emotions expressed. DEEP outputs probabilistic forecasts with uncertainty estimates, providing critical support for editorial planning and strategic decision-making. We evaluate DEEP with a case study of the #MeToo movement by creating a novel longitudinal dataset (433K Reddit posts and 121K news articles) from September 2024 to June 2025 that will be publicly released for research purposes upon publication of this paper.

cs.SI↗
arXiv2025

Recommended Practices for NPOV Research on Wikipedia

Writing Wikipedia with a neutral point of view is one of the five pillars of Wikipedia. Although the topic is core to Wikipedia, it is relatively understudied considering hundreds of research studies are published annually about the project. We hypothesize that part of the reason for the low research activity on the topic is that Wikipedia's definition of neutrality and its importance are not well understood within the research community. Neutrality is also an inherently challenging and contested concept. Our aim with this paper is to accelerate high quality research in this space that can help Wikipedia communities continue to improve their work in writing the encyclopedia. We do this by helping researchers to learn what Neutral Point of View means in the context of Wikipedia, identifying some common challenges with studying NPOV and how to navigate them, and offering guidance on how researchers can communicate the results of their work for increased impact on the ground for the benefit of Wikipedia.

cs.CY↗
arXiv2025

Governing Together: Toward Infrastructure for Community-Run Social Media

Decentralizing the governance of social computing systems to communities promises to empower them to make independent decisions, with nuance and in accordance with their values. Yet, communities do not govern in isolation. Many problems communities face are common, or move across their boundaries. We therefore propose designing for "inter-community governance:" mechanisms that support relationships and interactions between communities to coordinate on governance issues. Drawing from workshops with 24 individuals on decentralized, community-run social media, we present six challenges in designing for inter-community governance surfaced through ideas proposed in workshops. Together, these ideas come together as an ecosystem of resources, infrastructures, and tools that highlight three key principles for designing for inter-community governance: modularity, forkability, and polycentricity. We end with a discussion of how the ideas proposed in workshops might be implemented in future work aiming to support community governance in social computing systems broadly.

cs.HC↗
arXiv2025

The Relational Origins of Rules in Online Communities

Where do rules come from in online communities? This study investigates how and why online communities adopt and change their rules. We conducted a grounded theory-based analysis of 40 in-depth interviews with community leaders from subreddits, Fandom wikis, and Fediverse servers, and identified seven processes involved in the adoption of online community rules. Our findings reveal that, beyond operational reasons like regulating behavior and solving problems, rules are also adopted and changed for relational reasons, such as signaling or reinforcing community legitimacy and identity to other communities. While rule change was often prompted by challenges during community growth or decline, change also depended on volunteer leaders' work capacity, the presence of member feedback mechanisms, and relational dynamics between leaders and members. Our findings extend prior theories from social computing and organizational research, illustrating how institutionalist and ecological explanations of the relational origins of rules complement operational accounts. Finally, we build on these explanations to offer a set of design propositions that reflect the relational aspects of rules and rulemaking across communities' lifecycles.

cs.HC↗
arXiv2025

The Introduction of README and CONTRIBUTING Files in Open Source Software Development

README and CONTRIBUTING files can serve as the first point of contact for potential contributors to free/libre and open source software (FLOSS) projects. Prominent open source software organizations such as Mozilla, GitHub, and the Linux Foundation advocate that projects provide community-focused and process-oriented documentation early to foster recruitment and activity. In this paper we investigate the introduction of these documents in FLOSS projects, including whether early documentation conforms to these recommendations or explains subsequent activity. We use a novel dataset of FLOSS projects packaged by the Debian GNU/Linux distribution and conduct a quantitative analysis to examine README (n=4226) and CONTRIBUTING (n=714) files when they are first published into projects' repositories. We find that projects create minimal READMEs proactively, but often publish CONTRIBUTING files following an influx of contributions. The initial versions of these files rarely focus on community development, instead containing descriptions of project procedure for library usage or code contribution. The findings suggest that FLOSS projects do not create documentation with community-building in mind, but rather favor brevity and standardized instructions.

cs.SE↗
arXiv2024

LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals

Machine learning can predict human behavior well when substantial structured data are available for well-defined outcomes. Such models are typically outcome-specific, however, requiring training data for each target outcome, limiting their applicability to new domains. We test whether large language models (LLMs) can relax these requirements by using self-report data to build attitudinal and behavioral simulations, or "generative agents," that can predict responses across outcomes without outcome-specific training data. Using data from a diverse national sample of 1,052 Americans, we built agents from (i) two-hour, semi-structured interviews elicited using the American Voices Project interview schedule, (ii) structured surveys including General Social Survey items and the Big Five personality inventory, or (iii) both sources combined. On held-out General Social Survey items, interview-only, survey-only, and combined agents achieved accuracies equal to 83%, 82%, and 86% of participants' own two-week test-retest consistency benchmark, respectively, compared with 74% for demographics-only agents. Combining interviews and surveys produced the highest accuracy, though gains over either source alone were modest, suggesting that predictive benefits from data begin to asymptote once the model has observed sufficient evidence within a domain. We find that these agents also predict personality traits, economic-game behavior, and experimental responses, while reducing accuracy disparities across racial and ideological groups relative to demographics-only agents. Together, these results show that LLM agents grounded in qualitative or quantitative self-reports can support general-purpose simulation of individuals across outcomes, without requiring task-specific training data.

cs.AI↗
arXiv2024

The Effects of Group Sanctions on Participation and Toxicity: Quasi-experimental Evidence from the Fediverse

Online communities often overlap and coexist, despite incongruent norms and approaches to content moderation. When communities diverge, decentralized and federated communities may pursue group-level sanctions, including defederation (disconnection) to block communication between members of specific communities. We investigate the effects of defederation in the context of the Fediverse, a set of decentralized, interconnected social networks with independent governance. Mastodon and Pleroma, the most popular software powering the Fediverse, allow administrators on one server to defederate from another. We use a difference-in-differences approach and matched controls to estimate the effects of defederation events on participation and message toxicity among affected members of the blocked and blocking servers. We find that defederation causes a drop in activity for accounts on the blocked servers, but not on the blocking servers. Also, we find no evidence of an effect of defederation on message toxicity.

cs.SI↗
arXiv2021

The Hidden Costs of Requiring Accounts: Quasi-Experimental Evidence From Peer Production

Online communities, like Wikipedia, produce valuable public information goods. Whereas some of these communities require would-be contributors to create accounts, many do not. Does this requirement catalyze cooperation or inhibit participation? Prior research provides divergent predictions but little causal evidence. We conduct an empirical test using longitudinal data from 136 natural experiments where would-be contributors to wikis were suddenly required to log in to contribute. Requiring accounts leads to a small increase in account creation, but reduces both high- and low-quality contributions from registered and unregistered participants. Although the change deters a large portion of low-quality participation, the vast majority of deterred contributions are of higher quality. We conclude that requiring accounts introduces an undertheorized tradeoff for public goods production in interactive communication systems.

cs.CY↗
arXiv2020

How individual behaviors drive inequality in online community sizes: an agent-based simulation

Why are online community sizes so extremely unequal? Most answers to this question have pointed to general mathematical processes drawn from physics like cumulative advantage. These explanations provide little insight into specific social dynamics or decisions that individuals make when joining and leaving communities. In addition, explanations in terms of cumulative advantage do not draw from the enormous body of social computing research that studies individual behavior. Our work bridges this divide by testing whether two influential social mechanisms used to explain community joining can also explain the distribution of community sizes. Using agent-based simulations, we evaluate how well individual-level processes of social exposure and decisions based on individual expected benefits reproduce empirical community size data from Reddit. Our simulations contribute to social computing theory by providing evidence that both processes together---but neither alone---generate realistic distributions of community sizes. Our results also illustrate the potential value of agent-based simulation to online community researchers to both evaluate and bridge individual and group-level theories.

cs.CY↗
arXiv2016

Black Lives Matter in Wikipedia: Collaboration and Collective Memory around Online Social Movements

Social movements use social computing systems to complement offline mobilizations, but prior literature has focused almost exclusively on movement actors' use of social media. In this paper, we analyze participation and attention to topics connected with the Black Lives Matter movement in the English language version of Wikipedia between 2014 and 2016. Our results point to the use of Wikipedia to (1) intensively document and connect historical and contemporary events, (2) collaboratively migrate activity to support coverage of new events, and (3) dynamically re-appraise pre-existing knowledge in the aftermath of new events. These findings reveal patterns of behavior that complement theories of collective memory and collective action and help explain how social computing systems can encode and retrieve knowledge about social movements as they unfold.

cs.SI↗
arXiv2014

WeDo: Exploring Participatory, End-To-End Collective Action

Many celebrate the Internet's ability to connect individuals and facilitate collective action toward a common goal. While numerous systems have been designed to support particular aspects of collective action, few systems support participatory, end-to-end collective action in which a crowd or community identifies opportunities, formulates goals, brainstorms ideas, develops plans, mobilizes, and takes action. To explore the possibilities and barriers in supporting such interactions, we have developed WeDo, a system aimed at promoting simple forms of participatory, end-to-end collective action. Pilot deployments of WeDo illustrate that sociotechnical systems can support automated transitions through different phases of end-to-end collective action, but that challenges, such as the elicitation of leadership and the accommodation of existing group norms, remain.

cs.CY↗
arXiv2014

Laboratories of Oligarchy? How the Iron Law Extends to Peer Production

Peer production projects like Wikipedia have inspired voluntary associations, collectives, social movements, and scholars to embrace open online collaboration as a model of democratic organization. However, many peer production projects exhibit entrenched leadership and deep inequalities, suggesting that they may not fulfill democratic ideals. Instead, peer production projects may conform to Robert Michels' "iron law of oligarchy," which proposes that democratic membership organizations become increasingly oligarchic as they grow. Using exhaustive data of internal processes from a sample of 683 wikis, we construct empirical measures of participation and test for increases in oligarchy associated with growth. In contrast to previous studies, we find support for Michels' iron law and conclude that peer production entails oligarchic organizational forms.

cs.CY↗