arXiv subjects

Genevieve Smith

Publications and source records attributed to Genevieve Smith.

9 recordsLinked to original sources

arXiv2026

Standard Language Ideology in AI-Generated Language

Large language models (LLMs) generate text that reinforces standard language ideology: a bias towards certain language varieties that are granted more prestige, authority, and legitimacy than others. This paper contributes a sociotechnically grounded faceted taxonomy that illustrates how generative AI systems reproduce standard language ideology and its societal implications. We introduce the concept of standard AI-generated language ideology to explain how AI systems confer legitimacy on certain language varieties while marginalizing others, structuring patterns of performance disparity, stereotyping, appropriation, and erasure. We then discuss ongoing tensions around what constitutes desirable system behavior, as well as advantages and drawbacks of generative AI tools attempting or refusing to imitate different language varieties. To address the power relations shaping generative AI and the mechanisms identified in our taxonomy--legitimation, stereotyping, appropriation, and erasure--we offer recommendations that emphasize accountability, community agency, control, and ownership. These recommendations recognize linguistic diversity as a resource to be protected, valued, and sustained as part of a just AI future.

cs.CL↗

arXiv2026

Guardrails versus Gatekeepers: Understanding Product Managers' Ethical Decision-Making in Generative AI

What is the role of product managers in the responsible use of generative AI (genAI) in products and everyday work -- and what enables or constrains their ability to take action? Past literature has examined the ways in which organizational policies can become decoupled from practices when incentives for responsible action are misaligned or impeded by profit motives. While the role of engineers and professional ethicists in the context of AI has been examined in detail, the role of product managers -- who are frequently portrayed as "gatekeepers" or critical decision-makers in product teams -- remains unclear. In this paper, we examine what organizational conditions promote responsible use of genAI by product managers by drawing on twenty-five interviews and a global survey of over three hundred respondents in product management-related roles. We find that uncertainty around responsible AI and a sense of diffused responsibility constrain ethical action, while leadership commitment and organizational principles enable ethical action -- making some responsible practices up to fourteen times more likely. Further, we find two sets of actions product managers take to "recouple" ethical commitments and practices. The first includes low-resource, individual actions product managers can implement without explicit organizational incentives. The second includes high-resource, collective actions that require organizational incentives. Our research suggests recoupling ethical policies and practices at the level of product teams requires institutional buy-in and higher level leadership commitment. Nevertheless, we show that individual actors are able to exhibit agency through some meaningful, low resource actions, even in the absence of organizational incentives, though this alone is insufficient to operationalize responsible AI at scale.

cs.CY↗

arXiv2026

Reimagining Open Source and Openness in AI: Co-Creating Responsible Technological Futures

Debates over open source and openness in artificial intelligence have intensified as policymakers, researchers, and practitioners grapple with how foundation models should be developed and governed to balance innovation, accountability, and public interest. However, there has been limited empirical work examining how diverse stakeholders collectively understand and negotiate responsible openness in AI, particularly through participatory processes that extend beyond industry-led definitions and frameworks. This paper presents findings from a multi-sectoral workshop grounded in futures thinking and participatory design methods. The workshop generated co-created visions of desirable futures and the role of AI, alongside a set of action pathways and a research roadmap focused on responsible open source and openness in AI. This paper makes three key contributions. First, it empirically documents the co-created visions, actions, and research priorities. Second, it identifies four core tensions that emerged as participants translated high-level aspirations into concrete actions, revealing conflicting interpretations of openness regarding its purpose (as an end or a means), its scope (expansion versus meaningful access), and its operation (mandatory versus conditional, sufficient versus dependent on governance and use). These tensions illustrate that responsible openness is not a singular technical solution, but a negotiated sociotechnical project shaped by values, positionalities, and priorities. Third, the paper advances methodological approaches in AI governance by demonstrating how participatory futures methods can surface plural visions, actions, and research priorities that extend beyond dominant, largely corporate, narratives, offering empirical insight into how openness, power, and accountability are negotiated in practice.

cs.CY↗

arXiv2026

Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Models

Vision-language models trained on large-scale multimodal datasets show strong demographic biases, but the role of training data in producing these biases remains unclear. A major barrier has been the lack of demographic annotations in web-scale datasets such as LAION-400M. We address this gap by creating person-centric annotations for the full dataset, including over 276 million bounding boxes, perceived gender and race/ethnicity labels, and automatically generated captions. These annotations are produced through validated automatic labeling pipelines combining object detection, multimodal captioning, and finetuned classifiers. Using them, we uncover demographic imbalances and harmful associations, such as the disproportionate linking of men and individuals perceived as Black or Middle Eastern with crime-related and negative content. We also show that a linear fit predicts 60-70% of gender bias in CLIP and Stable Diffusion from direct co-occurrences in the data. Our resources establish the first large-scale empirical link between dataset composition and downstream model bias. Code is available at https://github.com/ExplainableML/LAION-400M-Person-Centric-Annotations.

cs.CV↗

arXiv2025

A Large Scale Analysis of Gender Biases in Text-to-Image Generative Models

With the increasing use of image generation technology, understanding its social biases, including gender bias, is essential. This paper presents a large-scale study on gender bias in text-to-image (T2I) models, focusing on everyday situations. While previous research has examined biases in occupations, we extend this analysis to gender associations in daily activities, objects, and contexts. We create a dataset of 3,217 gender-neutral prompts and generate 200 images over 5 prompt variations per prompt from five leading T2I models. We automatically detect the perceived gender of people in the generated images and filter out images with no person or multiple people of different genders, leaving 2,293,295 images. To enable a broad analysis of gender bias in T2I models, we group prompts into semantically similar concepts and calculate the proportion of male- and female-gendered images for each prompt. Our analysis shows that T2I models reinforce traditional gender roles and reflect common gender stereotypes in household roles. Women are predominantly portrayed in care and human-centered scenarios, and men in technical or physical labor scenarios.

cs.CV↗

arXiv2025

Mindsets and Management: AI and Gender (In)Equitable Access to Finance

A growing trend in financial technology (fintech) is the use of mobile phone data and machine learning (ML) to provide credit scores- and subsequently, opportunities to access loans- to groups left out of traditional banking. This paper draws on interview data with leaders, investors, and data scientists at fintech companies developing ML-based alternative lending apps in low- and middle-income countries to explore financial inclusion and gender implications. More specifically, it examines how the underlying logics, design choices, and management decisions of ML-based alternative lending tools by fintechs embed or challenge gender biases, and consequently influence gender equity in access to finance. Findings reveal developers follow 'gender blind' approaches, grounded in beliefs that ML is objective and data reflects the truth. This leads to a lack of grappling with the ways data, features for creditworthiness, and access to apps are gendered. Overall, tools increase access to finance, but not gender equitably: Interviewees report less women access loans and receive lower amounts than men, despite being better repayers. Fintechs identify demand- and supply-side reasons for gender differences, but frame them as outside their responsibility. However, that women are observed as better repayers reveals a market inefficiency and potential discriminatory effect, further linked to profit optimization objectives. This research introduces the concept of encoded gender norms, whereby without explicit attention to the gendered nature of data and algorithmic design, AI tools reproduce existing inequalities. In doing so, they reinforce gender norms as self-fulfilling prophecies. The idea that AI is inherently objective and, when left alone, 'fair', is seductive and misleading. In reality, algorithms reflect the perspectives, priorities, and values of the people and institutions that design them.

cs.CY↗

arXiv2024

Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination

We present a large-scale study of linguistic bias exhibited by ChatGPT covering ten dialects of English (Standard American English, Standard British English, and eight widely spoken non-"standard" varieties from around the world). We prompted GPT-3.5 Turbo and GPT-4 with text by native speakers of each variety and analyzed the responses via detailed linguistic feature annotation and native speaker evaluation. We find that the models default to "standard" varieties of English; based on evaluation by native speakers, we also find that model responses to non-"standard" varieties consistently exhibit a range of issues: stereotyping (19% worse than for "standard" varieties), demeaning content (25% worse), lack of comprehension (9% worse), and condescending responses (15% worse). We also find that if these models are asked to imitate the writing style of prompts in non-"standard" varieties, they produce text that exhibits lower comprehension of the input and is especially prone to stereotyping. GPT-4 improves on GPT-3.5 in terms of comprehension, warmth, and friendliness, but also exhibits a marked increase in stereotyping (+18%). The results indicate that GPT-3.5 Turbo and GPT-4 can perpetuate linguistic discrimination toward speakers of non-"standard" varieties.

cs.CL↗

arXiv2024

Risks and Opportunities of Open-Source Generative AI

Applications of Generative AI (Gen AI) are expected to revolutionize a number of different areas, ranging from science & medicine to education. The potential for these seismic changes has triggered a lively debate about the potential risks of the technology, and resulted in calls for tighter regulation, in particular from some of the major tech companies who are leading in AI development. This regulation is likely to put at risk the budding field of open-source generative AI. Using a three-stage framework for Gen AI development (near, mid and long-term), we analyze the risks and opportunities of open-source generative AI models with similar capabilities to the ones currently available (near to mid-term) and with greater capabilities (long-term). We argue that, overall, the benefits of open-source Gen AI outweigh its risks. As such, we encourage the open sourcing of models, training and evaluation data, and provide a set of recommendations and best practices for managing risks associated with open-source generative AI.

cs.LG↗

arXiv2024

Near to Mid-term Risks and Opportunities of Open-Source Generative AI

In the next few years, applications of Generative AI are expected to revolutionize a number of different areas, ranging from science & medicine to education. The potential for these seismic changes has triggered a lively debate about potential risks and resulted in calls for tighter regulation, in particular from some of the major tech companies who are leading in AI development. This regulation is likely to put at risk the budding field of open-source Generative AI. We argue for the responsible open sourcing of generative AI models in the near and medium term. To set the stage, we first introduce an AI openness taxonomy system and apply it to 40 current large language models. We then outline differential benefits and risks of open versus closed source AI and present potential risk mitigation, ranging from best practices to calls for technical and scientific contributions. We hope that this report will add a much needed missing voice to the current public discourse on near to mid-term AI safety and other societal impact.

cs.LG↗