Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2609.35781

Spotting (and Missing) Algorithmic Bias: Investigating User Understanding in a Fairness Assessment Tool

Anna Verheyden·Yizhe Zhang·Robin De Croon·Simone Stumpf·Katrien Verbert
View original source ↗Read source document ↗Export metadata ↓

Abstract

Fairness metric selection is typically left to data scientists, but which biases are problematic and which metric captures them best depends on stakeholders' experience and domain knowledge. This calls for involving non-technical stakeholders, but the research prototypes built for this purpose so far have not tested whether these stakeholders form accurate mental models of the metrics they interact with or can act on them to identify biases. We present FairAware, a fairness assessment tool co-designed with Human Resources (HR) domain experts. We evaluate stakeholders' understanding through a mixed-methods study with 70 participants (35 HR employees, 35 job seekers), measuring objective and subjective understanding, cognitive load, bias identification accuracy, and open-ended feedback. Most participants correctly identified the most disadvantaged group, with task duration being the only significant predictor. We also found a gap between subjective and objective understanding, with both groups performing similarly across all measures. These results suggest that fairness assessment tools for non-experts are usable for identifying biases but need built-in checks on understanding before stakeholders make higher-stakes decisions.

Explore related subjects

cs.HC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Anna Verheyden, Yizhe Zhang, Robin De Croon, Simone Stumpf, Katrien Verbert. 2026-08-05. Spotting (and Missing) Algorithmic Bias: Investigating User Understanding in a Fairness Assessment Tool. https://arxiv.org/abs/2609.35781

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Stayin' Aligned Over Time: Towards Longitudinal Human-LLM Alignment via Contextual Reflection and Privacy-Preserving Behavioral Data

Current human-AI alignment and evaluation methods for large language models (LLMs) often rely on preference signals collected immediately after an interaction. This practice implicitly treats preference as static, even though many LLM-mediated decisions unfold over time and may be re-evaluated differently after real-world consequences and observed outcomes. Therefore, we argue for a methodological shift from single-moment preference elicitation to longitudinal, context-situated alignment measurement. We present a methodological framework for collecting temporally grounded alignment signals by combining (1) in-situ preference capture, (2) context-triggered follow-up preference reflection, and (3) privacy-preserving behavioral traces that help interpret preference change. As an instantiation of this methodology, we introduce BITE, a browser-based system that detects consequential LLM interactions, prompts reflection across later decision points, and supports progressive, user-controlled consent for sharing behavioral data. Through a two week longitudinal deployment study with 8 participants, our approach surfaced differences between immediate and later user preferences in accuracy, relevance and other dimensions of the LLM output. Our findings highlight the limitations of single-moment preference datasets and underscore the importance of longitudinal methods for alignment evaluation in everyday use.

cs.HC↗
arXiv2026

Personalized to Persuade: The Effects of Contextualization and Warmth on Trust and Reliance in Conversational AI

Contextualization is a form of personalization in which the message is framed in a context meaningful to the recipient, using their interests and background. Personalization is a known persuasive strategy of artificial intelligence (AI) systems in politics and marketing. However, the persuasive effect of contextualization in everyday tasks, where users often lack prior knowledge, remains unclear. We ran a $2\times2$ between-subjects experiment ($N = 407$) in a fictional scenario where participants had no prior opinions or knowledge. We examined how contextualization, combined with conversational warmth, shapes the persuasive power of an AI assistant arguing against expert recommendations and users' reliance on it. Our findings show that the neutral, non-contextualized assistant was the most persuasive. Contextualization and warmth each reduced persuasiveness on their own, but their interaction increased it. Reliance was present in all conditions, and neither cue changed it detectably. Trust predicted persuasion and reliance, yet neither contextualization nor warmth had an indirect effect through trust. Participants with higher self-reported AI literacy relied more on the assistant without trusting it more. Our study extends AI personalization research from political opinions to everyday decisions made without prior knowledge. The results suggest that users are more likely to be persuaded by, and defer to, an AI assistant than an expert, even without personalization. AI persuasion experiments should therefore include untailored assistants and measure decisions; the cues that changed persuasion did not detectably change reliance.

cs.HC↗
arXiv2026

An Open Book, On Whose Terms? Power Dynamics in Sensor-Derived, LLM-Generated Summaries Sharing Between Older Adults and Remote Families

As older adults (OAs) increasingly favor aging in place, remote family members (RFMs) need meaningful awareness of their loved ones without intrusive monitoring. Large language models can provide rich interpretations of non-intrusive sensing data, yet recent work has largely emphasized technical accuracy over how families use or benefit from this information. We conducted formative interviews with 12 RFMs and 2 OAs, followed by a three-month deployment case study with one OA-RFM dyad. Our system provided daily AI-generated summaries of multimodal sensing data, which we iteratively refined with both parties using a research-through-design approach. Although the summaries integrated well into the dyad's daily routine, sustaining over 90% engagement, their lived experience revealed evolving tensions around agency, reassurance, privacy, trust, and communication, shaped by sensor limitations and existing communication misalignments. Our findings inform the design of future AI-mediated sharing systems that account for real-world sensing constraints and complex family care dynamics.

cs.HC↗