arXiv subjects

David W. McDonald

Publications and source records attributed to David W. McDonald.

15 recordsLinked to original sources

arXiv2026

How Much Can AI Understand? Toward AI-Assisted Sensemaking of Collaborative Discussion in Groups with Shared History

AI tools that support collaborative discussion typically treat the discussion as a standalone task, focusing only on its content and setting aside the social context of the group having it. But it is groups with a shared history, with their own norms, hierarchies, and relationships, where the most tangled and complex discussions tend to arise. These discussions cannot be understood apart from that context, and AI that overlooks it risks failing to convey what a discussion means, or even misrepresenting it. Drawing on two studies of how experienced Wikipedia editors read and make sense of discussions, we propose an AI-Assisted Sensemaking Model for Collaborative Discussions, which captures not only a discussion's arguments but also the norms and participants behind it, along with the context that gives each meaning. In this model, the system supports the early stages of the sensemaking process, and the degree to which it performs interpretive work can range from low to high. We argue that higher interpretive work reduces the burden on users but increases their reliance on the system's judgment. We then discuss the risks of an insufficiently intelligible system, what it would take to make one more intelligible, and the safeguards it still requires.

cs.HC↗

arXiv2026

Computer-Aided Tagging on Wikimedia Commons: Designing for Human-AI Collaboration in Open Knowledge Work

This study investigates Wikimedia Commons contributors' lived experiences with the Computer-Aided Tagging (CAT) tool, an AI-assisted image tagging system designed to improve Commons' discoverability, searchability, accessibility, and multilingual support. Using a qualitative analysis of 595 CAT-related community comments from 11 wiki pages and 16 in-depth interviews, we identify seven key issues that contributed to CAT's mixed reception and eventual deactivation. We also offer community-informed suggestions for improving the tool. We reflect on the implications for designing human-AI collaboration on Commons and for developing AI-assisted tools that support open knowledge work. This work contributes to HCI and CSCW research by extending the understanding of human-AI collaboration beyond Anglophone, text-centric, corporate platforms.

cs.HC↗

arXiv2026

Popularity Without Legitimacy? Comparing Trust in Television Meteorologists and YouTube Weatherfluencers

During severe weather events, people must interpret rapidly evolving information to make time-sensitive safety decisions. Broadcast meteorologists have traditionally served as credentialed intermediaries within established media organizations, while independent "weatherfluencers" on YouTube have emerged as prominent real-time interpreters for large and growing audiences. This mixed-methods study provides one of the first empirical comparisons of how viewers evaluate broadcast meteorologists against YouTube weatherfluencers across credibility, legitimacy, objectivity, and practical utility. Broadcast meteorologists were consistently rated higher on credibility, legitimacy, and safety utility, while weatherfluencers achieved parity on objectivity. Yet weatherfluencer audiences continue to grow, revealing a critical decoupling between audience attention and official or professional authorization that existing crisis communication models do not fully account for. Qualitative findings illuminate the mechanisms underlying these judgments and their implications for emergency communication in hybrid information ecosystems.

cs.HC↗

arXiv2026

Making Sense of the Weather, Together: Collaborative Sensemaking in Severe Weather Livestreams

This paper examines collaborative sensemaking during severe weather events through the emerging phenomenon of "weatherfluencers" or content creators who livestream meteorological interpretation on platforms like YouTube. Drawing from sensemaking theory, crisis informatics, and platform studies, we analyze how these creators navigate the sociotechnical dynamics of interpreting severe weather in real time with distributed audiences. Through critical incident analysis of 13 Particularly Dangerous Situation (PDS) storm warnings across three prominent weatherfluencers, we identify three key practices: multi-source information triangulation, temporal bridging techniques, and platform-specific adaptations that transform entertainment interfaces into safety-critical communication channels. Our analysis shows how these practices challenge existing models of crisis communication by integrating distributed expertise, collapsing temporal frames, and reconfiguring platform affordances. This research contributes to understanding how informal emergency communicators mediate between institutional alerting systems and public needs, and how visual, multimodal crisis communication differs from text-centered approaches.

cs.HC↗

arXiv2026

Not Another EHR: Reimagining Physician Information Needs with Generative AI Technology

Electronic health records (EHRs) have improved data accessibility but have also introduced cognitive burden for physicians, given the sheer volume and complexity of the data involved. Advances in large language models (LLMs) create new opportunities to rethink how clinicians interact with medical data through dynamic, adaptive interfaces. In this position paper, we explore how generative AI can support physicians' information needs by enabling more dynamic interactions with patient data. Through semi-structured interviews with internal physicians at Microsoft, we identify key challenges in data navigation and synthesis, and characterize clinicians' information needs during diagnostic workflows. We further examine how physicians conceptualize AI can help their work process and how these mental models shape expectations for interaction and trust. Based on these insights, we discuss design considerations for generative user interfaces that support clinician-centered workflows.

cs.HC↗

arXiv2025

Synthetic Cognitive Walkthrough: Aligning Large Language Model Performance with Human Cognitive Walkthrough

Conducting usability testing like cognitive walkthrough (CW) can be costly. Recent developments in large language models (LLMs), with visual reasoning and UI navigation capabilities, present opportunities to automate CW. We explored whether LLMs (GPT-4 and Gemini-2.5-pro) can simulate human behavior in CW by comparing their walkthroughs with human participants. While LLMs could navigate interfaces and provide reasonable rationales, their behavior differed from humans. LLM-prompted CW achieved higher task completion rates than humans and followed more optimal navigation paths, while identifying fewer potential failure points. However, follow-up studies demonstrated that with additional prompting, LLMs can predict human-identified failure points, aligning their performance with human participants. Our work highlights that while LLMs may not replicate human behaviors exactly, they can be leveraged for scaling usability walkthroughs and providing UI insights, offering a valuable complement to traditional usability testing.

cs.HC↗

arXiv2025

A Framework for AI-Supported Mediation in Community-based Online Collaboration

Online spaces involve diverse communities engaging in various forms of collaboration, which naturally give rise to discussions, some of which inevitably escalate into conflict or disputes. To address such situations, AI has primarily been used for moderation. While moderation systems are important because they help maintain order, common moderation strategies of removing or suppressing content and users rarely address the underlying disagreements or the substantive content of disputes. Mediation, by contrast, fosters understanding, reduces emotional tension, and facilitates consensus through guided negotiation. Mediation not only enhances the quality of collaborative decisions but also strengthens relationships among group members. For this reason, we argue for shifting focus toward AI-supported mediation. In this work, we propose an information-focused framework for AI-supported mediation designed for community-based collaboration. Within this framework, we hypothesize that AI must acquire and reason over three key types of information: content, culture, and people.

cs.HC↗

arXiv2025

Levels of Autonomy for AI Agents

Autonomy is a double-edged sword for AI agents, simultaneously unlocking transformative possibilities and serious risks. How can agent developers calibrate the appropriate levels of autonomy at which their agents should operate? We argue that an agent's level of autonomy can be treated as a deliberate design decision, separate from its capability and operational environment. In this work, we define five levels of escalating agent autonomy, characterized by the roles a user can take when interacting with an agent: operator, collaborator, consultant, approver, and observer. Within each level, we describe the ways by which a user can exert control over the agent and open questions for how to design the nature of user-agent interaction. We then highlight a potential application of our framework towards AI autonomy certificates to govern agent behavior in single- and multi-agent systems. We conclude by proposing early ideas for evaluating agents' autonomy. Our work aims to contribute meaningful, practical steps towards responsibly deployed and useful AI agents in the real world.

cs.HC↗

arXiv2025

Synthetic Heuristic Evaluation: A Comparison between AI- and Human-Powered Usability Evaluation

Usability evaluation is crucial in human-centered design but can be costly, requiring expert time and user compensation. In this work, we developed a method for synthetic heuristic evaluation using multimodal LLMs' ability to analyze images and provide design feedback. Comparing our synthetic evaluations to those by experienced UX practitioners across two apps, we found our evaluation identified 73% and 77% of usability issues, which exceeded the performance of 5 experienced human evaluators (57% and 63%). Compared to human evaluators, the synthetic evaluation's performance maintained consistent performance across tasks and excelled in detecting layout issues, highlighting potential attentional and perceptual strengths of synthetic evaluation. However, synthetic evaluation struggled with recognizing some UI components and design conventions, as well as identifying across screen violations. Additionally, testing synthetic evaluations over time and accounts revealed stable performance. Overall, our work highlights the performance differences between human and LLM-driven evaluations, informing the design of synthetic heuristic evaluations.

cs.HC↗

arXiv2025

Canvil: Designerly Adaptation for LLM-Powered User Experiences

Advancements in large language models (LLMs) are sparking a proliferation of LLM-powered user experiences (UX). In product teams, designers often craft UX to meet user needs, but it is unclear how they engage with LLMs as a novel design material. Through a formative study with 12 designers, we find that designers seek a translational process that enables design requirements to shape and be shaped by LLM behavior, motivating a need for designerly adaptation to facilitate this translation. We then built Canvil, a Figma widget that operationalizes designerly adaptation. We used Canvil as a probe to study designerly adaptation in a group-based design study (6 groups, N=17), finding that designers constructively iterated on both adaptation approaches and interface designs to enhance end-user interaction with LLMs. Furthermore, designers identified promising collaborative workflows for designerly adaptation. Our work opens new avenues for processes and tools that foreground designers' human-centered expertise when developing LLM-powered applications.

cs.HC↗

arXiv2024

Mapping the Design Space of Teachable Social Media Feed Experiences

Social media feeds are deeply personal spaces that reflect individual values and preferences. However, top-down, platform-wide content algorithms can reduce users' sense of agency and fail to account for nuanced experiences and values. Drawing on the paradigm of interactive machine teaching (IMT), an interaction framework for non-expert algorithmic adaptation, we map out a design space for teachable social media feed experiences to empower agential, personalized feed curation. To do so, we conducted a think-aloud study (N=24) featuring four social media platforms -- Instagram, Mastodon, TikTok, and Twitter -- to understand key signals users leveraged to determine the value of a post in their feed. We synthesized users' signals into taxonomies that, when combined with user interviews, inform five design principles that extend IMT into the social media setting. We finally embodied our principles into three feed designs that we present as sensitizing concepts for teachable feed experiences moving forward.

cs.HC↗

arXiv2023

How Do UX Practitioners Communicate AI as a Design Material? Artifacts, Conceptions, and Propositions

UX practitioners (UXPs) face novel challenges when working with and communicating artificial intelligence (AI) as a design material. We explore how UXPs communicate AI concepts when given hands-on experience training and experimenting with AI models. To do so, we conducted a task-based design study with 27 UXPs in which they prototyped and created a design presentation for a AI-enabled interface while having access to a simple AI model training tool. Through analyzing UXPs' design presentations and post-activity interviews, we found that although UXPs struggled to clearly communicate some AI concepts, tinkering with AI broadened common ground when communicating with technical stakeholders. UXPs also identified key risks and benefits of AI in their designs, and proposed concrete next steps for both UX and AI work. We conclude with a sensitizing concept and recommendations for design and AI tools to enhance multi-stakeholder communication and collaboration when crafting human-centered AI experiences.

cs.HC↗

arXiv2023

Addressing UX Practitioners' Challenges in Designing ML Applications: an Interactive Machine Learning Approach

UX practitioners face novel challenges when designing user interfaces for machine learning (ML)-enabled applications. Interactive ML paradigms, like AutoML and interactive machine teaching, lower the barrier for non-expert end users to create, understand, and use ML models, but their application to UX practice is largely unstudied. We conducted a task-based design study with 27 UX practitioners where we asked them to propose a proof-of-concept design for a new ML-enabled application. During the task, our participants were given opportunities to create, test, and modify ML models as part of their workflows. Through a qualitative analysis of our post-task interview, we found that direct, interactive experimentation with ML allowed UX practitioners to tie ML capabilities and underlying data to user goals, compose affordances to enhance end-user interactions with ML, and identify ML-related ethical risks and challenges. We discuss our findings in the context of previously established human-AI guidelines. We also identify some limitations of interactive ML in UX processes and propose research-informed machine teaching as a supplement to future design tools alongside interactive ML.

cs.HC↗

arXiv2018

Ease on Down the Code: Complex Collaborative Qualitative Coding Simplified with 'Code Wizard'

This paper describes the design and development of a preliminary qualitative coding tool as well as a method to improve the process of achieving inter-coder reliability (ICR) in small teams. Software applications that support qualitative coding do not sufficiently assist collaboration among coders and overlook some fundamental issues related to ICR. We propose a new dimension of collaborative coding called "coders' certainty" and demonstrate its ability to illustrate valuable code disagreements that are missing from existing approaches. Through a case study, we describe the utility of our tool, Code Wizard, and how it helped a group of researchers effectively collaborate to code naturalistic observation data. We report the valuable lessons we learned from the development of our tool and method: (1) identifying coders' certainty constitutes an important part of determining the quality of data analysis and facilitates identifying overlapping and ambiguous codes, (2) making the details of coding process visible helps streamline the coding process and leads to a sense of ownership of the research results, and (3) there is valuable information hidden in coding disagreements that can be leveraged for improving the process of data analysis.

cs.HC↗

arXiv2016

Dissecting a Social Botnet: Growth, Content and Influence in Twitter

Social botnets have become an important phenomenon on social media. There are many ways in which social bots can disrupt or influence online discourse, such as, spam hashtags, scam twitter users, and astroturfing. In this paper we considered one specific social botnet in Twitter to understand how it grows over time, how the content of tweets by the social botnet differ from regular users in the same dataset, and lastly, how the social botnet may have influenced the relevant discussions. Our analysis is based on a qualitative coding for approximately 3000 tweets in Arabic and English from the Syrian social bot that was active for 35 weeks on Twitter before it was shutdown. We find that the growth, behavior and content of this particular botnet did not specifically align with common conceptions of botnets. Further we identify interesting aspects of the botnet that distinguish it from regular users.

cs.CY↗