arXiv subjects

Haiyi Zhu

Publications and source records attributed to Haiyi Zhu.

At least 19 recordsLinked to original sources

arXiv2026

"I want to be pushed, I want to grow": Enabling social workers to design evaluations of LLM augmentation in their work

Workers are increasingly asked to adopt AI systems to assist their work, yet are rarely given a voice in defining what meaningful AI augmentation should look like or how to evaluate for it. In this paper, we propose worker-driven AI measurement---a bottom-up approach to AI evaluation where workers collaboratively shape decisions about which tasks AI should augment, what "successful" augmentation looks like, and how it should be measured. We explore how to support this through a case study with 19 workers from a local school social work organization. Through a series of eight workshops, workers iteratively develop their own measurement goals for AI evaluation, systematize these goals, and then design a benchmark to capture how effectively an LLM can "challenge" them to reflect on their own assumptions and biases in the context of their day-to-day work. Workers collaboratively design and refine an LLM-as-a-judge rubric based on their professional and lived expertise. In validations of the worker-created benchmark, we find that there is strong agreement between worker and LLM judge ratings and that the resulting benchmark can differentiate performance across six state-of-the-art LLMs. Based on our case study, we discuss opportunities for future work to support worker-driven AI measurement as a complementary approach to existing top-down AI evaluation approaches.

cs.HC↗

arXiv2026

Empirical Modeling of Therapist-Client Dynamics in Psychotherapy Using LLM-Based Assessments

Psychotherapy is a primary treatment for many mental health conditions, yet the interplay among therapist behaviors, client responses, and the therapeutic relationship is difficult to study at scale, as process research has relied on labor-intensive human coding. We develop and validate a computational framework for modeling therapist-client interaction, using large language models (LLMs) to measure therapist behaviors (empathy, exploration), relational quality (rapport), and client outcomes (self-disclosure, self-directed and outward-directed negative emotion). After validating model-generated scores against human annotations (ICC = 0.45-0.81; rapport 0.81, self-disclosure 0.78), we apply these measures to roughly 2,000 hours of transcripts from the Alexander Street corpus and use Structural Equation Modeling to estimate moment-to-moment relationships among therapist behaviors, rapport, and subsequent client responses, controlling for prior client state and context. Therapist empathy and exploration directly predict increased client disclosure and shifts in emotional expression; empathy is more strongly associated with self-directed than outward-directed negative emotion, suggesting greater acknowledgment of internal distress, while exploration increases disclosure and emotional elaboration. Rapport does not directly amplify disclosure or emotional intensity but instead moderates the associations between therapist behaviors and client affect, potentially contributing to reductions in internal distress. These results show that LLM-based measurement combined with structural modeling can capture core therapeutic processes at scale, with empathy and exploration acting directly and rapport as a contextual moderator, providing a foundation for precision modeling of psychotherapy and for scalable therapist training and AI-supported clinical education.

cs.CY↗

arXiv2026

The Empirically Grounded Adaptive Virtual Patient for Psychotherapy Training: Disclosure That Responds to Therapist Micro-Skills

Simulated patients offer a scalable way to train psychotherapy micro-skills such as empathic responding and exploratory probing, but current systems either follow fixed scripts or rely on LLMs that drift unpredictably over long sessions. We present the Adaptive Virtual Patient (AVP), which adapts its disclosure behavior -- from guarded, through moderate openness, to full disclosure -- in response to trainee skill. The AVP is grounded in a structural equation model fit to nearly 2{,}000 hours of real-world psychotherapy transcripts, which quantifies how therapist empathy and exploration shift a patient's openness over time. An LLM generates the AVP's utterances conditioned on a disclosure level that the dynamics module updates each turn. In an evaluation with 20 clinicians and trainees over 80 sessions (1{,}033 turns), the AVP's disclosure rises in response to therapist empathy and exploration, while a prompt-only baseline stays flat; ablations confirm that the empirically motivated parameterization outperforms alternatives, with exploration carrying most of the adaptive signal.

cs.CY↗

arXiv2026

Design Principles for Human-Agent Interaction

AI agents are rapidly evolving into autonomous systems capable of sustained interaction, tool use, and long-term collaboration. Yet their real-world adoption remains limited, suggesting that the key barrier lies not only in technical capability but also in a lack of design knowledge for successful human-agent interaction. This position paper argues that AI agents should not be solely evaluated or deployed based on autonomous task capability alone; because agents interact with, adapt to, influence, and sometimes fail humans, human-agent interaction must be treated as a core design and evaluation target for agentic AI. We present 14 design principles that articulate the ideal human-agent relationship across four interaction stages: initially, during interaction, over time, and when things go wrong. We use these principles to evaluate nine agent systems to illustrate that these design principles can provide actionable guidance for AI design teams to systematically design and evaluate agents that are usable, trustworthy, and effective in real-world interactive settings.

cs.HC↗

arXiv2026

Understanding, Challenging, and Demystifying Perceptions of Gig Worker Vulnerabilities

Across service domains, platform-based gig workers often face a wide range of severe yet hidden vulnerabilities, including opaque pay practices, illusions of flexibility, health and safety risks, and privacy violations. To the general public and inexperienced workers such latent vulnerabilities remain largely unknown and concealed by intentional platform design that gives illusions of adequate labor protections, or $\textit{myths}$. This study examines how workers perceive (and shift their beliefs away from) five commonly held misconceptions regarding gig worker vulnerabilities. In $Phase~I$, crowdworkers ($N~=~236$) rated their agreement with five common myths surrounding vulnerabilities in gig work:$~227$ of them believed one or more myth(s). In $Phase~II$, we challenged these workers to defend their views by presenting an expert- or LLM-generated counterargument. Our findings show workers' underexposure to personal and shared vulnerabilities of gig work, revealing a knowledge gap where persuasive interventions can scalably raise awareness around such hidden labor conditions. We reflect on the effectiveness of different persuasion strategies and discuss implications for promoting more accurate public perceptions that support collective bargaining of workers' rights.

cs.HC↗

arXiv2026

The Algorithmic Gaze of Image Quality Assessment: An Audit and Trace Ethnography of the LAION-Aesthetics Predictor

Visual generative AI models are trained using a one-size-fits-all measure of aesthetic appeal. However, what is deemed "aesthetic" is inextricably linked to personal taste and cultural values, raising the question of whose taste is represented in visual generative AI models. In this work, we study an aesthetic evaluation model--LAION-Aesthetics Predictor (LAP)--that is widely used to curate datasets to train visual generative image models, like Stable Diffusion, and evaluate the quality of AI-generated images. To understand what LAP measures, we audited the model across three datasets. First, we examined the impact of aesthetic filtering on the LAION-Aesthetics Dataset (approximately 1.2B images), which was curated from LAION-5B using LAP. We find that the LAP disproportionally filters in images with captions mentioning women, while filtering out images with captions mentioning men or LGBTQ+ people. Then, we used LAP to score approximately 330k images across two art datasets, finding the model rates realistic images of landscapes, cityscapes, and portraits from western and Japanese artists most highly. In doing so, the algorithmic gaze of this aesthetic evaluation model reinforces the imperial and male gazes found within western art history. In order to understand where these biases may have originated, we performed a digital ethnography of public materials related to the creation of LAP. We find that the development of LAP reflects the biases we found in our audits, such as the aesthetic scores used to train LAP primarily coming from English-speaking photographers and western AI-enthusiasts. In response, we discuss how aesthetic evaluation can perpetuate representational harms and call on AI developers to shift away from prescriptive measures of "aesthetics" toward more pluralistic evaluation.

cs.HC↗

arXiv2026

"I Just Don't Want My Work Being Fed Into The AI Blender": Queer Artists on Refusing and Resisting Generative AI

Art-making is a collective social activity through which queer people engage in political resistance, develop identities, archive queer memory, and form community. However, in recent years, generative AI has disrupted queer artistic communities. Through 15 semi-structured interviews, we examine how queer artists are making sense of the encroachment of GenAI into their art worlds. Our findings surface significant tensions between the relationality of our participants' queer art practices and the perceived anti-relationality of GenAI development and use. We detail how our participants refuse and resist GenAI use and development in response and highlight the limited role our participants saw for GenAI within art-making, such as the queer aesthetic potential of surreal image models. Drawing on queer theory, we discuss how CSCW researchers might support queer artists by refusing dominant AI imaginaries and supporting queer world-building.

cs.HC↗

arXiv2026

Simulating Couple Conflict: Designing A Multi-Agent System for Therapy Training and Practice

Couples therapy requires managing complex, evolving emotional dynamics between partners, but traditional training methods for therapists, like role-play, lack realism, consistency, and control. We present a multi-modal simulation that models therapy as a controlled, multi-agent dynamical system with structured interaction stages. Therapists practice with a pair of client-agents who go through six evolving stages that respond to therapist actions. This simulation enables practice with demand-withdraw conflict patterns in a closed-loop environment. The simulation uses a sense-plan-act architecture: it detects the therapist's input, updates agents' interaction states based on psychotherapy theory and transcript analysis, and generates realistic verbal and emotional responses. In an experiment with 21 licensed U.S. therapists, participants more accurately identified state transitions and rated the system as more realistic and responsive than a prompt-based baseline, demonstrating the value of stateful, interpretable simulation for therapist training.

cs.CY↗

arXiv2026

Botender: Supporting Communities in Collaboratively Designing AI Agents through Case-Based Provocations

AI agents, or bots, serve important roles in online communities. However, they are often designed by outsiders or a few tech-savvy members, leading to bots that may not align with the broader community's needs. How might communities collectively shape the behavior of community bots? We present Botender, a system that enables communities to collaboratively design LLM-powered bots without coding. With Botender, community members can directly propose, iterate on, and deploy custom bot behaviors tailored to community needs. Botender facilitates testing and iteration on bot behavior through case-based provocations: interaction scenarios generated to spark user reflection and discussion around desirable bot behavior. A validation study found these provocations more useful than standard test cases for revealing improvement opportunities and surfacing disagreements. During a five-day deployment across six Discord servers, Botender supported communities in tailoring bot behavior to their specific needs, showcasing the usefulness of case-based provocations in facilitating collaborative bot design.

cs.HC↗

arXiv2026

AI Failure Loops in Devalued Work: The Confluence of Overconfidence in AI and Underconfidence in Worker Expertise

A growing body of literature has focused on understanding and addressing workplace AI design failures. However, past work has largely overlooked the role of the devaluation of worker expertise in shaping the dynamics of AI development and deployment. In this paper, we examine the case of feminized labor: a class of devalued occupations historically misnomered as ``women's work,'' such as social work, K-12 teaching, and home healthcare. Drawing on literature on AI deployments in feminized labor contexts, we conceptualize AI Failure Loops: a set of interwoven, socio-technical failure modes that help explain how the systemic devaluation of workers' expertise negatively impacts, and is impacted by, AI design, evaluation, and governance practices. These failures demonstrate how misjudgments on the automatability of workers' skills can lead to AI deployments that fail to bring value to workers and, instead, further diminish the visibility of workers' expertise. We discuss research and design implications for workplace AI, especially for devalued occupations.

cs.CY↗

arXiv2025

Beyond Riding: Passenger Engagement with Driver Labor through Gamified Interactions

Modern cities increasingly rely on ridesharing services for on-demand transportation, which offer consumers convenience and mobility across the globe. However, these marketed consumer affordances give rise to burdens and vulnerabilities that drivers shoulder alone, without adequate infrastructures for labor regulations or consumer-led advocacy. To effectively and sustainably advance protections and oversight for drivers, consumers must first be aware of the labor, logistics and costs involved with ridehail driving. To motivate consumers to practice more socially responsible consumption behaviors and foster solidarity with drivers, we explore the potential for gamified in-ride interactions to facilitate engagement with real (and lived) driver experiences. Through nine workshops with 19 drivers and 15 passengers, we surface how gamified in-ride interactions revealed passenger knowledge gaps around latent ridehail conditions, prompt reflection and shifts in perception of their relative power and consumption behaviors, and highlight drivers' preferences for creating more immersive and contextualized service experiences, and identify opportunities to design safe and appropriate passenger-driver interactions that motivate solidarity with drivers. In sum, we advance conceptual understandings of in-ride social and managerial relations, demonstrate potential for future worker advocacy in algorithmically-managed labor, and offer design guidelines for more human-centered workplace technologies.

cs.HC↗

arXiv2025

Social Simulation for Integrating Self-Care: Measuring the Effects of Contextual Environments in Augmented Reality for Mental Health Practice

Despite growing interest in virtual and augmented reality (VR/AR) for mental well-being, prior work using immersive interventions to teach mental health skills has largely focused on calming or abstract settings. As a result, little is known about how realistic social simulation may better support the transfer and application of skills to in-person environments. In this work, we present a 14-day user study with 43-participants comparing an augmented reality intervention simulating a realistic contextual environment against a matched non-contextual control, applied to the public speaking context. We found that participants who practice mental health skills in the contextual environment showed significantly greater likelihood to apply self-care techniques and greater physiological stress reduction when using skills in mock in-person tasks. Overall, our work provides empirical evidence for the effects of realistic stressor simulation, and offers design implications for mental health technology that supports effective transfer of skills to the real-world.

cs.HC↗

arXiv2025

Exploring Opportunities to Support Novice Visual Artists' Inspiration and Ideation with Generative AI

Recent generative AI advances present new possibilities for supporting visual art creation, but how such promise might assist novice artists during early-stage processes requires investigation. How novices adopt or resist these tools can shift the relationship between the art community and generative systems. We interviewed 13 artists to uncover needs in key dimensions during early stages of creation: (1) quicker and better access to references, (2) visualizations of reference combinations, (3) external artistic feedback, and (4) personalized support to learn new techniques and styles. Mapping such needs to state-of-the-art open-sourced advances, we developed a set of six interactive prototypes to expose emerging capabilities to novice artists. Afterward, we conducted co-design workshops with 13 novice visual artists through which artists articulated requirements and tensions for artist-centered AI development. Our work reveals opportunities to design novice-targeted tools that foreground artists' needs, offering alternative visions for generative AI to serve visual creativity.

cs.HC↗

arXiv2025

POET: Supporting Prompting Creativity and Personalization with Automated Expansion of Text-to-Image Generation

State-of-the-art visual generative AI tools hold immense potential to assist users in the early ideation stages of creative tasks -- offering the ability to generate (rather than search for) novel and unprecedented (instead of existing) images of considerable quality that also adhere to boundless combinations of user specifications. However, many large-scale text-to-image systems are designed for broad applicability, yielding conventional output that may limit creative exploration. They also employ interaction methods that may be difficult for beginners. Given that creative end users often operate in diverse, context-specific ways that are often unpredictable, more variation and personalization are necessary. We introduce POET, a real-time interactive tool that (1) automatically discovers dimensions of homogeneity in text-to-image generative models, (2) expands these dimensions to diversify the output space of generated images, and (3) learns from user feedback to personalize expansions. An evaluation with 28 users spanning four creative task domains demonstrated POET's ability to generate results with higher perceived diversity and help users reach satisfaction in fewer prompts during creative tasks, thereby prompting them to deliberate and reflect more on a wider range of possible produced results during the co-creative process. Focusing on visual creativity, POET offers a first glimpse of how interaction techniques of future text-to-image generation tools may support and align with more pluralistic values and the needs of end users during the ideation stages of their work.

cs.CV↗

arXiv2025

Adaptive-VP: A Framework for LLM-Based Virtual Patients that Adapts to Trainees' Dialogue to Facilitate Nurse Communication Training

Effective communication training is essential to preparing nurses for high-quality patient care. While standardized patient (SP) simulations provide valuable experiential learning, they are often costly and inflexible. Virtual patient (VP) systems offer a scalable alternative, but most fail to adapt to the varying communication skills of trainees. In particular, when trainees respond ineffectively, VPs should escalate in hostility or become uncooperative--yet this level of adaptive interaction remains largely unsupported. To address this gap, we introduce Adaptive-VP, a VP dialogue generation framework that leverages large language models (LLMs) to dynamically adapt VP behavior based on trainee input. The framework features a pipeline for constructing clinically grounded yet flexible VP scenarios and a modular system for assessing trainee communication and adjusting VP responses in real time, while ensuring learner safety. We validated Adaptive-VP by simulating challenging patient conversations. Automated evaluation using a corpus from practicing nurses showed that our communication skill evaluation mechanism reflected real-world proficiency levels. Expert nurses further confirmed that Adaptive-VP produced more natural and realistic interactions than existing approaches, demonstrating its potential as a scalable and effective tool for nursing communication training.

cs.CL↗

arXiv2025

Un-Straightening Generative AI: How Queer Artists Surface and Challenge the Normativity of Generative AI Models

Queer people are often discussed as targets of bias, harm, or discrimination in research on generative AI. However, the specific ways that queer people engage with generative AI, and thus possible uses that support queer people, have yet to be explored. We conducted a workshop study with 13 queer artists, during which we gave participants access to GPT-4 and DALL-E 3 and facilitated group sensemaking activities. We found our participants struggled to use these models due to various normative values embedded in their designs, such as hyper-positivity and anti-sexuality. We describe various strategies our participants developed to overcome these models' limitations and how, nevertheless, our participants found value in these highly-normative technologies. Drawing on queer feminist theory, we discuss implications for the conceptualization of "state-of-the-art" models and consider how FAccT researchers might support queer alternatives.

cs.HC↗

arXiv2025

Practicing Stress Relief for the Everyday: Designing Social Simulation Using VR, AR, and LLMs

Stress is an inevitable part of day-to-day life yet many find themselves unable to manage it themselves, particularly when professional or peer support are not always readily available. As self-care becomes increasingly vital for mental well-being, this paper explores the potential of social simulation as a safe, virtual environment for practicing stress relief for everyday situations. Leveraging the immersive capabilities of VR, AR, and LLMs, we developed eight interactive prototypes for various everyday stressful scenarios (e.g. public speaking) then conducted prototype-driven semi-structured interviews with 19 participants. We reveal that people currently lack effective means to support themselves through everyday stress and found that social simulation fills a gap for simulating real environments for training mental health practices. We outline key considerations for future development of simulation for self-care, including risks of trauma from hyper-realism, distrust of LLM-recommended timing for mental health recommendations, and the value of accessibility for self-care interventions.

cs.HC↗

arXiv2025

PolicyCraft: Supporting Collaborative and Participatory Policy Design through Case-Grounded Deliberation

Community and organizational policies are typically designed in a top-down, centralized fashion, with limited input from impacted stakeholders. This can result in policies that are misaligned with community needs or perceived as illegitimate. How can we support more collaborative, participatory approaches to policy design? In this paper, we present PolicyCraft, a system that structures collaborative policy design through case-grounded deliberation. Building on past research that highlights the value of concrete cases in establishing common ground, PolicyCraft supports users in collaboratively proposing, critiquing, and revising policies through discussion and voting on cases. A field study across two university courses showed that students using PolicyCraft reached greater consensus and developed better-supported course policies, compared with those using a baseline system that did not scaffold their use of concrete cases. Reflecting on our findings, we discuss opportunities for future HCI systems to help groups more effectively bridge between abstract policies and concrete cases.

cs.HC↗