arXiv subjects

Siying Hu

Publications and source records attributed to Siying Hu.

At least 19 recordsLinked to original sources

arXiv2026

You Really Didn't Get That? Benchmarking Social Pragmatic Inference for Indirect and Playful Chinese Online Comments

Chinese online comments often convey social meaning through indirect and playful language that is hard to interpret without context. Existing evaluations largely organize items around predefined phenomena or controlled pragmatic categories, leaving open whether models can distinguish plausible readings of what a naturally occurring comment is doing in a particular exchange. We introduce a benchmark for evaluating whether LLMs can recover such situated pragmatic meanings. From more than 200,000 public Chinese social media interaction records, we construct 4,735 human-validated diagnostic items, each pairing a target comment with reconstructed preceding context and plausible misreadings. We evaluate eight LLMs as both question writers and solvers in a cross-writer setting. The task is challenging: the strongest model achieves 81.42% leave-writer-out accuracy. Across all eight models, the mean leave-writer-out accuracy is 68.70% while human accuracy was 90.8%. Case analysis shows that models often recognize broad irony or playfulness while misidentifying the mechanism or interactional move.

cs.CL↗

arXiv2026

AuraDesk: Data Physicalization through Olfaction Metaphors for Representing and Mitigating Workplace Stress

Workplace stress is often addressed through visual or auditory interventions, yet these modalities can compete with attention and contribute to sensory overload. We explore olfaction as an alternative ambient medium for representing stress-related physiological signals in office settings. We present AuraDesk, an olfactory data physicalization system that translates wearable-derived physiological cues into situated scent expressions at the workstation. The system combines local physiological state inference with a constrained actuation strategy to produce temporally regulated and spatially localized scent output suitable for everyday work environments. To examine the feasibility and experiential qualities of this approach, we conducted a one-day in-situ field deployment with 25 knowledge workers at their actual workstations. Our findings show that participants often interpreted the scent output not as an explicit alert, but as a subtle atmospheric cue that supported momentary awareness, micro-break taking, and perceived environmental attunement. At the same time, participants raised important concerns regarding scent preference, habituation, and contextual appropriateness in shared offices. This work contributes (1) an olfactory interface for physiologically driven ambient feedback in the workplace, (2) a hybrid mapping approach for coupling continuous biosignal interpretation with constrained scent actuation, and (3) empirical insights into how workers perceive, negotiate, and appropriate ambient olfactory feedback in real office contexts. Rather than claiming therapeutic efficacy, we position AuraDesk as a probe into the design space of olfactory data physicalization for workplace wellbeing and attention-sensitive interaction.

cs.HC↗

arXiv2026

PuppetChat: Fostering Intimate Communication through Bidirectional Actions and Micronarratives

As a primary channel for sustaining modern intimate relationships, instant messaging facilitates frequent connection across distances. However, today's tools often dilute care; they favor single tap reactions and vague emojis that do not support two way action responses, do not preserve the feeling that the exchange keeps going without breaking, and are weakly tied to who we are and what we share. To address this challenge, we present PuppetChat, a dyadic messaging prototype that restores this expressive depth through embodied interaction. PuppetChat uses a reciprocity aware recommender to encourage responsive actions and generates personalized micronarratives from user stories to ground interactions in personal history. Our 10-day field study with 11 dyads of close partners or friends revealed that this approach enhanced social presence, supported more expressive self disclosure, and sustained continuity and shared memories.

cs.HC↗

arXiv2026

ImmuniFraug: A Metacognitive Intervention Anti-Fraud Approach to Enhance Undergraduate Students' Cyber Fraud Awareness

Cyber fraud now constitutes over half of criminal cases in China, with undergraduate students experiencing a disproportionate rise in victimization. Traditional anti-fraud training remains predominantly passive, yielding limited engagement and retention. This paper introduces ImmuniFraug, a Large Language Model (LLM)-based metacognitive intervention that delivers immersive, multimodal fraud simulations integrating text, voice, and visual avatars across ten prevalent fraud types. Each scenario is designed to replicate real-world persuasion tactics and psychological pressure, while post-interaction debriefs provide grounded feedback in protection motivation theory and reflective prompts to reinforce learning. In a controlled study with 846 Chinese undergraduates, ImmuniFraug was compared to official text-based materials. Linear Mixed-Effects Modeling (LMEM) reveals that the interactive intervention significantly improved fraud awareness (p = 0.026), successfully providing incremental learning value even when controlling for participants' extensive prior exposure to anti-fraud education, alongside high narrative immersion (M = 56.95/77). Thematic analysis of interviews revealed key effectiveness factors: perceived realism, adaptive deception, enforced time pressure, emotional manipulation awareness, and enhanced self-efficacy. Findings demonstrate that by shifting the focus from passive knowledge acquisition to active metacognitive engagement, LLM-based simulations offer a scalable and ecologically valid new paradigm for anti-fraud training and fostering fraud resilience.

cs.HC↗

arXiv2026

The Anatomy of Conversational Scams: A Topic-Based Red Teaming Analysis of Multi-Turn Interactions in LLMs

As LLMs gain persuasive capabilities through extended dialogues, they create new opportunities for studying adversarial conversational behavior in extended interaction settings that traditional single-turn safety evaluations fail to capture. We systematically study these interactional dynamics using a controlled LLM-to-LLM simulation framework for automated red-teaming across bilingual social engineering scenarios. Evaluating eight state-of-the-art models in English and Chinese, we analyze dialogue-level outcomes, annotate attacker and defender strategy families, and model interaction dynamics between them. Results show that multi-turn adversarial dialogues follow recurrent escalation patterns, while defensive responses frequently rely on verification, delay, and channel control. We further find statistically significant cross-model and cross-lingual differences in outcome distributions, and transition analysis reveals systematic structural variation in how defender strategies respond to attacker tactics across languages. These findings highlight the importance of studying interactional structure in multi-turn adversarial dialogue settings and demonstrate how controlled LLM-to-LLM simulations can support mechanistic analysis of adversarial conversational dynamics.

cs.CL↗

arXiv2025

Towards Human-Centered RegTech: Unpacking Professionals' Strategies and Needs for Using LLMs Safely

Large Language Models are profoundly changing work patterns in high-risk professional domains, yet their application also introduces severe and underexplored compliance risks. To investigate this issue, we conducted semi-structured interviews with 24 highly-skilled knowledge workers from industries such as law, healthcare, and finance. The study found that these experts are commonly concerned about sensitive information leakage, intellectual property infringement, and uncertainty regarding the quality of model outputs. In response, they spontaneously adopt various mitigation strategies, such as actively distorting input data and limiting the details in their prompts. However, the effectiveness of these spontaneous efforts is limited due to a lack of specific compliance guidance and training for Large Language Models. Our research reveals a significant gap between current NLP tools and the actual compliance needs of experts. This paper positions these valuable empirical findings as foundational work for building the next generation of Human-Centered, Compliance-Driven Natural Language Processing for Regulatory Technology (RegTech), providing a critical human-centered perspective and design requirements for engineering NLP systems that can proactively support expert compliance workflows.

cs.HC↗

arXiv2025

Vocabuild: An Accessible Augmented Tangible Interface for Gamified Vocabulary Learning of Constructing Meaning

Vocabulary acquisition in early education often relies on rote memorization and passive screen-based tools, which can fail to engage students kinesthetically and collaboratively. This paper introduces Vocabuild, an augmented tangible interface designed to transform vocabulary learning into an active, embodied, and playful experience. The system combines physical letter blocks with a projection-augmented surface. As children physically construct words with the blocks, the system provides real-time, dynamic feedback, such as displaying corresponding images and animations, thus helping them construct semantic meaning. Deployed in a classroom context, our gamified approach fosters both individual exploration and peer collaboration. A user study conducted with elementary school children demonstrates that our tangible interface leads to higher engagement, increased collaboration, and a more positive attitude towards learning compared to traditional methods. Our contributions are twofold: (1) the design and implementation of Vocabuild, a projection-augmented tangible system that transforms vocabulary learning into an embodied and collaborative activity; and (2) empirical findings from a classroom study showing that our tangible approach significantly increases engagement, peer collaboration, and positive learning attitudes compared to traditional methods.

cs.HC↗

arXiv2025

Living with Data: Exploring Physicalization Approaches to Sedentary Behavior Intervention for Older Adults in Everyday Life

Sedentary behavior is a critical health risk for older adults. Although digital interventions are widely available, they primarily rely on screen-based notifications that can feel clinical or cognitively demanding, and are thus often ignored over time. This paper presents a three-phase Research through Design methodology to explore data physicalization approaches that ambiently represent sedentary data patterns using decor artifacts in older adults' homes. These artifacts transformed abstract data into aesthetic, evolving forms that became part of the domestic landscape. Our research revealed how these physicalizations fostered self-reflection, family conversations, and encouraged active lifestyles. We demonstrate how qualities like aesthetic ambiguity and slow revelation can empower older adults, fostering a reflective relationship with their well-being. Ultimately, we argue that creating data physicalizations for older adults necessitates a shift from merely informing users to enabling them to live with and through their data.

cs.HC↗

arXiv2025

A Day in Their Shoes: Using LLM-Based Perspective-Taking Interactive Fiction to Reduce Stigma Toward Dirty Work

Occupations referred to as "dirty work" often face entrenched social stigma, which adversely affects the mental health of workers in these fields and impedes occupational equity. In this study, we propose a novel Interactive Fiction (IF) framework powered by Large Language Models (LLMs) to encourage perspective-taking and reduce biases against these stigmatized yet essential roles. Through an experiment with participants (n = 100) across four such occupations, we observed a significant increase in participants' understanding of these occupations, as well as a high level of empathy and a strong sense of connection to individuals in these roles. Additionally, qualitative interviews with participants (n = 15) revealed that the LLM-based perspective-taking IF enhanced immersion, deepened emotional resonance and empathy toward "dirty work," and allowed participants to experience a sense of professional fulfillment in these occupations. However, participants also highlighted ongoing challenges, such as limited contextual details generated by the LLM and the unintentional reinforcement of existing stereotypes. Overall, our findings underscore that an LLM-based perspective-taking IF framework offers a promising and scalable strategy for mitigating stigma and promoting social equity in marginalized professions.

cs.HC↗

arXiv2025

SpeechCap: Leveraging Playful Impact Captions to Facilitate Interpersonal Communication in Social Virtual Reality

Social Virtual Reality (VR) emerges as a promising platform bringing immersive, interactive, and engaging mechanisms for collaborative activities in virtual spaces. However, interpersonal communication in social VR is still limited with existing mediums and channels. To bridge the gap, we propose a novel method for mediating real-time conversation in social VR, which uses impact captions, a type of typographic visual effect widely used in videos, to convey both verbal and non-verbal information. We first investigated the design space of impact captions by content analysis and a co-design session with four experts. Next, we implemented SpeechCap as a proof-of-concept system, with which users can communicate with each other using speech-driven impact captions in VR. Through a user study (n=14), we evaluated the effectiveness of the visual and interaction design of impact captions, highlighting the interactivity and the integration of verbal and non-verbal information in communication mediums. Finally, we discussed topics of visual rhetoric, interactivity, and ambiguity as the main findings from the study, and further provided design implications for future work for facilitating interpersonal communication in social VR.

cs.HC↗

arXiv2024

Exploring the Danmaku Content Moderation on Video-Sharing Platforms: Existing Limitations, Challenges, and Design Opportunities

Video-sharing platforms (VSPs) have been increasingly embracing social features such as likes, comments, and Danmaku to boost user engagement. However, viewers may post inappropriate content through video commentary to gain attention or express themselves anonymously and even toxically. For example, on VSPs that support Danmaku, users may even intentionally create a "flood" of Danmaku with inappropriate content shown overlain on videos, disrupting the overall user experience. Despite of the prevalence of inappropriate Danmaku on these VSPs, there is a lack of understanding about the challenges and limitations of Danmaku content moderation on video-sharing platforms. To explore how users perceive the challenges and limitations of current Danmaku moderation methods on VSPs, we conducted probe-based interviews and co-design activities with 21 active end-users. Our findings reveal that the one-size-fits-all rules set by users or customizaibility moderation cannot accurately match the continuous Danmaku. Additionally, the moderation requirements of the Danmaku and the definition of offensive content must dynamically adjust to the video content. Non-intrusive methods should be used to maintain the coherence of the video browsing experience. Our findings inform the design of future Danmaku moderation tools on video-sharing platforms.

cs.HC↗

arXiv2024

"I Always Felt that SomethingWasWrong.": Understanding Compliance Risks and Mitigation Strategies when Highly-Skilled Compliance Knowledge Workers Use Large Language Models

The rapid advancement of Large Language Models (LLMs) has transformed knowledge-intensive has led to its widespread usage by knowledge workers to enhance their productivity. As these professionals handle sensitive information, and the training of text-based GenAI models involves the use of extensive data, there are thus concerns about privacy, security, and broader compliance with regulations and laws. While existing research has addressed privacy and security concerns, the specific compliance risks faced by highly-skilled knowledge workers when using the LLMs, and their mitigation strategies, remain underexplored. As understanding these risks and strategies is crucial for the development of industry-specific compliant LLM mechanisms, this research conducted semi-structured interviews with 24 knowledge workers from knowledge-intensive industries to understand their practices and experiences when integrating LLMs into their workflows. Our research explored how these workers ensure compliance and the resources and challenges they encounter when minimizing risks. Our preliminary findings showed that knowledge workers were concerned about the leakage of sensitive information and took proactive measures such as distorting input data and limiting prompt details to mitigate such risks. Their ability to identify and mitigate risks, however, was significantly hampered by a lack of LLM-specific compliance guidance and training. Our findings highlight the importance of improving knowledge workers' compliance awareness and establishing support systems and compliance cultures within organizations.

cs.HC↗

arXiv2024

Sportoonizer: Augmenting Sports Highlights' Narration and Visual Impact via Automatic Manga B-Roll Generation

Sports highlights are becoming increasingly popular on video-sharing platforms. Yet, crafting sport highlight videos is challenging, which requires producing engaging narratives from different angles, and conforming to different platform affordances with constantly changing audiences. Many content creators therefore create derivative work of the original sports video through manga styles to enhance its expressiveness. But manually creating and inserting tailored manga-style content can still be time-consuming. We introduce Sportoonizer, a system embedding the pipeline for automatic generation of manga-style animations for highlights in sports videos and insertion into original videos. It seamlessly merges dynamic manga sequences with live-action footage, enriching the visual tapestry and deepening narrative scope. By leveraging genAIs, Sportoonizer crafts compelling storylines encapsulating the intensity of sports moments and athletes' personal journeys. Our evaluation study demonstrates that integrating manga B-rolls significantly enhances viewer engagement, visual interest, and emotional connection towards athletes' stories in the viewing experience.

cs.HC↗

arXiv2024

AI vs. Human Paintings? Deciphering Public Interactions and Perceptions towards AI-Generated Paintings on TikTok

With the development of generative AI technology, a vast array of AI-generated paintings (AIGP) have gone viral on social media like TikTok. However, some negative news about AIGP has also emerged. For example, in 2022, numerous painters worldwide organized a large-scale anti-AI movement because of the infringement in generative AI model training. This event reflected a social issue that, with the development and application of generative AI, public feedback and feelings towards it may have been overlooked. Therefore, to investigate public interactions and perceptions towards AIGP on social media, we analyzed user engagement level and comment sentiment scores of AIGP using human painting videos as a baseline. In analyzing user engagement, we also considered the possible moderating effect of the aesthetic quality of Paintings. Utilizing topic modeling, we identified seven reasons, including hyperrealistic quality, ambivalent reactions, perceived theft of art, etc., leading to negative public perceptions of AIGP. Our work may provide instructive suggestions for future generative AI technology development and avoid potential crises in human-AI collaboration.

cs.HC↗

arXiv2024

KoroT-3E: A Personalized Musical Mnemonics Tool for Enhancing Memory Retention of Complex Computer Science Concepts

As the demand for computer science (CS) skills grows, mastering foundational concepts is crucial yet challenging for novice learners. To address this challenge, we present KoroT-3E, an AI-based system that creates personalized musical mnemonics to enhance both memory retention and understanding of concepts in CS. KoroT-3E enables users to transform complex concepts into memorable lyrics and compose melodies that suit their musical preferences. We conducted semi-structured interviews (n=12) to investigate why novice learners find it challenging to memorize and understand CS concepts. The findings, combined with constructivist learning theory, established our initial design, which was then refined following consultations with CS education experts. An empirical experiment(n=36) showed that those using KoroT-3E (n=18) significantly outperformed the control group (n=18), with improved memory efficiency, increased motivation, and a positive learning experience. These findings demonstrate the effectiveness of integrating multimodal generative AI into CS education to create personalized and interactive learning experiences.

cs.HC↗

arXiv2024

DanModCap: Designing a Danmaku Moderation Tool for Video-Sharing Platforms that Leverages Impact Captions with Large Language Models

Online video platforms have gained increased popularity due to their ability to support information consumption and sharing and the diverse social interactions they afford. Danmaku, a real-time commentary feature that overlays user comments on a video, has been found to improve user engagement, however, the use of Danmaku can lead to toxic behaviors and inappropriate comments. To address these issues, we propose a proactive moderation approach inspired by Impact Captions, a visual technique used in East Asian variety shows. Impact Captions combine textual content and visual elements to construct emotional and cognitive resonance. Within the context of this work, Impact Captions were used to guide viewers towards positive Danmaku-related activities and elicit more pro-social behaviors. Leveraging Impact Captions, we developed DanModCap, an moderation tool that collected and analyzed Danmaku and used it as input to large generative language models to produce Impact Captions. Our evaluation of DanModCap demonstrated that Impact Captions reduced negative antagonistic emotions, increased users' desire to share positive content, and elicited self-control in Danmaku social action to fostering proactive community maintenance behaviors. Our approach highlights the benefits of using LLM-supported content moderation methods for proactive moderation in a large-scale live content contexts.

cs.HC↗

arXiv2024

The Odyssey Journey: Top-Tier Medical Resource Seeking for Specialized Disorder in China

It is pivotal for patients to receive accurate health information, diagnoses, and timely treatments. However, in China, the significant imbalanced doctor-to-patient ratio intensifies the information and power asymmetries in doctor-patient relationships. Health information-seeking, which enables patients to collect information from sources beyond doctors, is a potential approach to mitigate these asymmetries. While HCI research predominantly focuses on common chronic conditions, our study focuses on specialized disorders, which are often familiar to specialists but not to general practitioners and the public. With Hemifacial Spasm (HFS) as an example, we aim to understand patients' health information and top-tier medical resource seeking journeys in China. Through interviews with three neurosurgeons and 12 HFS patients from rural and urban areas, and applying Actor-Network Theory, we provide empirical insights into the roles, interactions, and workflows of various actors in the health information-seeking network. We also identified five strategies patients adopted to mitigate asymmetries and access top-tier medical resources, illustrating these strategies as subnetworks within the broader health information-seeking network and outlining their advantages and challenges.

cs.HC↗

arXiv2024

Typist Experiment: an Investigation of Human-to-Human Dictation via Role-play to Inform Voice-based Text Authoring

Voice dictation is increasingly used for text entry, especially in mobile scenarios. However, the speech-based experience gets disrupted when users must go back to a screen and keyboard to review and edit the text. While existing dictation systems focus on improving transcription and error correction, little is known about how to support speech input for the entire text creation process, including composition, reviewing and editing. We conducted an experiment in which ten pairs of participants took on the roles of authors and typists to work on a text authoring task. By analysing the natural language patterns of both authors and typists, we identified new challenges and opportunities for the design of future dictation interfaces, including the ambiguity of human dictation, the differences between audio-only and with screen, and various passive and active assistance that can potentially be provided by future systems.

cs.HC↗