Searcharxiv⌕ Search

subject

cs.HC

cs.HC: explore 317 source-linked works published from 2015 to 2026, with original documents and citations.

This collection is a preview while coverage and quality are evaluated.

Search within this collection

Coverage and selection

Includes records with this source-supplied label or an explicit phrase match in their metadata. Matches indicate a mention, not proof that a paper uses a method or tests a material. Source versions are consolidated by DOI.

Sources: arxiv. Collection updated 2026-09-16. Counts describe this index, not the complete source archives.

Orthogonal Ensembles and Tested Explanations for Performer-Independent Body-Motion Emotion Recognition

We study body-only, 12-class acted-emotion classification from skeleton motion under leave-performer-out (LPO) evaluation, a hard, underdetermined setting: chance is 8.3%, and a protocol-matched reproduced STGCN++ baseline reaches only 25.73 +/- 4.03% Macro-F1. We show that reliable gains come not from a new architecture but from combining eleven models with orthogonal error modes: under 10-fold LPO cross-validation on the labeled training performers, an equal-weight logit-mean ensemble reaches 36.80 +/- 4.00% per-fold Macro-F1, a protocol-matched +11.07 pp (+43% relative) over the same-split reproduced baseline. Our central contribution is a tested explanation suite: for a strong ensemble member, part-masking and counterfactual edits show (rather than assert) that its decisions depend on motion-grounded body-region evidence, and this region saliency aligns with rule-based Laban Movement Analysis (LMA) attributes far more than with classical kinematics: region-level saliency-LMA Spearman rho = +0.500 versus +0.033, roughly 15x, and the alignment holds for the submitted 11-way ensemble itself at rho = +0.517; the audit is post hoc and needs no retraining. The same suite faithfully reports a negative: within-window temporal saliency is diffuse rather than localized. On the hidden challenge test set the submitted ensemble scored 37.23 % Macro-F1 and received the Best Performance Award of the MMAC Challenge 2026 (Human score is 39 %). Code is available at https://github.com/nawta/diema-challenge and the presentation at https://nawta.github.io/mmac2026/.

cs.CV↗

Beauty is in the AI of the beholder: MLLMs systematically overrate facial attractiveness

Beauty assessments from Multimodal Large Language Models (MLLMs) are increasingly popular amongst users, companies, and aestheticians. This raises the question of whether these AI models can accurately reflect human judgments of attractiveness. In a pre- registered exploratory study, we compared the attractiveness ratings of 2,513 human participants to four widely used commercial AI models: Claude, Gemini, GPT, and Grok. Results showed that MLLMs systematically rate faces more favourably and within a narrower range than humans and, at the time of study, do not reproduce human ratings in absolute terms. However, MLLMs exhibit strong correlations with human attractiveness judgments, accurately tracking the rank-ordering of faces. MLLMs may judge faces by different cues than humans; only face age was a predictor of facial attractiveness in both humans and MLLMs, with inconsistent patterns across models for ethnicity and gender. AI models strongly agree with one another, except for Grok, which also showed the lowest agreement with humans. Our findings suggest that while they may be able to approximate rank-orderings of human attractiveness, current off-the-shelf commercial MLLMs systematically overrate the beauty of human faces.

cs.CV↗

EEG-based Visual Retrieval and Reconstruction: From Neurally Visible Optimal Layer to Hierarchical Diffusion Generation

Decoding visual perception from electroencephalography (EEG) is important for non-invasive brain-computer interfaces (BCIs). However, most existing visual decoding pipelines directly align EEG features with semantic features from pretrained vision models. Those EEG signals carry information at more than one level and this practice disregards the varying neural visibility of different visual components in EEG signals, leading to cross modal mismatches and incomplete information use. In this work, we address this limitation through layer-wise contrastive learning. For each subject, the intermediate CLIP layer that maximizes retrieval performance is selected as the Neural Visibility Optimal Layer (NVOL). Built on NVOL, a hierarchical framework couples retrieval and generation through a shared intermediate representation. The retrieval branch fuses multi-NVOL features, aligns them to image embeddings via contrastive learning, and applies cross-domain similarity local scaling (CSLS) at test time to mitigate hubness. The generation branch reconstructs subject-specific NVOL features from EEG using a conditional diffusion prior, maps them to CLIP space through a lightweight adapter, and drives a pretrained Stable Diffusion XL model. Experimental validation on THINGS-EEG showed that, NVOL-based retrieval achieves 78.1\% mean Top-1 accuracy in 200-way retrieval, rising to 86.4\% with CSLS. Two-stage NVOL-to-semantic reconstruction also outperforms single-stage final-layer diffusion on semantic and structural metrics. By aligning EEG with layer-wise neural visibility rather than fixed high-level semantics, the proposed framework improves both retrieval accuracy and image reconstruction in EEG-based visual decoding.

cs.HC↗

Beyond Problem Solving: Large Language Models for Emotional and Reflective Support in Mathematics Learning

Intelligent Tutoring Systems (ITSs) traditionally focus their adaptive support on cognitive aspects of learning. Although effective, little is known about how such systems can be enhanced by addressing students' emotional states. In particular, the role of mindful interventions for supporting student learning and experiences in adaptive math learning remains underexplored. We developed "Math with Matt", an ITS that leverages Large Language Models (LLMs) to provide both cognitive and emotional support in algebra learning. The system offers 1) an LLM-based mindful chat that delivers context-sensitive emotional support through a pedagogical agent Matt, and 2) mindful feedback and hint messages (not just evaluative) to enhance learning experiences and reduce math anxiety. We conducted a classroom study with 7th graders, comparing a Mindful version against a version with cognitive support only. Overall, the ITS reduced executive state-math anxiety and improved students' math learning, though no significant differences emerged between the conditions. However, students with the mindfulness interventions showed higher learning efficiency and well-balanced problem-solving behavior, since they achieve a similar level of math learning with less learning time and fewer requested hints compared to the Cognitive version. Additionally, they reported that the pedagogical agent felt more supportive and caring than students in the cognitive condition. Our study demonstrates the feasibility and scalability of integrating mindfulness into ITSs through LLM-based interactions and positions LLMs as an adaptive, socio-emotional layer within cognitive math tutoring.

cs.HC↗

The PIONEER Project: A PrIvacy companion for mOtivatioN and knowlEdge transfER

Remaining control over their private data is one of the key challenges in this century for users. We know from prior work that users are often neither in a position to fully grasp the content of the usually complicated texts, nor are they motivated to spend the time necessary to do so. We report on the progress made by the PIONEER project on a privacy support tool that combines knowledge transfer and persuasive elements to increase users' privacy awareness and motivation; thus empowering them to more privacy sovereignty. Throughout the research and design process, we consider user group specifics that may result in different requirements, e.g., for children, adolescents, parents, or elderly people. We further target sustainable behavior change by addressing different states of change, precisely: spark initial motivation, facilitate the creation of new habits, and encourage habituation of these habits in the long term (volition). Finally, we provide a privacy support tool demonstrator that can be utilized for research and education purposes, e.g., in school contexts.

cs.HC↗

BuildOcc: A Large Language Model Occupant Agent Platform for Building Energy Research

Occupants are a primary source of uncertainty in building energy consumption and management, yet existing occupant behavior models cannot capture adaptive and reasoning responses considering the occupant's personal history, current context, and the type of energy signal being delivered. This study presents BuildOcc, an open-source Python platform that grounds large language model agents in the American Time Use Survey (ATUS), a nationally representative diary dataset covering 16,684 respondents. Through BuildOcc, each simulated occupant agent can be instantiated with a demographic persona drawn from ATUS population statistics, a memory stream that accumulates and reflects on timestep-level observations, and an activity scheduler that samples empirically from ATUS time-at-activity distributions. The platform exposes a three-layer interface - Python library, REST API, and Model Context Protocol server - so that any building energy tool (EnergyPlus, Home Assistant) can integrate behavioral intelligence without bespoke coupling code. A plugin registry lets the community add new occupant strata, custom schedulers, and alternative memory backends as separate installable packages. Two validation tiers show that ATUS-grounded sampling reproduces empirically calibrated activity distributions and that demographic priors propagate into persona-consistent agent reasoning across timesteps, establishing internal consistency across strata. BuildOcc provides the building energy community with a reusable, openly available implementation of the occupant behavioral layer. BuildOcc is openly released at https://doi.org/10.5281/zenodo.21192895 under the Apache License 2.0 and installable via pip install buildocc.

cs.HC↗

Beyond Blur: A Semantic Tri-view Pipeline for Teledermatology Gradability via Skin Micro-relief

Smartphone skin photographs are indispensable to teledermatology, yet assessing the diagnostic suitability of submitted cases (gradability) remains a critical bottleneck in mobile care workflows. Dermatologists routinely review multiple photographic views (regional, angled, and close-up) to identify consistent textural detail rather than relying on a single image. We present the Semantic Tri-view Pipeline, an interpretable architecture for automated teledermatology gradability screening that formalizes epidermal micro-relief as a computable biomarker of image quality. Using an expert-annotated subset of the public SCIN dataset, we train a lightweight DeepLabV3+ model to segment micro-relief fidelity. These spatial masks are then aggregated across up to three case views with a logistic regression classifier, leveraging viewpoint redundancy to support robustness under uncontrolled smartphone acquisition. This approach learns context-aware, clinically intelligible heuristics, such as penalizing high-fidelity texture in regional distance views. Evaluated at a predefined 90% sensitivity operating point, the system's apparent errors largely reflect subjective clinical variance on borderline cases where clinicians rely on non-visual metadata. On SCIN, performance improves from an AUC of 0.81 (80.6% PPV) on variance-heavy majority-consensus cases to 0.96 (97.7% PPV) on optically unambiguous unanimous cases. Overall, this work delivers an interpretable, privacy-by-design, edge-ready system that can provide real-time feedback during case submission to filter ungradable photo sets before review.

eess.IV↗

Signal-Driven Pervasive Game Design: The LifeSync-Games Framework as a Player Experience Integration Layer

Pervasive games extend the magic circle across spatial, temporal, and social dimensions, yet treat the player's physiological and cognitive state as a passive receptor rather than an active signal. This paper presents LifeSync-Games (LSG), a framework that (unlike proposals treating player signals as an additional dimension), operationalizes them as a Player Experience Integration Layer (PEIL) acting transversally across the three existing pervasive dimensions through verified real-world signals: physical activity, sleep quality, memory, and decision speed. The framework introduces a gamified integration artifact (the LSG portal) that mediates the real <-> virtual exchange through redeemable points, real-world missions, and structural gamification. Five HCI design principles grounded in Self-Determination Theory and Flow Theory are proposed, instantiated across six commercial video games, together with a study protocol (n = 70-80 participants, quasi-experimental design). This paper reports the design stage of LSG: rule thresholds and portal parameters are design decisions pending empirical calibration; no data collection has yet been conducted. The main contribution is a theoretically grounded framework and validation protocol positioning player-sensitive integration as the mechanism enabling pervasive games to respond to the player's actual biological and cognitive state.

cs.HC↗

Do Depressive Facial Patterns Transfer Across Cultures and Contexts? Evidence from a German RCT and E-DAIC

Automated assessment of depression from facial dynamics holds promise for scalable mental health monitoring, yet cross-corpus generalization of learned biomarkers remains an open challenge. We present a systematic bidirectional transfer study pairing the EmpkinS-EKSpression randomized controlled trial (RCT; N = 256, SCID-5-CV diagnoses) with the Extended Distress Analysis Interview Corpus (E-DAIC; N = 275, semi-structured clinical interviews), predicting depression severity and binary diagnostic status from facial action units, head pose, and gaze. Cross-corpus binary classification proves more robust than continuous PHQ-8 severity regression, with forward transfer achieving AUC = 0.70. Regression transfer is governed by functional context alignment: passive observation phases yield the most transferable models, while active emotion regulation phases elicit stronger within-corpus signals. These findings establish functional context alignment as the primary determinant of cross-corpus generalization, with passive elicitation contexts offering the best trade-off between within-corpus sensitivity and cross-corpus robustness.

cs.CV↗

Modeling and Optimizing User Preferences in AI Copilots: A Comprehensive Survey and Taxonomy

AI copilots represent a new generation of AI-powered systems designed to assist users, particularly knowledge workers and developers, in complex, context-rich tasks. As these systems become more embedded in daily workflows, personalization has emerged as a critical factor for improving usability, effectiveness, and user satisfaction. Central to this personalization is preference optimization: the system's ability to detect, interpret, and align with individual user preferences. While prior work in intelligent assistants and optimization algorithms is extensive, their intersection within AI copilots remains underexplored. This survey addresses that gap by examining how user preferences are operationalized in AI copilots. We investigate how preference signals are sourced, modeled across different interaction stages, and refined through feedback loops. Building on a comprehensive literature review, we define the concept of an AI copilot and introduce a taxonomy of preference optimization techniques across pre-, mid-, and post-interaction phases. Each technique is evaluated in terms of advantages, limitations, and design implications. By consolidating fragmented efforts across AI personalization, human-AI interaction, and language model adaptation, this work offers both a unified conceptual foundation and a practical design perspective for building user-aligned, persona-aware AI copilots that support end-to-end adaptability and deployment.

cs.AI↗

Unsupervised Partner Design Enables Robust Ad-hoc Teamwork

We introduce Unsupervised Partner Design (UPD), a population-free multi-agent reinforcement learning method for robust ad-hoc teamwork. UPD generates training partners on-the-fly and selects them adaptively based on a learnability criterion, removing the need for pre-trained partner populations or manual parameter tuning. We show that this simple mechanism enables effective partner diversity and can be extended to joint partner-environment selection when a procedural level generator is available. Across Level-Based Foraging, Overcooked-AI, and the Overcooked Generalisation Challenge, UPD consistently achieves strong performance compared to both population-based and population-free baselines. In a human-AI user study, agents trained with UPD achieve higher returns and are rated as more adaptive, more human-like, and less frustrating than all evaluated baseline methods.

cs.LG↗

Fact-Checking Comments Precede Community Notes to Misleading Posts on X

Community-based fact-checking systems, such as X's Community Notes program, offer a potentially scalable approach against online misinformation diffusion. However, their efficacy is substantially undermined by the delay of fact-check delivery. To address this challenge, we investigated the promise of organic user comments as rapid corrective signals by analyzing a large-scale dataset of over 2.2 million comments directed to 1,841 community fact-checked misleading posts. We developed a high-performance language model pipeline to identify ``fact-checking (FC) comments'', i.e., comments that correct source posts with reasoning or evidence (91% accuracy). Using this pipeline, we find that 99.4% of misleading posts receive their first FC comments before the official community note is created. Notably, the median time to the initial FC comment is only 0.1 hours, while the creation of community notes has a median latency of 9.7 hours since the publication of misleading posts. Additionally, compared to random posts, those misleading posts with displayed notes have significantly more FC comments. This suggests that FC comments are a unique pattern for misleading posts. We further identified characteristics of FC comments' intensity and speed: (i) FC comments' volume is significantly associated with content richness and misinformation type, favoring multimedia content and missing-context scenarios; (ii) the speed of FC comments remains robust across diverse topics and emotional intensities. These results suggest that platforms could use FC comments as early-warning signals, and potential resource for complementing community notes or automated synthesis.

cs.HC↗

Toward Interaction Dynamics: A Predictive Framework for Safe Physical Human Robot Interaction

Physical human-robot interaction requires yielding transiently to contact yet recovering the commanded reference under sustained load. Finite-stiffness impedance control retains a static deflection there, while predictive alternatives typically optimize a nonlinear robot or impedance model online. Operational-space cancellation instead exposes a translational error double integrator with a fixed transition matrix and a configuration-scheduled input map, making interaction a predictive quantity rather than a property re-derived per configuration. We build on it a compact offset-free interaction-error MPC for torque-controlled manipulators: a force-domain random-walk state estimates persistent interaction and model error, and a 30-variable convex QP maps the correction through the current task inertia while constraining the applied joint torque. Conditional results establish impedance equivalence of the unconstrained passive feedback, offset-free regulation at feasible frozen configurations, and quadratic stabilizability of the scheduled backbone. In a 1kHz MuJoCo simulation of a 7-DOF Franka FR3, the estimator cuts steady-state error under a repeated 15N step from 2.77mm to 0.042mm when added to the otherwise identical 100Hz MPC. A stiffness-and-damping-calibrated impedance baseline attains 2.59mm but briefly saturates and needs 3.3x the peak positive joint power. Adding ideal measured-force cancellation to that baseline gives 1.39mm, so constant-load rejection is not unique to MPC; the sensorless controller still reaches 0.042mm in the moving task, a 65x reduction without force sensing and without the baseline's saturation or power cost. Demonstrated in simulation under a shared actuator budget, the contribution is an efficient operational-space realization complementing rather than replacing broader interaction-control architectures.

cs.RO↗

Self-EmoQ: Plutchik-Guided Value-based Planning to Drive Streaming Emotional TTS

Emotional interaction is increasingly crucial for conversational AI, yet current systems lack a self-emotion determination mechanism to drive the streaming text-to-speech (TTS) synthesis. We propose an emotion-planning framework that determines the emotion prior to the textual generation, grounding the downstream emotional TTS in a streaming manner. The framework is implemented by a plug-and-play LLM module, initialized from pretrained LLMs, and trained by reinforcement learning (RL) with emotions as the actions. A hybrid reward is employed which combines imitation signals with theory-driven scoring, in which the theory of Plutchik's wheel of emotions is adopted. By experiments on DailyDialog, EmoryNLP, IMEOCAP, and MELD, our method outperforms prompting and finetuning baselines on both emotion determination and response quality. We finally implement an entire streaming pipeline for real-time deployment, with the speech quality confirming the framework's emotional alignment, contextual coherence, and expressive fluency. Codes, cases, and demos are available in https://sixingdeguo.github.io/EmoQ-page/.

cs.HC↗

DigitalCoach: Communication and Grounding Gaps in Human and Agentic Computer Use Coaching

Agents are increasingly capable of automating software tasks, but can they teach humans how to use software themselves? We introduce DigitalCoach, a multimodal dataset of 72 human expert-novice computer use coaching sessions consisting of 22,752 dialogue turns grounded in 28.1 hours of screen and input event recordings across five software applications. We use DigitalCoach to evaluate whether state-of-the-art models can teach humans how to use computers. Automated evaluation shows that models differ from humans in how they coach: models provide more direct instructions, but fewer explanations, error diagnoses, and knowledge-check questions. When we fix the coaching method, models produce utterances similar to human references yet poorly grounded in visual context. Interactive evaluation confirms that model coaches cause learners to passively follow instructions without deeper engagement and fall short in visual grounding. DigitalCoach lays a foundation for collaborative and proactive computer use coaching agents. Data and code are available at https://project-digital-coach.vercel.app.

cs.CL↗

Deceptive Patterns as a Sociotechnical Phenomenon: Review, Catalog, and Discussion

Background: Deceptive patterns are interface design strategies aimed at misleading users or favoring specific interests, compromising user experiences and ethical privacy principles. These patterns involve exploit different issues of the interaction between humans and technology, revealing gaps in the scientific literature regarding their understanding and working mechanisms. Purpose: This research characterizes deceptive patterns as a sociotechnical phenomenon, integrating human and technical dimensions while offering a self-explainable interactive catalog to raise awareness among users and designers. Methods: To characterize deceptive patterns as a sociotechnical phenomenon, a systematic literature review in Computer Science was conducted, and the Semiotic Framework was applied to analyze and organize the sociotechnical aspects of identified patterns in an integrated manner. An exploratory evaluation of the interactive catalog was conducted, combining heuristic evaluation and focus group. Results: The research identified multiple deceptive patterns in scientific literature. Analysis revealed that existing studies frequently addressed specific aspects of patterns, such as typologies or effects, and do not focus on sociotechnical issues. The exploratory evaluation suggested that the self-explainable catalog is useful and easy to use, with potential to inform users and promote awareness regarding the deceptive patterns existence and functioning. Conclusion: The study contributes to an informed understanding of deceptive patterns, highlighting the need for an approach that integrates human and technical dimensions. The self-explainable catalog is a promising tool to inform and promote awareness about the topic. By exposing the risks and mechanisms of these patterns, the research seeks to promote awareness among users and designers towards ethical interface design practices.

cs.HC↗

DOBI: Dynamic Opportunistic Body Input via Spare Joint Recruitment for Hands-Free XR

Extended Reality (XR) systems are often most useful when users are engaged in ongoing physical tasks, yet current interaction techniques still largely assume the hands are available. We present opportunistic body input, an interaction paradigm that redirects continuous XR control to whichever available body region remains free in the moment. To investigate how users naturally coordinate these spare-body movements, we conducted an elicitation study across six hand-busy scenarios. We found that while users' preferred spare body regions shift dynamically based on physical constraints, the resulting spontaneous movements share a consistent, low-dimensional kinematic structure organized around a dominant principal axis. Building on these findings, we present DOBI (Dynamic Opportunistic Body Input), a real-time XR technique that uses gaze to target a UI element, a brief trigger gesture to identify the recruited spare body region, and the region's subsequent motion to drive continuous 1D control. A 1D Fitts' law study establishes the baseline motor performance of this paradigm across four distinct body regions, achieving throughputs up to 2.62 bits/s with an overall 5.0% error rate, and a dual-task usability study shows that DOBI supports reliable, low-effort control (SUS = 84.2) while users remain engaged in realistic hand-busy activities.

cs.HC↗

UniScale: Exploring Unimanual Gesture Mapping Strategies for Gaze+Pinch-based Scaling Interaction

Object scaling serves as a fundamental spatial manipulation that enables complex and productive tasks in XR environments. This paper investigates unimanual scaling techniques for XR using gaze and hand interactions. We propose UniScale, a set of unimanual alternatives to the standard bimanual pinch, allowing users to scale objects while preserving hand availability for concurrent spatial manipulations. We design five distinct mapping strategies based on physical metaphors, exploring unimanual control that varies depth, angle, micro-gestures, and finger-distance input. We then compare these techniques against a standard bimanual baseline, in which users adjust the inter-hand distance via a bimanual pinch gesture. In a user study, we evaluate their effectiveness in a 3D object scaling task under both clutching and clutching-free conditions. The results indicate that while bimanual scaling relies on clutching for stable control, unimanual techniques excel in clutching-free conditions, significantly reducing physical hand movement. From the results, we derive valuable design implications for developing efficient 3D multimodal interactions in XR.

cs.HC↗
Compare source metadata on this page
WorkPublishedSource identifierSource
Orthogonal Ensembles and Tested Explanations for Performer-Independent Body-Motion Emotion Recognition2026-09-152609.02510arxiv
Beauty is in the AI of the beholder: MLLMs systematically overrate facial attractiveness2026-09-022609.02512arxiv
EEG-based Visual Retrieval and Reconstruction: From Neurally Visible Optimal Layer to Hierarchical Diffusion Generation2026-09-022609.02582arxiv
Beyond Problem Solving: Large Language Models for Emotional and Reflective Support in Mathematics Learning2026-09-022609.02611arxiv
The PIONEER Project: A PrIvacy companion for mOtivatioN and knowlEdge transfER2026-09-022609.02700arxiv
BuildOcc: A Large Language Model Occupant Agent Platform for Building Energy Research2026-09-252609.02729arxiv
Beyond Blur: A Semantic Tri-view Pipeline for Teledermatology Gradability via Skin Micro-relief2026-09-022609.03095arxiv
Signal-Driven Pervasive Game Design: The LifeSync-Games Framework as a Player Experience Integration Layer2026-09-022609.03169arxiv
Do Depressive Facial Patterns Transfer Across Cultures and Contexts? Evidence from a German RCT and E-DAIC2026-09-022609.05543arxiv
Modeling and Optimizing User Preferences in AI Copilots: A Comprehensive Survey and Taxonomy2026-09-012505.21907arxiv
Unsupervised Partner Design Enables Robust Ad-hoc Teamwork2026-09-012508.06336arxiv
Fact-Checking Comments Precede Community Notes to Misleading Posts on X2026-09-012509.11052arxiv
Toward Interaction Dynamics: A Predictive Framework for Safe Physical Human Robot Interaction2026-09-012606.08281arxiv
Self-EmoQ: Plutchik-Guided Value-based Planning to Drive Streaming Emotional TTS2026-09-012606.09837arxiv
DigitalCoach: Communication and Grounding Gaps in Human and Agentic Computer Use Coaching2026-09-012606.31980arxiv
Deceptive Patterns as a Sociotechnical Phenomenon: Review, Catalog, and Discussion2026-09-012608.27684arxiv
DOBI: Dynamic Opportunistic Body Input via Spare Joint Recruitment for Hands-Free XR2026-09-012608.30341arxiv
UniScale: Exploring Unimanual Gesture Mapping Strategies for Gaze+Pinch-based Scaling Interaction2026-09-012609.00500arxiv

These are bibliographic comparisons, not experimental rankings. Follow the original document for methods and conditions.