arXiv subjects

Zhanna Sarsenbayeva

Publications and source records attributed to Zhanna Sarsenbayeva.

18 recordsLinked to original sources

arXiv2026

Graphing the Everyday: A Neurosymbolic Approach to Eliciting Routines for Just-In-Time Adaptive Interventions

Just-In-Time Adaptive Interventions (JITAIs) increasingly rely on conversational agents to elicit user routines, yet translating fluid human dialogue into rigid schedule data remains a significant challenge. We conducted a qualitative investigation of a neurosymbolic pipeline, combining Large Language Models (LLMs) with a Neo4j knowledge graph, to map unstructured verbal narratives into actionable interventions. Through human-centric evaluation using natural-language playbacks, we identified a critical "mental-model gap," where the linear extraction of LLMs clashes with hierarchical, non-linear human storytelling, causing severe entity fragmentation. Furthermore, we articulate an "ecological mismatch," demonstrating that algorithmic schedule availability frequently ignores the user's fluctuating psychological receptivity and physical energy levels. To resolve these tensions, we propose actionable design heuristics, including routine piggybacking, adaptive negotiation, and scalable transparency. Ultimately, these guidelines provide a foundational framework for evolving rigid schedule-trackers into empathetic, context-aware proactive agents capable of supporting long-term health behavior change.

cs.HC↗

arXiv2026

CoGen3D: An Agentic Human-AI Co-Design Pipeline for 3D Asset Generation for Virtual Reality

Creating 3D assets for virtual reality requires modeling expertise, which restricts the authorship of immersive experiences. Existing generative AI tools rely on unconstrained, command-driven prompting, lacking the conversational scaffolding needed for users to articulate their intent and validate designs prior to rendering. To address this, we introduce CoGen3D, an agentic human-AI co-design pipeline that proactively guides users through conversational intent elicitation, a concept image confirmation, and image-to-3D generation that directly deploys to immersive scenes. We evaluated this system through a user study (N=120) across six affectively diverse immersive scenes, observing 60 Design group participants who co-created 3D assets for the scenes, and 60 Validation group participants who experienced the scenes with generated assets. Our findings show that co-designed assets are associated with higher scene engagement and shifted affective responses, while participants generally preferred concept images over the final 3D assets, with no increased leniency toward degradation in their own creations. Analysis of the human-AI conversations further shows that target environments shape users' conversational patterns. Our results suggest that our staged, intent-based co-design can democratize virtual reality authoring and shift immersive content creation from technical execution toward collaborative spatial design.

cs.HC↗

arXiv2026

From Fixed to Flexible: Shaping AI Personality in Context-Sensitive Interaction

Conversational agents are increasingly expected to adapt across contexts and evolve their personalities through interactions, yet most remain static once configured. We present an exploratory study of how user expectations form and evolve when agent personality is made dynamically adjustable. To investigate this, we designed a prototype conversational interface that enabled users to adjust an agent's personality along eight research-grounded dimensions across three task contexts: informational, emotional, and appraisal. We conducted an online mixed-methods study with 60 participants, employing latent profile analysis to characterize personality classes and trajectory analysis to trace evolving patterns of personality adjustment. These approaches revealed distinct personality profiles at initial and final configuration stages, and adjustment trajectories, shaped by context-sensitivity. Participants also valued the autonomy, perceived the agent as more anthropomorphic, and reported greater trust. Our findings highlight the importance of designing conversational agents that adapt alongside their users, advancing more responsive and human-centred AI.

cs.HC↗

arXiv2026

An LLM-Assisted Toolkit for Inspectable Multimodal Emotion Data Annotation

Multimodal Emotion Recognition (MER) increasingly depends on fine grained, evidence grounded annotations, yet inspection and label construction are hard to scale when cues are dynamic and misaligned across modalities. We present an LLM-assisted toolkit that supports multimodal emotion data annotation through an inspectable, event centered workflow. The toolkit preprocesses and aligns heterogeneous recordings, visualizes all modalities on an interactive shared timeline, and renders structured signals as video tracks for cross modal consistency checks. It then detects candidate events and packages synchronized keyframes and time windows as event packets with traceable pointers to the source data. Finally, the toolkit integrates an LLM with modality specific tools and prompt templates to draft structured annotations for analyst verification and editing. We demonstrate the workflow on multimodal VR emotion recordings with representative examples.

cs.HC↗

arXiv2026

Understanding the Effects of Interaction on Emotional Experiences in VR

Virtual reality has been effectively used for eliciting emotions, yet most research focuses on the intensity of affective responses rather than on how interaction influences those experiences. To address this gap, we advance a validated VR emotion-elicitation dataset through two key extensions. First, we add a new high-arousal, high-valence scene and validate its effectiveness in a within-subject study (N=24). Second, we incorporate interactive elements into each scene, creating both interactive and non-interactive versions to examine the impact of interaction on emotional responses. We evaluate interaction through a multimodal approach combining subjective ratings and physiological signals to capture both conscious and unconscious affective responses. Our evaluation study (N=84) shows that interaction not only amplifies emotions but modulates them in context, supporting coping in negative scenes and enhancing enjoyment in positive scenes. These findings highlight the potential of scene-tailored interaction for different applications, where regulating emotions is as important as eliciting them.

cs.HC↗

arXiv2026

Searching Through Complex Worlds: Visual Search and Spatial Regularity Memory in Mixed Reality

Visual search is a core component of mixed reality (MR) interactions, influenced by the complexities of MR application contexts. In this paper, we investigate how prevalent factors in MR influence visual search performance and spatial regularity memory -- including the physical environment complexity, secondary task presence, virtual content depth and spatial layout configurations. Contrary to prior work, we found that the secondary auditory task did not have a significant main effect on visual search performance, while significantly elevating higher perceived workload measures in all conditions. Complex environments and varied virtual elements depths significantly hinder visual search, but did not significantly increase perceived workload measures. Finally, participants did not explicitly recognize repeated spatial configurations of virtual elements, but performed significantly better when searching repeated spatial configurations, suggesting implicit memory of spatial regularities. Our work presents novel insights on visual search and highlights key considerations when designing MR for different application contexts.

cs.HC↗

arXiv2026

SRL Proxemics: Spatial Guidelines for Supernumerary Robotic Limbs in Near-Body Interactions

Wearable supernumerary robotic limbs (SRLs) sit at the intersection of human augmentation and embodied AI, transforming into extensions of the human body. However, their movements within the intimate near-body space raise unresolved challenges for perceived safety, user control, and trust. In this paper, we present results from a Wizard-of-Oz study (n=18), where participants completed near-body collaboration tasks with SRLs to explore these challenges. We collected qualitative data through think-aloud protocols and semi-structured interviews, complemented by physiological signals and post-task ratings. Findings indicate that greater autonomy did not inherently enhance perceived safety or trust. Instead, participants identified near-body zones and paired them with clear coordination rules. They also expressed expectations for how different arm components should behave, shaping preferences around autonomy, perceived safety, and trust. Building on these insights, we introduce SRL Proxemics, a zone- and segment-level design framework showing that autonomy is not monolithic: perceived safety hinges on spatially calibrated, legible behaviors, not higher autonomy.

cs.HC↗

arXiv2026

One Body, Two Minds: Alternating VR Perspective During Remote Teleoperation of Supernumerary Limbs

Remote VR teleoperation with supernumerary robotic limbs enables distant users to operate in another's local space. While a shared first-person view aids hand-eye coordination, locking the guest's camera to the host's head can degrade comfort, embodiment, and coordination. Based on a formative study (N=10) using a virtual supernumerary robotic limbs configuration to stress-test coordination, we propose guest-driven perspective switching from a shared first-person baseline (Shared Embodied View) to two alternatives: (a) a stabilized view with guest-controlled rotation (Embedded Anchored View), and (b) a fully decoupled third-person view (Out-of-body View). We ran a user study with 24 pairs (N=48) who switched between the baseline and proposed views as task demands changed. We measured performance, embodiment, fatigue, physiological arousal, and switching behaviors. Our results reveal role-dependent trade-offs: Out-of-body View improves navigation efficiency and reduces errors, while Embedded Anchored View supports embodiment. We conclude with guidelines: use Embedded Anchored View for hand-centric adjustments, Out-of-body View for navigation and object placement, and ensure smooth transitions.

cs.HC↗

arXiv2025

A Survey on Methodological Approaches to Collaborative Embodiment in Virtual Reality

The application and implementation of collaborative embodiment in virtual reality (VR) are a critical aspect of the computer science landscape, aiming to enhance multi-user interaction and teamwork in immersive environments. A notable and enduring area of collaborative embodiment research focuses on approaches that enable multiple users to share control, interact, and investigate scenarios involving supernumerary arms in virtual spaces. In this survey, we will present an extensive overview of the methodologies employed in the past decade to enable collaboration in VR environments, particularly through embodiment. Using the PRISMA guidelines, we plan to analyze the study details from over 137 relevant research papers. Through this analysis, a critical assessment of the effectiveness of these methodologies will be conducted, highlighting current challenges and limitations in implementing collaborative embodiment in VR. Lastly, we discuss potential future research directions and opportunities for enhancing collaboration embodiment in virtual environments.

cs.HC↗

arXiv2025

Raising Awareness of Location Information Vulnerabilities in Social Media Photos using LLMs

Location privacy leaks can lead to unauthorised tracking, identity theft, and targeted attacks, compromising personal security and privacy. This study explores LLM-powered location privacy leaks associated with photo sharing on social media, focusing on user awareness, attitudes, and opinions. We developed and introduced an LLM-powered location privacy intervention app to 19 participants, who used it over a two-week period. The app prompted users to reflect on potential privacy leaks that a widely available LLM could easily detect, such as visual landmarks & cues that could reveal their location, and provided ways to conceal this information. Through in-depth interviews, we found that our intervention effectively increased users' awareness of location privacy and the risks posed by LLMs. It also encouraged users to consider the importance of maintaining control over their privacy data and sparked discussions about the future of location privacy-preserving technologies. Based on these insights, we offer design implications to support the development of future user-centred, location privacy-preserving technologies for social media photos.

cs.HC↗

arXiv2025

TA-GNN: Physics Inspired Time-Agnostic Graph Neural Network for Finger Motion Prediction

Continuous prediction of finger joint movement using historical joint positions/rotations is vital in a multitude of applications, especially related to virtual reality, computer graphics, robotics, and rehabilitation. However, finger motions are highly articulated with multiple degrees of freedom, making them significantly harder to model and predict. To address this challenge, we propose a physics-inspired time-agnostic graph neural network (TA-GNN) to accurately predict human finger motions. The proposed encoder comprises a kinematic feature extractor to generate filtered velocity and acceleration and a physics-based encoder that follows linear kinematics. The model is designed to be prediction-time-agnostic so that it can seamlessly provide continuous predictions. The graph-based decoder for learning the topological motion between finger joints is designed to address the higher degree articulation of fingers. We show the superiority of our model performance in virtual reality context. This novel approach enhances finger tracking without additional sensors, enabling predictive interactions such as haptic re-targeting and improving predictive rendering quality.

cs.HC↗

arXiv2023

Safeguarding Crowdsourcing Surveys from ChatGPT with Prompt Injection

ChatGPT and other large language models (LLMs) have proven useful in crowdsourcing tasks, where they can effectively annotate machine learning training data. However, this means that they also have the potential for misuse, specifically to automatically answer surveys. LLMs can potentially circumvent quality assurance measures, thereby threatening the integrity of methodologies that rely on crowdsourcing surveys. In this paper, we propose a mechanism to detect LLM-generated responses to surveys. The mechanism uses "prompt injection", such as directions that can mislead LLMs into giving predictable responses. We evaluate our technique against a range of question scenarios, types, and positions, and find that it can reliably detect LLM-generated responses with more than 93% effectiveness. We also provide an open-source software to help survey designers use our technique to detect LLM responses. Our work is a step in ensuring that survey methodologies remain rigorous vis-a-vis LLMs.

cs.HC↗

arXiv2023

Using Thermal Imaging to Measure Hand Hygiene Quality

Hand hygiene has long been promoted as the most effective way to prevent the transmission of infection. However, due to the low compliance and quality of hand hygiene reported in previous studies, constant monitoring of healthcare workers' hand hygiene compliance and quality is crucial. In this study, we investigate the feasibility of using a thermal camera together with an RGB camera to detect hand coverage of alcohol-based formulation, thereby monitoring handrub quality. The system yields promising results in terms of accuracy (93.5%) and Dice coefficient (87.1%) when observations take place 10 seconds after performing handrub. In addition, we also examine the system performance change over a 60-second observation period, and the accuracy and Dice coefficient still remain at about 92.4% and 85.7% when observation happens at the 60-second time point. Given these encouraging results, thermal imaging shows its potential feasibility in providing accurate, constant, and systematic hand hygiene quality monitoring.

cs.HC↗

arXiv2022

Current Challenges of Using Wearable Devices for Online Emotion Sensing

A growing number of wearable devices is becoming increasingly non-invasive, readily available, and versatile for measuring different physiological signals. This renders them ideal for inferring the emotional states of their users. Despite the success of wearable devices in recent emotion studies, there are still several challenges to be addressed. In this position paper, we compare currently available wearables that can be used for emotion-sensing and identify the challenges and opportunities for future researchers. Our investigation opens the discussion of what is missing for in-the-wild for emotion-sensing studies.

cs.HC↗

arXiv2021

UbiNIRS: A Software Framework for Miniaturized NIRS-based Applications

We present UbiNIRS, a software framework for rapid development and deployment of applications using miniaturized near-infrared spectroscopy (NIRS). NIRS is an emerging material sensing technology that has shown a great potential in recent work from the HCI community such as in situ pill testing. However, existing methods require significant programming efforts and professional knowledge of NIRS, and hence, challenge the creation of new NIRS based applications. Our system helps to resolve this issue by providing a generic server and a mobile app, using the best practices for NIRS applications in literature. The server creates and manages UbiNIRS instances without the need for any coding or professional knowledge of NIRS. The mobile app can register multiple UbiNIRS instances by communicating with the server for different NIRS based applications. Furthermore, UbiNIRS enables NIRS spectrum crowdsourcing for building a knowledge base.

cs.HC↗

arXiv2021

InfoPrint: Embedding Information into 3D Printed Objects

We present a technique to embed information invisible to the eye inside 3D printed objects. The information is integrated in the object model, and then fabricated using off-the-shelf dual-head FDM (Fused Deposition Modeling) 3D printers. Our process does not require human intervention during or after printing with the integrated model. The information can be arbitrary symbols, such as icons, text,binary, or handwriting. To retrieve the information, we evaluate two different infrared-based imaging devices that are readily available-thermal cameras and near-infrared scanners. Based on our results, we propose design guidelines for a range of use cases to embed and extract hidden information. We demonstrate how our method can be used for different applications, such as interactive thermal displays, hidden board game tokens, tagging functional printed objects, and autographing non-fungible fabrication work.

cs.HC↗

arXiv2019

Situationally-Induced Impairments and Disabilities Research

Research has shown that various environmental factors impact smartphone interaction and lead to Situationally-Induced Impairments and Disabilities. In this work we discuss the importance of thoroughly understanding the effects of these situational impairments on smartphone interaction. We argue that systematic investigation of the effects of different situational impairments is quintessential for conducting successful research in the field of SIIDs that might lead to building appropriate sensing, modelling, and adapting techniques. We also provide insights for future work identifying potential directions to conduct research in SIIDs.

cs.HC↗

arXiv2019

Proceedings of the CHI'19 Workshop: Addressing the Challenges of Situationally-Induced Impairments and Disabilities in Mobile Interaction

Situationally-induced impairments and disabilities (SIIDs) make it difficult for users of interactive computing systems to perform tasks due to context (e.g., listening to a phone call when in a noisy crowd) rather than a result of a congenital or acquired impairment (e.g., hearing damage). SIIDs are a great concern when considering the ubiquitousness of technology in a wide range of contexts. Considering our daily reliance on technology, and mobile technology in particular, it is increasingly important that we fully understand and model how SIIDs occur. Similarly, we must identify appropriate methods for sensing and adapting technology to reduce the effects of SIIDs. In this workshop, we will bring together researchers working on understanding, sensing, modelling, and adapting technologies to ameliorate the effects of SIIDs. This workshop will provide a venue to identify existing research gaps, new directions for future research, and opportunities for future collaboration.

cs.HC↗