Searcharxiv⌕ Search

arXiv subjects

Eileen McGivney

Publications and source records attributed to Eileen McGivney.

4 recordsLinked to original sources

Assessing Learning Processes with Multimodal Data in Virtual Reality Learning Environments

Assessing learning in virtual reality (VR) environments typically relied on traditional pre-post content retention tests, revealing little about the process of learnng within such immersive environments. Multimodal data from player activity in VR is promising to better measure learning processes and higher-order skills, but little research in the learning sciences has explored how such data can be combined to provide meaningful measures. To address this, we explored multimodal sources of data from a VR escape room containing hands-on logic puzzles that require problem-solving strategies and engagement in verbal metacognitive reflections. We leverage VR's affordances to immerse users in sustained dialog and promote multimodal interactions in a digital environment to understand how logfile and verbal data reveal participants' reasoning skills versus guesswork.

cs.HC↗

Conversational Successes and Breakdowns in Everyday Smart Glasses Use

Non-Display Smart Glasses hold the potential to support everyday activities by combining continuous environmental sensing with voice-only interaction powered by large language models (LLMs). Understanding how conversational successes and breakdowns arise in everyday contexts can better inform the design of future voice-only interfaces. To investigate this, we conducted a month-long collaborative autoethnography (n=2) to identify patterns of successes and breakdowns when using such devices. We then compare these patterns with prior findings on voice-only interactions to highlight the unique affordances and opportunities offered by non-display smart glasses.

cs.HC↗

Leveraging Agency in Virtual Reality to Enable Situated Learning

Learning is an active process that is deeply tied to physical and social contexts. Yet schools traditionally place learners in a passive role and focus on decontextualizing knowledge. Situating learning in more authentic tasks and contexts typically requires taking it outside the classroom via field trips and apprenticeships, but virtual reality (VR) is a promising tool to bring more authentically situated learning experiences into classrooms. In this position paper, I discuss how one of VR's primary affordances for learning is heightening agenct, and how such heightened agency can facilitate more authenticlaly situated learning by allowing learners legitimate peripheral participation.

cs.HC↗

Designing VR Simulation System for Clinical Communication Training with LLMs-Based Embodied Conversational Agents

VR simulation in Health Professions (HP) education demonstrates huge potential, but fixed learning content with little customization limits its application beyond lab environments. To address these limitations in the context of VR for patient communication training, we conducted a user-centered study involving semi-structured interviews with advanced HP students to understand their challenges in clinical communication training and perceptions of VR-based solutions. From this, we derived design insights emphasizing the importance of realistic scenarios, simple interactions, and unpredictable dialogues. Building on these insights, we developed the Virtual AI Patient Simulator (VAPS), a novel VR system powered by Large Language Models (LLMs) and Embodied Conversational Agents (ECAs), supporting dynamic and customizable patient interactions for immersive learning. We also provided an example of how clinical professors could use user-friendly design forms to create personalized scenarios that align with course objectives in VAPS and discuss future implications of integrating AI-driven technologies into VR education.

cs.HC↗