arXiv subjects

Ryan P. McMahan

Publications and source records attributed to Ryan P. McMahan.

At least 19 recordsLinked to original sources

arXiv2026

Can You Say This for Me? Speaking Up by Proxy in Co-Located Discussion

Equal participation in co-located discussion is important for effective collaboration, yet people often hold back when they anticipate negative interpersonal or professional consequences, especially when raising a point requires voicing it themselves. We present SecondVoice, a mixed-reality system that enables people to speak up through an embodied virtual proxy. By separating what is said from who says it, SecondVoice brings hesitant points into the live spoken discussion without putting the speaker on the spot. Using a private overlay, users specify their intent through a structured specification process rather than composing a full utterance. The system reformulates the input and voices it into the conversation through the proxy. We characterize a design space of participation channels under social risk. In a preliminary within-subject study (N = 16), we compare the complete SecondVoice system with an anonymous text-board channel across two group discussion tasks. Half of participants reported using SecondVoice for a point they did not say aloud, compared with 18.8% for the text board. Proxy-delivered points entered the spoken floor and were followed by multi-turn group engagement, which we did not observe after text-board posts. Participants described the channel as situationally valuable but identified tradeoffs around timing, ownership, and trust in reformulation.

cs.AI↗

arXiv2026

Generating Synthetic Behavioral Populations from XR Motion

Large-scale behavioral datasets are becoming increasingly important for machine learning, personalization, and behavioral modeling in extended reality (XR). However, collecting XR motion data from hundreds or thousands of participants remains expensive, time-consuming, and difficult to reproduce across research groups. As a result, many XR studies continue to rely on relatively small datasets that limit the scale and diversity of behavioral evaluation. To address this limitation, we investigate synthetic behavioral populations as a complementary approach to traditional XR data collection. We present an interpolation-based motion synthesis pipeline that combines dynamic time warping (DTW) with trajectory interpolation to generate synthetic behavioral trajectories from existing XR datasets while preserving task structure and incorporating motion characteristics from contributing participants. Using the publicly available FAST VR assembly dataset, we generated and openly released 100 synthetic behavioral trajectories. We evaluated the synthesized trajectories through motion-based user identification. Hybrid datasets containing both real and synthesized trajectories achieved performance comparable to similarly sized real-only datasets while maintaining low confusion between synthesized trajectories and their contributing participants. Rather than serving as conventional data augmentation, the proposed approach generates distinguishable behavioral trajectories that expand XR behavioral populations for larger-scale behavioral modeling and machine learning evaluation. Our findings demonstrate that synthetic behavioral populations provide a promising approach to expanding XR behavioral datasets and supporting future data-driven immersive systems.

cs.HC↗

arXiv2026

The Capturing and Logging Ecological Virtual Experiences and Reality (CLEVER) - Job Simulator Dataset

Virtual reality (VR) motion tracking and interaction data has become increasingly recognized as valuable for machine learning experiments for a variety of purposes, including predicting user identities, predicting user attributes like gender and age, predicting retention and learning, and more. However, there exist a limited number of publicly accessible VR motion datasets. In this paper, we present a new open-source dataset of 95 participants playing the SteamVR game Job Simulator. Additionally, we review existing datasets, detail our study procedure, describe our data collection process, list attributes of our dataset, and suggest future work, impact, and applications.

cs.HC↗

arXiv2025

Motion-Based User Identification across XR and Metaverse Applications by Deep Classification and Similarity Learning

This paper examines the generalization capacity of two state-of-the-art classification and similarity learning models in reliably identifying users based on their motions in various Extended Reality (XR) applications. We developed a novel dataset containing a wide range of motion data from 49 users in five different XR applications: four XR games with distinct tasks and action patterns, and an additional social XR application with no predefined task sets. The dataset is used to evaluate the performance and, in particular, the generalization capacity of the two models across applications. Our results indicate that while the models can accurately identify individuals within the same application, their ability to identify users across different XR applications remains limited. Overall, our results provide insight into current models generalization capabilities and suitability as biometric methods for user verification and identification. The results also serve as a much-needed risk assessment of hazardous and unwanted user identification in XR and Metaverse applications. Our cross-application XR motion dataset and code are made available to the public to encourage similar research on the generalization of motion-based user identification in typical Metaverse application use cases.

cs.HC↗

arXiv2025

Mitigating Response Delays in Free-Form Conversations with LLM-powered Intelligent Virtual Agents

We investigated the challenges of mitigating response delays in free-form conversations with virtual agents powered by Large Language Models (LLMs) within Virtual Reality (VR). For this, we used conversational fillers, such as gestures and verbal cues, to bridge delays between user input and system responses and evaluate their effectiveness across various latency levels and interaction scenarios. We found that latency above 4 seconds degrades quality of experience, while natural conversational fillers improve perceived response time, especially in high-delay conditions. Our findings provide insights for practitioners and researchers to optimize user engagement whenever conversational systems' responses are delayed by network limitations or slow hardware. We also contribute an open-source pipeline that streamlines deploying conversational agents in virtual environments.

cs.HC↗

arXiv2025

The Full-scale Assembly Simulation Testbed (FAST) Dataset

In recent years, numerous researchers have begun investigating how virtual reality (VR) tracking and interaction data can be used for a variety of machine learning purposes, including user identification, predicting cybersickness, and estimating learning gains. One constraint for this research area is the dearth of open datasets. In this paper, we present a new open dataset captured with our VR-based Full-scale Assembly Simulation Testbed (FAST). This dataset consists of data collected from 108 participants (50 females, 56 males, 2 non-binary) learning how to assemble two distinct full-scale structures in VR. In addition to explaining how the dataset was collected and describing the data included, we discuss how the dataset may be used by future researchers.

cs.HC↗

arXiv2025

QISCIT: A validated concept inventory assessment for quantum information science

Quantum information science (QIS) is a critical interdisciplinary field that requires a well-educated workforce in the near future. Numerous researchers and educators have been actively investigating how to best educate and prepare such a workforce. An open issue has been the lack of a validated tool to asses QIS understanding without requiring college-level math. In this paper, we present the systematic development and content validation of a new assessment instrument called the Quantum Information Science Concept Introductory Test (QISCIT). With feedback from 11 QIS experts, we have developed and validated a 31-item version of QISCIT that covers concepts like quantum states, quantum measurement, qubits, entanglement, coherence and decoherence, quantum gates and computing, and quantum communication. In addition to openly sharing our new concept inventory, we discuss how introductory QIS instructors can use it in their courses.

physics.ed-ph↗

arXiv2025

The Fidelity-based Presence Scale (FPS): Modeling the Effects of Fidelity on Sense of Presence

Within the virtual reality (VR) research community, there have been several efforts to develop questionnaires with the aim of better understanding the sense of presence. Despite having numerous surveys, the community does not have a questionnaire that informs which components of a VR application contributed to the sense of presence. Furthermore, previous literature notes the absence of consensus on which questionnaire or questions should be used. Therefore, we conducted a Delphi study, engaging presence experts to establish a consensus on the most important presence questions and their respective verbiage. We then conducted a validation study with an exploratory factor analysis (EFA). The efforts between our two studies led to the creation of the Fidelity-based Presence Scale (FPS). With our consensus-driven approach and fidelity-based factoring, we hope the FPS will enable better communication within the research community and yield important future results regarding the relationship between VR system fidelity and presence.

cs.HC↗

arXiv2025

AdaptiveCoPilot: Design and Testing of a NeuroAdaptive LLM Cockpit Guidance System in both Novice and Expert Pilots

Pilots operating modern cockpits often face high cognitive demands due to complex interfaces and multitasking requirements, which can lead to overload and decreased performance. This study introduces AdaptiveCoPilot, a neuroadaptive guidance system that adapts visual, auditory, and textual cues in real time based on the pilot's cognitive workload, measured via functional Near-Infrared Spectroscopy (fNIRS). A formative study with expert pilots (N=3) identified adaptive rules for modality switching and information load adjustments during preflight tasks. These insights informed the design of AdaptiveCoPilot, which integrates cognitive state assessments, behavioral data, and adaptive strategies within a context-aware Large Language Model (LLM). The system was evaluated in a virtual reality (VR) simulated cockpit with licensed pilots (N=8), comparing its performance against baseline and random feedback conditions. The results indicate that the pilots using AdaptiveCoPilot exhibited higher rates of optimal cognitive load states on the facets of working memory and perception, along with reduced task completion times. Based on the formative study, experimental findings, qualitative interviews, we propose a set of strategies for future development of neuroadaptive pilot guidance systems and highlight the potential of neuroadaptive systems to enhance pilot performance and safety in aviation environments.

cs.HC↗

arXiv2024

Cultural Reflections in Virtual Reality: The Effects of User Ethnicity in Avatar Matching Experiences on Sense of Embodiment

Matching avatar characteristics to a user can impact sense of embodiment (SoE) in VR. However, few studies have examined how participant demographics may interact with these matching effects. We recruited a diverse and racially balanced sample of 78 participants to investigate the differences among participant groups when embodying both demographically matched and unmatched avatars. We found that participant ethnicity emerged as a significant factor, with Asian and Black participants reporting lower total SoE compared to Hispanic participants. Furthermore, we found that user ethnicity significantly influences ownership (a subscale of SoE), with Asian and Black participants exhibiting stronger effects of matched avatar ethnicity compared to White participants. Additionally, Hispanic participants showed no significant differences, suggesting complex dynamics in ethnic-racial identity. Our results also reveal significant main effects of matched avatar ethnicity and gender on SoE, indicating the importance of considering these factors in VR experiences. These findings contribute valuable insights into understanding the complex dynamics shaping VR experiences across different demographic groups.

cs.HC↗

arXiv2024

Virtual Reality Games: Extending Unity Learn Games to VR

Research involving virtual reality (VR) has dramatically increased since the introduction of consumer VR systems. In turn, research on VR games has gained popularity within several fields. However, most VR games are closed source, which limits research opportunities. Some VR games are open source, but most of them are either very basic or too complex to be easily used in research. In this paper, we present two source-available VR games developed from freely available Unity Learn games: a kart racing game and a 3D adventure game. Our hope is that other researchers find them easy to use for VR studies, as Unity Technologies developed the games for beginners and has provided tutorials on using them.

cs.HC↗

arXiv2024

Stepping into the Right Shoes: The Effects of User-Matched Avatar Ethnicity and Gender on Sense of Embodiment in Virtual Reality

In many consumer virtual reality (VR) applications, users embody predefined characters that offer minimal customization options, frequently emphasizing storytelling over user choice. We explore whether matching a user's physical characteristics, specifically ethnicity and gender, with their virtual self-avatar affects their sense of embodiment in VR. We conducted a 2 x 2 within-subjects experiment (n=32) with a diverse user population to explore the impact of matching or not matching a user's self-avatar to their ethnicity and gender on their sense of embodiment. Our results indicate that matching the ethnicity of the user and their self-avatar significantly enhances sense of embodiment regardless of gender, extending across various aspects, including appearance, response, and ownership. We also found that matching gender significantly enhanced ownership, suggesting that this aspect is influenced by matching both ethnicity and gender. Interestingly, we found that matching ethnicity specifically affects self-location while matching gender specifically affects one's body ownership.

cs.HC↗

arXiv2024

The Interaction Fidelity Model: A Taxonomy to Distinguish the Aspects of Fidelity in Virtual Reality

Fidelity describes how closely a replication resembles the original. It can be helpful to analyze how faithful interactions in virtual reality (VR) are to a reference interaction. In prior research, fidelity has been restricted to the simulation of reality - also called realism. Our definition includes other reference interactions, such as superpowers or fiction. Interaction fidelity is a multilayered concept. Unfortunately, different aspects of fidelity have either not been distinguished in scientific discourse or referred to with inconsistent terminology. Therefore, we present the Interaction Fidelity Model (IntFi Model). Based on the human-computer interaction loop, it systematically covers all stages of VR interactions. The conceptual model establishes a clear structure and precise definitions of eight distinct components. It was reviewed through interviews with fourteen VR experts. We provide guidelines, diverse examples, and educational material to universally apply the IntFi Model to any VR experience. We identify common patterns and propose foundational research opportunities.

cs.HC↗

arXiv2023

VALID: A perceptually validated Virtual Avatar Library for Inclusion and Diversity

As consumer adoption of immersive technologies grows, virtual avatars will play a prominent role in the future of social computing. However, as people begin to interact more frequently through virtual avatars, it is important to ensure that the research community has validated tools to evaluate the effects and consequences of such technologies. We present the first iteration of a new, freely available 3D avatar library called the Virtual Avatar Library for Inclusion and Diversity (VALID), which includes 210 fully rigged avatars with a focus on advancing racial diversity and inclusion. We present a detailed process for creating, iterating, and validating avatars of diversity. Through a large online study (n=132) with participants from 33 countries, we provide statistically validated labels for each avatar's perceived race and gender. Through our validation study, we also advance knowledge pertaining to the perception of an avatar's race. In particular, we found that avatars of some races were more accurately identified by participants of the same race.

cs.HC↗

arXiv2023

RecolorCloud: A Point Cloud Tool for Recoloring, Segmentation, and Conversion

Point clouds are a 3D space representation of an environment that was recorded with a high precision laser scanner. These scanners can suffer from environmental interference such as surface shading, texturing, and reflections. Because of this, point clouds may be contaminated with fake or incorrect colors. Current open source or proprietary tools offer limited or no access to correcting these visual errors automatically. RecolorCloud is a tool developed to resolve these color conflicts by utilizing automated color recoloring. We offer the ability to deleting or recoloring outlier points automatically with users only needing to specify bounding box regions to effect colors. Results show a vast improvement of the photo-realistic quality of large point clouds. Additionally, users can quickly recolor a point cloud with set semantic segmentation colors.

cs.CV↗

arXiv2021

Using Machine Learning to Predict Game Outcomes Based on Player-Champion Experience in League of Legends

League of Legends (LoL) is the most widely played multiplayer online battle arena (MOBA) game in the world. An important aspect of LoL is competitive ranked play, which utilizes a skill-based matchmaking system to form fair teams. However, players' skill levels vary widely depending on which champion, or hero, that they choose to play as. In this paper, we propose a method for predicting game outcomes in ranked LoL games based on players' experience with their selected champion. Using a deep neural network, we found that game outcomes can be predicted with 75.1% accuracy after all players have selected champions, which occurs before gameplay begins. Our results have important implications for playing LoL and matchmaking. Firstly, individual champion skill plays a significant role in the outcome of a match, regardless of team composition. Secondly, even after the skill-based matchmaking, there is still a wide variance in team skill before gameplay begins. Finally, players should only play champions that they have mastered, if they want to win games.

cs.LG↗

arXiv2020

The Effects of Object Shape, Fidelity, Color, and Luminance on Depth Perception in Handheld Mobile Augmented Reality

Depth perception of objects can greatly affect a user's experience of an augmented reality (AR) application. Many AR applications require depth matching of real and virtual objects and have the possibility to be influenced by depth cues. Color and luminance are depth cues that have been traditionally studied in two-dimensional (2D) objects. However, there is little research investigating how the properties of three-dimensional (3D) virtual objects interact with color and luminance to affect depth perception, despite the substantial use of 3D objects in visual applications. In this paper, we present the results of a paired comparison experiment that investigates the effects of object shape, fidelity, color, and luminance on depth perception of 3D objects in handheld mobile AR. The results of our study indicate that bright colors are perceived as nearer than dark colors for a high-fidelity, simple 3D object, regardless of hue. Additionally, bright red is perceived as nearer than any other color. These effects were not observed for a low-fidelity version of the simple object or for a more-complex 3D object. High-fidelity objects had more perceptual differences than low-fidelity objects, indicating that fidelity interacts with color and luminance to affect depth perception. These findings reveal how the properties of 3D models influence the effects of color and luminance on depth perception in handheld mobile AR and can help developers select colors for their applications.

cs.HC↗

arXiv2019

A Taxonomy and Dataset for 360° Videos

In this paper, we propose a taxonomy for 360° videos that categorizes videos based on moving objects and camera motion. We gathered and produced 28 videos based on the taxonomy, and recorded viewport traces from 60 participants watching the videos. In addition to the viewport traces, we provide the viewers' feedback on their experience watching the videos, and we also analyze viewport patterns on each category.

cs.MM↗