SearcharxivSearch

arXiv subjects

Ada Yi Zhao

Publications and source records attributed to Ada Yi Zhao.

3 recordsLinked to original sources

Studying Mobile Spatial Collaboration across Video Calls and Augmented Reality

Mobile video calls are widely used to share information about real-world objects and environments with remote collaborators. While these calls provide valuable visual context in real time, the experience of interacting with people and moving around a space is significantly reduced when compared to co-located conversations. Recent work has demonstrated the potential of Mobile Augmented Reality applications to enable more spatial forms of collaboration across distance. To better understand the dynamics of mobile AR collaboration and how this medium compares against the status quo, we conducted a comparative structured observation study to analyze people's perception of space and interaction with remote collaborators across mobile video calls and AR-based calls. Fourteen pairs of participants completed a spatial collaboration task using each medium. Through a mixed-methods analysis of session videos, transcripts, motion logs, post-task exercises, and interviews, we highlight how the choice of medium influences the roles and responsibilities that collaborators take on and the construction of a shared language for coordination. We discuss the importance of spatial reasoning with one's body, how video calls help participants "be on the same page" more directly, and how AR calls enable both onsite and remote collaborators to engage with the space and each other in ways that resemble in-person interaction. Our study offers a nuanced view of the benefits and limitations of both mediums, and we conclude with a discussion of design implications for future systems that integrate mobile video and AR to better support spatial collaboration in its many forms.

cs.HC

Generative Lecture: Making Lecture Videos Interactive with LLMs and AI Clone Instructors

We introduce Generative Lecture, a concept that makes existing lecture videos interactive through generative AI and AI clone instructors. By leveraging interactive avatars powered by HeyGen, ElevenLabs, and GPT-5, we embed an AI instructor into the video and augment the video content in response to students' questions. This allows students to personalize the lecture material, directly ask questions in the video, and receive tailored explanations generated and delivered by the AI-cloned instructor. From a design elicitation study (N=8), we identified four goals that guided the development of eight system features: 1) on-demand clarification, 2) enhanced visuals, 3) interactive example, 4) personalized explanation, 5) adaptive quiz, 6) study summary, 7) automatic highlight, and 8) adaptive break. We then conducted a user study (N=12) to evaluate the usability and effectiveness of the system and collected expert feedback (N=5). The results suggest that our system enables effective two-way communication and supports personalized learning.

cs.HC

Guided Reality: Generating Visually-Enriched AR Task Guidance with LLMs and Vision Models

Large language models (LLMs) have enabled the automatic generation of step-by-step augmented reality (AR) instructions for a wide range of physical tasks. However, existing LLM-based AR guidance often lacks rich visual augmentations to effectively embed instructions into spatial context for a better user understanding. We present Guided Reality, a fully automated AR system that generates embedded and dynamic visual guidance based on step-by-step instructions. Our system integrates LLMs and vision models to: 1) generate multi-step instructions from user queries, 2) identify appropriate types of visual guidance, 3) extract spatial information about key interaction points in the real world, and 4) embed visual guidance in physical space to support task execution. Drawing from a corpus of user manuals, we define five categories of visual guidance and propose an identification strategy based on the current step. We evaluate the system through a user study (N=16), completing real-world tasks and exploring the system in the wild. Additionally, four instructors shared insights on how Guided Reality could be integrated into their training workflows.

cs.HC