Xinnian Zhao | Searcharxiv

arXiv2026

"What Can I Do for You'': How Should AI Companions Provide Assistance to Players in Virtual Reality Games

Recent advances in artificial intelligence (AI) have expanded the capabilities of non-player characters (NPCs), enabling them to perceive game states, perform in-game actions. In immersive virtual reality (VR) games, such assistance is not limited to providing hints or interface-level support: an AI companion can appear as a co-present character, share the player's spatial environment, and visibly act on game objects. This raises a design question for VR gameplay: how can AI companions best assist players while preserving their active participation in the virtual world? To explore this question, we developed a VR puzzle game for Apple Vision Pro featuring an AI companion across four gameplay modes: no assistance, command-based assistance, discussion-based assistance, and autonomous agent play. A within-subjects study with 24 participants showed that AI assistance significantly reduced players' workload. However, autonomous agent play, despite producing the lowest workload, substantially diminished player experience by reducing challenge, autonomy, immersion, and enjoyment. Qualitative analysis further showed that players evaluated the companion not only by its usefulness, but also by whether it felt like a tool, a teammate, or an integrated character in the game world. We categorised participants into four player types and summarised their expectations of AI companions. These findings provide design implications for AI companions as embodied participants in VR games.

cs.HC↗

arXiv2025

Refining Transcripts With TV Subtitles by Prompt-Based Weakly Supervised Training of ASR

This study proposes a novel approach to using TV subtitles within a weakly supervised (WS) Automatic Speech Recognition (ASR) framework. Although TV subtitles are readily available, their imprecise alignment with corresponding audio limits their applicability as supervised targets for verbatim transcription. Rather than using subtitles as direct supervision signals, our method reimagines them as context-rich prompts. This design enables the model to handle discrepancies between spoken audio and subtitle text. Instead, generated pseudo transcripts become the primary targets, with subtitles acting as guiding cues for iterative refinement. To further enhance the process, we introduce a weighted attention mechanism that emphasizes relevant subtitle tokens during inference. Our experiments demonstrate significant improvements in transcription accuracy, highlighting the effectiveness of the proposed method in refining transcripts. These enhanced pseudo-labeled datasets provide high-quality foundational resources for training robust ASR systems.

cs.CL↗