Signals of AI Hallucination: Designing Hallucination-Aware Cues for Embodied Conversational Agents in VR

Abstract

LLM-powered conversational agents (CAs) often present uncertainty and provenance cues alongside their responses to help users assess response reliability and identify potential hallucinations. In immersive environments such as Virtual Reality (VR), CAs often take the form of speech-based embodied conversational agents (ECAs), where uncertainty and provenance cues cannot rely on persistent inline text and may be missed or disrupt comprehension when delivered through speech. We conducted a within-subjects study (N = 24) to compare three designs for presenting the hallucination-awareness information (uncertainty and provenance) in ECAs in VR against a no-cue baseline: embodied cues using gestures and posture, icon cues using visual indicators, and text cues using color-coded text with inline citations. We evaluated how these designs affect users' ability to identify hallucination-related information, trust in the ECA, and interaction experience (immersion and task load). Our results show that all three designs support users in identifying hallucinations. Embodied cues were associated with higher trust and immersion, text cues offered clearer interpretability, and icon cues preserved relatively good interpretability while causing less disruption to immersion compared with embodied cues and text cues. This work contributes to the VR and AI research community by comparing different designs of hallucination cues in immersive ECA settings and examining how they affect users' ability and experiences to identify hallucinations. It also offers practical insights and design implications for developing future hallucination-awareness interfaces for ECA.