Sven Eckhardt | Searcharxiv

arXiv2024

A Survey of AI Reliance

Although artificial intelligence (AI) systems are becoming increasingly indispensable, research into how humans rely on these systems (AI reliance) is lagging behind. To advance this research, this survey presents a novel, comprehensive sociotechnical perspective on AI reliance, essential to fully understand the phenomenon. To address these challenges, the survey introduces a categorization framework resulting in a morphological box, which guides rigorous AI reliance research. Further, the survey identifies the core influences on AI reliance within the components of a sociotechnical system and discusses current limitations alongside emerging future research avenues to form a research agenda.

cs.HC↗

arXiv2024

HILL: A Hallucination Identifier for Large Language Models

Large language models (LLMs) are prone to hallucinations, i.e., nonsensical, unfaithful, and undesirable text. Users tend to overrely on LLMs and corresponding hallucinations which can lead to misinterpretations and errors. To tackle the problem of overreliance, we propose HILL, the "Hallucination Identifier for Large Language Models". First, we identified design features for HILL with a Wizard of Oz approach with nine participants. Subsequently, we implemented HILL based on the identified design features and evaluated HILL's interface design by surveying 17 participants. Further, we investigated HILL's functionality to identify hallucinations based on an existing question-answering dataset and five user interviews. We find that HILL can correctly identify and highlight hallucinations in LLM responses which enables users to handle LLM responses with more caution. With that, we propose an easy-to-implement adaptation to existing LLMs and demonstrate the relevance of user-centered designs of AI artifacts.

cs.HC↗