Searcharxiv⌕ Search

arXiv subjects

Sam Yu-Te Lee

Publications and source records attributed to Sam Yu-Te Lee.

13 recordsLinked to original sources

VizCopilot: Fostering Appropriate Reliance on Enterprise Chatbots with Context Visualization

Enterprise AI chatbots show promise in supporting knowledge workers in information synthesis tasks by retrieving context from large, heterogeneous databases before generating answers. However, when the retrieved context misaligns with user intentions, these chatbots often produce ``irrelevantly right'' responses that raise the risk of overreliance. To foster appropriate reliance, we need a different approach than asking users to verify chatbot outputs. We posit that human oversight and modification of the context retrieval process can improve the quality of chatbots' outputs and users' awareness, resulting in more appropriate reliance and higher accountability. In this work, we explore this approach through a Research-through-Design study using VizCopilot, a chatbot extended with corpus visualization techniques. Through a within-subject study comparing VizCopilot and a simplified pure-text chatbot, we evaluate the role of visualization in enterprise chatbots and surface future design opportunities for fostering appropriate reliance through context engineering. Our findings show that visualization not only helps users detect and correct misaligned context, but also enhances user agency and system transparency. At the same time, the study reveals limitations regarding verification support and trust in AI summaries. Based on these findings, we discuss how visualization can foster appropriate reliance, the associated design trade-offs, and our vision towards transparent and sustainable human-AI interaction.

cs.HC↗

VArify: A Visual Analytics System for Verifying Knowledge Enhanced Large Language Model Responses in Food Science

Graph Retrieval-Augmented Generation (GraphRAG) grounds Large Language Model (LLM) responses in structured, domain-specific knowledge. However, verification remains difficult because errors can arise at multiple layers: an answer may not be faithful to its retrieved context, the retrieval pipeline may select irrelevant evidence, and the knowledge graph may omit relevant facts or contain incorrect triplets. We present VArify, a visual analytics system for food-science researchers to inspect and repair these layers in a GraphRAG workflow. A formative study with five participants using a conventional chat-and-table interface revealed behaviors reflecting answer anchoring, over-trust, and difficulty tracing multi-hop relationships. In response, VArify combines an evidence-first answer reveal with coordinated chat and hierarchical graph views, provenance inspection, and subquery editing. It connects the generated answer, the retrieval state, and the provenance-linked graph so that researchers can trace discrepancies across them and repair the retrieved context. From a qualitative system-use study on six food science researchers, we characterize how they scrutinize GraphRAG evidence with progressive visual inspection. Our study also reveals an important verification boundary: information absent from both retrieval and a user's knowledge remains unverifiable. We discuss the broader role of visualization in supporting oversight of LLM responses and outline future research directions.

cs.HC↗

Detecting and Understanding Ambiguity in Visualizations with Role-Playing Vision Language Model Interpretations

Communicative visualizations can support multiple chart-grounded takeaways, including divergent or conflicting ones; we refer to this variation as interpretive ambiguity. Such ambiguity can undermine a visualization's communicative goal when the supported takeaways diverge from its intended message. Evaluating visualizations with target audiences can surface this variation, but such evaluations can be costly and difficult to scale. In this work, we explore persona-conditioned Vision Language Models (VLMs) as a complementary evaluation tool for interpretive ambiguity. For a given visualization, our pipeline uses topic-relevant personas to elicit candidate contextual takeaways supported by interpretations of perceived visual patterns, such as trends and clusters. We derive a Target--Pattern--Scope framework for comparing these interpretations, use LLM judges to assign Compatible, Disjoint, or Opposing ambiguity labels to statement pairs, and summarize their distribution in an ambiguity profile. To improve chart grounding, we benchmark candidate VLMs on chart comprehension and select the highest-performing model for each chart type. We evaluate statement-pair labeling against human-authored annotations and apply the approach to real-world, public-facing visualizations curated from established datasets. Through case studies, we examine how perceived visual patterns relate to contextual takeaways and how a visualization redesign changes the sampled interpretations. Our findings show how persona-conditioned VLMs can complement human feedback by generating candidate contextual takeaways and surfacing potential ambiguity for further inspection.

cs.HC↗

Agentic Dashboards: A New Design Paradigm for Visual Analytics

Visual analytics systems traditionally combine analytical methods and interactive visualizations into a fixed application specified at the design phase. Coding agents offer a flexible approach: experts could conduct analysis through natural-language instructions. We envision an agentic dashboard as a persistent visual analytics workspace where an agent makes decisions about analytical methods, views, and their coordination within a designer-defined decision space. This approach shifts part of the visualization designer's role from engineering a fixed interface to defining the decisions an agent may make at runtime. We illustrate how such a dashboard could operate with current capabilities and propose criteria for evaluating its behavior and potential failures. As analytical workflows become more fluid and agent-mediated, visualization can help experts maintain awareness, coordinate with agents, and exercise judgment.

cs.HC↗

Vibe Visualizing: How Visualization Novices Try (and Fail) to Generate and Interpret Visualizations with Conversational AI

Conversational AI lowers technical barriers to generating and interpreting visualizations. People without visualization training or practical experience can therefore perform these tasks, but may lack the expertise needed to evaluate the resulting charts and interpretations. Through a study with 20 visualization novices, we examine conversation logs from 60 sessions with ChatGPT and develop a codebook that characterizes participant and AI behaviors in AI-assisted visualization. Paired with interview transcripts and survey responses, we derive three themes under outcome quality, AI non-compliance, and trust, verification, and repair. An exploratory role analysis identifies two session-level roles distinguished by user initiative and associated with different failure patterns. A comparative analysis with Gemini and Claude further reveals distinct system-specific behaviors. Based on these findings, we derive five design recommendations for AI-assisted visualization design and discuss implications for visualization literacy, human-AI roles, accountability, and agentic visualization.

cs.HC↗

Vibe-GUIDE: A Graph-based User Interface in IDEs for Oversight in Vibe Coding

In agentic coding, developers shift from implementing changes themselves to specifying intent, evaluating the agent's work, and making approval decisions. However, delegating implementation can introduce cognitive debt that erodes project comprehension over time, constraining developers' ability to provide oversight. In this work, we investigate the role of persistent shared representations in supporting project comprehension and oversight of coding agents. We present Vibe-GUIDE, an agentic coding interface built around a structural, live, manipulable, and adaptive graph representation organized by functional modules. We evaluated our interface in a randomized between-subjects study comparing how 16 developers completed three cumulative coding tasks using our interface or a Chat-only baseline. We found that Vibe-GUIDE can support project comprehension and sustained task performance while keeping developers cognitively involved in oversight. These findings show how persistent shared representations can complement natural-language interaction and help developers maintain the understanding needed to oversee agent-generated changes as projects evolve.

cs.HC↗

What Makes a Great Co-Worker in an AI-Native Workplace?

As knowledge work grows interdependent between humans and AI, we ask what makes a great co-worker in an AI-native workplace. To answer this, we conducted 22 interviews and a large-scale mixed-methods survey of 1,534 knowledge workers at a multinational technology company. We contribute BACI, a framework of 75 co-worker qualities that apply to humans and AI, spanning Benevolence, Ability, Cooperativeness, and Integrity. Comparing priorities for humans and AI identified 11 co-worker archetypes and revealed disagreement over whether AI should have warmth, take initiative, or own outcomes. We also show how priorities for these archetypes varied with workers' individual characteristics. Lastly, we contribute a taxonomy of AI work etiquette capturing the obligations co-workers expect of one another when preparing, sharing, and taking responsibility for AI-supported work. Based on these findings, we derive implications to inform worker-centric AI and workplace design.

cs.HC↗

A Design Study on Voice-based Interaction for Immersive Network Visualization and Analysis

Visual network analysis leverages network visualization authoring techniques to facilitate sensemaking, serendipitous discovery, and hypothesis verification on network data. However, transferring the same paradigm to immersive environments is non-trivial due to insufficient UI affordance for authoring operations. Researchers have studied combining multiple modalities for interactions, but the high learning curve of such input systems limits their adoption by typical data analysts, let alone for network analytics. In this work, we investigate the advantages and limitations of voice as the primary input modality with a research-through-design (RtD) study, in which we design a system that supports voice-based interactions for immersive network visualization facilitated by Large Language Models (LLMs). Through a user study on social network data analysis with participants from social science and computer science backgrounds, we find that voice interactions can improve perceived usability relative to controller-based interaction and lower the cognitive effort of formulating commands, since users can express intent in natural language rather than compressing it into terse instructions. We discuss design implications for immersive visualizations, highlighting how usability limits adoption while simplified interactions and voice-based controls enhance fluidity and support complex, multi-parameter operations.

cs.HC↗

VIDEE: Visual and Interactive Decomposition, Execution, and Evaluation of Text Analytics with Intelligent Agents

Text analytics has traditionally required specialized knowledge in Natural Language Processing (NLP) or text analysis, which presents a barrier for entry-level analysts. Recent advances in large language models (LLMs) have changed the landscape of NLP by enabling more accessible and automated text analysis (e.g., topic detection, summarization, information extraction, etc.). We introduce VIDEE, a system that supports entry-level data analysts to conduct advanced text analytics with intelligent agents. VIDEE instantiates a human-agent collaroration workflow consisting of three stages: (1) Decomposition, which incorporates a human-in-the-loop Monte-Carlo Tree Search algorithm to support generative reasoning with human feedback, (2) Execution, which generates an executable text analytics pipeline, and (3) Evaluation, which integrates LLM-based evaluation and visualizations to support user validation of execution results. We conduct two quantitative experiments to evaluate VIDEE's effectiveness and analyze common agent errors. A user study involving participants with varying levels of NLP and text analytics experience -- from none to expert -- demonstrates the system's usability and reveals distinct user behavior patterns. The findings identify design implications for human-agent collaboration, validate the practical utility of VIDEE for non-expert users, and inform future improvements to intelligent text analytics systems.

cs.CL↗

HINTs: Sensemaking on large collections of documents with Hypergraph visualization and INTelligent agents

Sensemaking on a large collection of documents (corpus) is a challenging task often found in fields such as market research, legal studies, intelligence analysis, political science, computational linguistics, etc. Previous works approach this problem either from a topic- or entity-based perspective, but they lack interpretability and trust due to poor model alignment. In this paper, we present HINTs, a visual analytics approach that combines topic- and entity-based techniques seamlessly and integrates Large Language Models (LLMs) as both a general NLP task solver and an intelligent agent. By leveraging the extraction capability of LLMs in the data preparation stage, we model the corpus as a hypergraph that matches the user's mental model when making sense of the corpus. The constructed hypergraph is hierarchically organized with an agglomerative clustering algorithm by combining semantic and connectivity similarity. The system further integrates an LLM-based intelligent chatbot agent in the interface to facilitate sensemaking. To demonstrate the generalizability and effectiveness of the HINTs system, we present two case studies on different domains and a comparative user study. We report our insights on the behavior patterns and challenges when intelligent agents are used to facilitate sensemaking. We find that while intelligent agents can address many challenges in sensemaking, the visual hints that visualizations provide are necessary to address the new problems brought by intelligent agents. We discuss limitations and future work for combining interactive visualization and LLMs more profoundly to better support corpus analysis.

cs.HC↗

Visual Text Mining with Progressive Taxonomy Construction for Environmental Studies

Environmental experts have developed the DPSIR (Driver, Pressure, State, Impact, Response) framework to systematically study and communicate key relationships between society and the environment. Using this framework requires experts to construct a DPSIR taxonomy from a corpus, annotate the documents, and identify DPSIR variables and relationships, which is laborious and inflexible. Automating it with conventional text mining faces technical challenges, primarily because the taxonomy often begins with abstract definitions, which experts progressively refine and contextualize as they annotate the corpus. In response, we develop GreenMine, a system that supports interactive text mining with prompt engineering. The system implements a prompting pipeline consisting of three simple and evaluable subtasks. In each subtask, the DPSIR taxonomy can be defined in natural language and iteratively refined as experts analyze the corpus. To support users evaluate the taxonomy, we introduce an uncertainty score based on response consistency. Then, we design a radial uncertainty chart that visualizes uncertainties and corpus topics, which supports interleaved evaluation and exploration. Using the system, experts can progressively construct the DPSIR taxonomy and annotate the corpus with LLMs. Using real-world interview transcripts, we present a case study to demonstrate the capability of the system in supporting interactive mining of DPSIR relationships, and an expert review in the form of collaborative discussion to understand the potential and limitations of the system. We discuss the lessons learned from developing the system and future opportunities for supporting interactive text mining in knowledge-intensive tasks for other application scenarios.

cs.HC↗

Towards Dataset-scale and Feature-oriented Evaluation of Text Summarization in Large Language Model Prompts

Recent advancements in Large Language Models (LLMs) and Prompt Engineering have made chatbot customization more accessible, significantly reducing barriers to tasks that previously required programming skills. However, prompt evaluation, especially at the dataset scale, remains complex due to the need to assess prompts across thousands of test instances within a dataset. Our study, based on a comprehensive literature review and pilot study, summarized five critical challenges in prompt evaluation. In response, we introduce a feature-oriented workflow for systematic prompt evaluation. In the context of text summarization, our workflow advocates evaluation with summary characteristics (feature metrics) such as complexity, formality, or naturalness, instead of using traditional quality metrics like ROUGE. This design choice enables a more user-friendly evaluation of prompts, as it guides users in sorting through the ambiguity inherent in natural language. To support this workflow, we introduce Awesum, a visual analytics system that facilitates identifying optimal prompt refinements for text summarization through interactive visualizations, featuring a novel Prompt Comparator design that employs a BubbleSet-inspired design enhanced by dimensionality reduction techniques. We evaluate the effectiveness and general applicability of the system with practitioners from various domains and found that (1) our design helps overcome the learning curve for non-technical people to conduct a systematic evaluation of summarization prompts, and (2) our feature-oriented workflow has the potential to generalize to other NLG and image-generation tasks. For future works, we advocate moving towards feature-oriented evaluation of LLM prompts and discuss unsolved challenges in terms of human-agent interaction.

cs.HC↗

NOVA: A visual interface for assessing polarizing media coverage

Within the United States, the majority of the populace receives their news online. U.S mainstream media outlets both generate and influence the news consumed by U.S citizens. Many of these citizens have their personal beliefs about these outlets and question the fairness of their reporting. We offer an interactive visualization system for the public to assess their perception of the mainstream media's coverage of a topic against the data. Our system combines belief elicitation techniques and narrative structure designs, emphasizing transparency and user-friendliness to facilitate users' self-assessment on personal beliefs. We gathered $\sim${25k} articles from the span of 2020-2022 from six mainstream media outlets as a testbed. To evaluate our system, we present usage scenarios alongside a user study with a qualitative analysis of user exploration strategies for personal belief assessment. We report our observations from this study and discuss future work and challenges of developing tools for the public to assess media outlet coverage and belief updating on provocative topics.

cs.HC↗