arXiv subjects

Niklas Elmqvist

Publications and source records attributed to Niklas Elmqvist.

At least 37 records · Page 2Linked to original sources

arXiv2026

Ambient Analytics: Calm Technology for Immersive Visualization and Sensemaking

Augmented reality has great potential for embedding data visualizations in the world around the user. While this can enhance users' understanding of their surroundings, it also bears the risk of overwhelming their senses with a barrage of information. In contrast, calm technologies aim to place information in the user's attentional periphery, minimizing cognitive load instead of demanding focused engagement. In this column, we explore how visualizations can be harmoniously integrated into our everyday life through augmented reality, progressing from visual analytics to ambient analytics.

cs.HC↗

arXiv2026

A Multiliteracy Model for Interactive Visualization Literacy: Definitions, Literacies, and Steps for Future Research

This paper presents a theoretical model for interactive visualization literacy to describe how people use interactive data visualizations and systems. Literacies have become an important concept in describing modern life skills, with visualization literacy generally referring to the use and interpretation of data visualizations. However, prior work on visualization literacy overlooks interaction and its associated challenges, despite it being an intrinsic aspect of using visualizations. Based on existing theoretical frameworks, we derive a two-dimensional model that combines four well-known literacies with five novel ones. We found evidence for our model through analyzing existing visualization systems as well as through observations from an exploratory study involving such systems. We conclude by outlining steps towards measuring, evaluating, designing for, and teaching interactive visualization literacy.

cs.HC↗

arXiv2026

Hybrid User Interfaces: Past, Present, and Future of Complementary Cross-Device Interaction in Mixed Reality

We investigate hybrid user interfaces (HUIs), aiming to establish a cohesive understanding and adopt consistent terminology for this nascent research area. HUIs combine heterogeneous devices in complementary roles, leveraging the distinct benefits of each. Our work focuses on cross-device interaction between 2D devices and mixed reality environments, which are particularly compelling, leveraging the familiarity of traditional 2D platforms while providing spatial awareness and immersion. Although such HUIs have been prominently explored in the context of mixed reality by prior work, we still lack a cohesive understanding of the unique design possibilities and challenges of such combinations, resulting in a fragmented research landscape. We conducted a systematic survey and present a taxonomy of HUIs that combine conventional display technology and mixed reality environments. Based on this, we discuss past and current challenges, the evolution of definitions, and prospective opportunities to tie together the past 30 years of research with our vision of future HUIs.

cs.HC↗

arXiv2025

Eye of the Beholder: Towards Measuring Visualization Complexity

Constructing expressive and legible visualizations is a key activity for visualization designers. While numerous design guidelines exist, research on how specific graphical features affect perceived visual complexity remains limited. In this paper, we report on a crowdsourced study to collect human ratings of perceived complexity for diverse visualizations. Using these ratings as ground truth, we then evaluated three methods to estimate this perceived complexity: image analysis metrics, multilinear regression using manually coded visualization features, and automated feature extraction using a large language model (LLM). Image complexity metrics showed no correlation with human-perceived visualization complexity. Manual feature coding produced a reasonable predictive model but required substantial effort. In contrast, a zero-shot LLM (GPT-4o mini) demonstrated strong capabilities in both rating complexity and extracting relevant features. Our findings suggest that visualization complexity is truly in the eye of the beholder, yet can be effectively approximated using zero-shot LLM prompting, offering a scalable approach for evaluating the complexity of visualizations. The dataset and code for the study and data analysis can be found at https://osf.io/w85a4/

cs.HC↗

arXiv2025

Data Therapist: Eliciting Domain Knowledge from Subject Matter Experts Using Large Language Models

Effective data visualization requires not only technical proficiency but also a deep understanding of the domain-specific context in which data exists. This context often includes tacit knowledge about data provenance, quality, and intended use, which is rarely explicit in the dataset itself. Motivated by growing demands to surface tacit knowledge, we present the Data Therapist, a web-based system that helps domain experts externalize such implicit knowledge through a mixed-initiative process combining iterative Q&A with interactive annotation. Powered by a large language model, the system automatically analyzes user-supplied datasets, prompts users with targeted questions, and supports annotation at varying levels of granularity. The resulting structured knowledge base can inform both human and automated visualization design. A qualitative study with expert pairs from Accounting, Political Science, and Computer Security revealed recurring patterns in how expert reason about their data and highlighted opportunities for AI support to enhance visualization design.

cs.HC↗

arXiv2025

Agentic Visualization: Extracting Agent-based Design Patterns from Visualization Systems

Autonomous agents powered by Large Language Models are transforming AI, creating an imperative for the visualization field to embrace agentic frameworks. However, our field's focus on a human in the sensemaking loop raises critical questions about autonomy, delegation, and coordination for such \textit{agentic visualization} that preserve human agency while amplifying analytical capabilities. This paper addresses these questions by reinterpreting existing visualization systems with semi-automated or fully automatic AI components through an agentic lens. Based on this analysis, we extract a collection of design patterns for agentic visualization, including agentic roles, communication and coordination. These patterns provide a foundation for future agentic visualization systems that effectively harness AI agents while maintaining human insight and control.

cs.HC↗

arXiv2025

EventLines: Time Compression for Discrete Event Timelines

Discrete event sequences serve as models for numerous real-world datasets, including publications over time, project milestones, and medication dosing during patient treatments. These event sequences typically exhibit bursty behavior, where events cluster together in rapid succession, interspersed with periods of inactivity. Standard timeline charts with linear time axes fail to adequately represent such data, resulting in cluttered regions during event bursts while leaving other areas unutilized. We introduce EventLines, a novel technique that dynamically adjusts the time scale to match the underlying event distribution, enabling more efficient use of screen space. To address the challenges of non-linear time scaling, EventLines employs the time axis's visual representation itself to communicate the varying scale. We present findings from a crowdsourced graphical perception study that examines how different time scale representations influence temporal perception.

cs.HC↗

arXiv2025

Visualizationary: Automating Design Feedback for Visualization Designers using LLMs

Interactive visualization editors empower users to author visualizations without writing code, but do not provide guidance on the art and craft of effective visual communication. In this paper, we explore the potential of using an off-the-shelf large language models (LLMs) to provide actionable and customized feedback to visualization designers. Our implementation, VISUALIZATIONARY, demonstrates how ChatGPT can be used for this purpose through two key components: a preamble of visualization design guidelines and a suite of perceptual filters that extract salient metrics from a visualization image. We present findings from a longitudinal user study involving 13 visualization designers-6 novices, 4 intermediates, and 3 experts-who authored a new visualization from scratch over several days. Our results indicate that providing guidance in natural language via an LLM can aid even seasoned designers in refining their visualizations. All our supplemental materials are available at https://osf.io/v7hu8.

cs.HC↗

arXiv2025

Automating the Path: An R&D Agenda for Human-Centered AI and Visualization

The emergence of generative AI, large language models (LLMs), and foundation models is fundamentally reshaping computer science, and visualization and visual analytics are no exception. We present a systematic framework for understanding how human-centered AI (HCAI) can transform the visualization discipline. Our framework maps four key HCAI tool capabilities -- amplify, augment, empower, and enhance -- onto the four phases of visual sensemaking: view, explore, schematize, and report. For each combination, we review existing tools, envision future possibilities, identify challenges and pitfalls, and examine ethical considerations. This design space can serve as an R\&D agenda for both visualization researchers and practitioners to integrate AI into their work as well as understanding how visualization can support HCAI research.

cs.HC↗

arXiv2025

The Plot Thickens: Quantitative Part-by-Part Exploration of MLLM Visualization Literacy

Multimodal Large Language Models (MLLMs) can interpret data visualizations, but what makes a visualization understandable to these models? Do factors like color, shape, and text influence legibility, and how does this compare to human perception? In this paper, we build on prior work to systematically assess which visualization characteristics impact MLLM interpretability. We expanded the Visualization Literacy Assessment Test (VLAT) test set from 12 to 380 visualizations by varying plot types, colors, and titles. This allowed us to statistically analyze how these features affect model performance. Our findings suggest that while color palettes have no significant impact on accuracy, plot types and the type of title significantly affect MLLM performance. We observe similar trends for model omissions. Based on these insights, we look into which plot types are beneficial for MLLMs in different tasks and propose visualization design principles that enhance MLLM readability. Additionally, we make the extended VLAT test set, VLAT ex, publicly available on https://osf.io/ermwx/ together with our supplemental material for future model testing and evaluation.

cs.HC↗

arXiv2025

"I Feel Like I'm Teaching in a Gladiator Ring": Barriers and Benefits of Live Coding in Classroom Settings

Live coding for teaching-synchronously writing software in front of students-can be an effective method for engaging students and instilling practical programming skills. However, not all settings are conducive to live coding and not all instructors are successful in this challenging task. We present results from a study involving university instructors, teaching assistants, and students identifying both barriers and benefits of live coding. Physical infrastructure, a positive classroom community with psychological safety, and opportunities for teacher development are practical considerations for live coding. In order for live coding to be an active learning experience, we recommend that tools support multiple mechanisms for engaging students, directing audience attention, and encouraging student-led live coding.

cs.HC↗

arXiv2025

Drillboards: Adaptive Visualization Dashboards for Dynamic Personalization of Visualization Experiences

We present drillboards, a technique for adaptive visualization dashboards consisting of a hierarchy of coordinated charts that the user can drill down to reach a desired level of detail depending on their expertise, interest, and desired effort. This functionality allows different users to personalize the same dashboard to their specific needs and expertise. The technique is based on a formal vocabulary of chart representations and rules for merging multiple charts of different types and data into single composite representations. The drillboard hierarchy is created by iteratively applying these rules starting from a baseline dashboard, with each consecutive operation yielding a new dashboard with fewer charts and progressively more abstract and simplified views. We also present an authoring tool for building drillboards and show how experts users can use to build up and deliver personalized experiences to a wide audience. Our evaluation asked three domain experts to author drillboards for their own datasets, which we then showed to casual end-users with favorable outcomes.

cs.HC↗

arXiv2024

DG Comics: Semi-Automatically Authoring Graph Comics for Dynamic Graphs

Comics are an effective method for sequential data-driven storytelling, especially for dynamic graphs -- graphs whose vertices and edges change over time. However, manually creating such comics is currently time-consuming, complex, and error-prone. In this paper, we propose DG Comics, a novel comic authoring tool for dynamic graphs that allows users to semi-automatically build and annotate comics. The tool uses a newly developed hierarchical clustering algorithm to segment consecutive snapshots of dynamic graphs while preserving their chronological order. It also presents rich information on both individuals and communities extracted from dynamic graphs in multiple views, where users can explore dynamic graphs and choose what to tell in comics. For evaluation, we provide an example and report the results of a user study and an expert review.

cs.HC↗

arXiv2024

Attention-Aware Visualization: Tracking and Responding to User Perception Over Time

We propose the notion of Attention-Aware Visualizations (AAVs) that track the user's perception of a visual representation over time and feed this information back to the visualization. Such context awareness is particularly useful for ubiquitous and immersive analytics where knowing which embedded visualizations the user is looking at can be used to make visualizations react appropriately to the user's attention: for example, by highlighting data the user has not yet seen. We can separate the approach into three components: (1) measuring the user's gaze on a visualization and its parts; (2) tracking the user's attention over time; and (3) reactively modifying the visual representation based on the current attention metric. In this paper, we present two separate implementations of AAV: a 2D data-agnostic method for web-based visualizations that can use an embodied eyetracker to capture the user's gaze, and a 3D data-aware one that uses the stencil buffer to track the visibility of each individual mark in a visualization. Both methods provide similar mechanisms for accumulating attention over time and changing the appearance of marks in response. We also present results from a qualitative evaluation studying visual feedback and triggering mechanisms for capturing and revisualizing attention.

cs.HC↗

arXiv2024

The HaLLMark Effect: Supporting Provenance and Transparent Use of Large Language Models in Writing with Interactive Visualization

The use of Large Language Models (LLMs) for writing has sparked controversy both among readers and writers. On one hand, writers are concerned that LLMs will deprive them of agency and ownership, and readers are concerned about spending their time on text generated by soulless machines. On the other hand, AI-assistance can improve writing as long as writers can conform to publisher policies, and as long as readers can be assured that a text has been verified by a human. We argue that a system that captures the provenance of interaction with an LLM can help writers retain their agency, conform to policies, and communicate their use of AI to publishers and readers transparently. Thus we propose HaLLMark, a tool for visualizing the writer's interaction with the LLM. We evaluated HaLLMark with 13 creative writers, and found that it helped them retain a sense of control and ownership of the text.

cs.HC↗

arXiv2023

The Reality of the Situation: A Survey of Situated Analytics

The advent of low cost, accessible, and high performance augmented reality (AR) has shed light on a situated form of analytics where in-situ visualizations embedded in the real world can facilitate sensemaking based on the user's physical location. In this work, we identify prior literature in this emerging field with a focus on situated analytics. After collecting 47 relevant situated analytics systems, we classify them using a taxonomy of three dimensions: situating triggers, view situatedness, and data depiction. We then identify four archetypical patterns in our classification using an ensemble cluster analysis. We also assess the level which these systems support the sensemaking process. Finally, we discuss insights and design guidelines that we learned from our analysis.

cs.HC↗

arXiv2023

Textiverse: A Scalable Visual Analytics System for Exploring Geotagged and Timestamped Text Corpora

We propose Textiverse, a big data approach for mining geotagged timestamped textual data on a map, such as for Twitter feeds, crime reports, or restaurant reviews. We use a scalable data management pipeline that extracts keyphrases from online databases in parallel. We speed up this time-consuming step so that it outpaces the content creation rate of popular social media. The result is presented in a web-based interface that integrates with Google Maps to visualize textual content of massive scale. The visual design is based on aggregating spatial regions into discrete sites and rendering each such site as a circular tag cloud. To demonstrate the intended use of our technique, we first show how it can be used to characterize the U.S.\ National Science Foundation funding status based on all 489,151 awards. We then apply the same technique on visually representing a more spatially scattered and linguistically informal dataset: 1.2 million Twitter posts about the Android mobile operating system.

cs.HC↗

arXiv2023

uxSense: Supporting User Experience Analysis with Visualization and Computer Vision

Analyzing user behavior from usability evaluation can be a challenging and time-consuming task, especially as the number of participants and the scale and complexity of the evaluation grows. We propose uxSense, a visual analytics system using machine learning methods to extract user behavior from audio and video recordings as parallel time-stamped data streams. Our implementation draws on pattern recognition, computer vision, natural language processing, and machine learning to extract user sentiment, actions, posture, spoken words, and other features from such recordings. These streams are visualized as parallel timelines in a web-based front-end, enabling the researcher to search, filter, and annotate data across time and space. We present the results of a user study involving professional UX researchers evaluating user data using uxSense. In fact, we used uxSense itself to evaluate their sessions.

cs.HC↗