arXiv subjects

Laura Koesten

Publications and source records attributed to Laura Koesten.

At least 19 recordsLinked to original sources

arXiv2025

Structured Visualization Design Knowledge for Grounding Generative Reasoning and Situated Feedback

Automated visualization design navigates a tension between symbolic systems and generative models. Constraint solvers enforce structural and perceptual validity, but the rules they require are difficult to author and too rigid to capture situated design knowledge. Large language models require no formal rules and can reason about contextual nuance, but they prioritize popular conventions over empirically grounded best practices. We address this tension by proposing a cataloging scheme that structures visualization design knowledge as natural-language guidelines with semantically typed metadata. This allows experts to author knowledge that machines can query. An expert study ($N=18$) indicates that practitioners routinely adapt heuristics to situational factors such as audience and communicative intent. To capture this reasoning, guideline sections specify not only advice but also the contexts where it applies, exceptions that invalidate it, and the sources from which it derives. We demonstrate the scheme's expressiveness by cataloging 744 guidelines drawn from cognitive science, accessibility standards, data journalism, and research on rhetorical aspects of visual communication. We embed guideline sections in a vector space, opening the knowledge itself to structural analysis. This reveals conflicting advice across sources and transferable principles between domains. Rather than replacing constraint-based tools, our scheme provides what they lack: situated guidance that generative systems can retrieve to ground their reasoning, users can verify against cited sources, and experts can author as knowledge evolves.

cs.HC↗

arXiv2025

PLUTO: A Public Value Assessment Tool

We present PLUTO (Public VaLUe Assessment TOol), a framework for assessing the public value of specific instances of data use. Grounded in the concept of data solidarity, PLUTO aims to empower diverse stakeholders-including regulatory bodies, private enterprises, NGOs, and individuals-to critically engage with data projects through a structured assessment of the risks and benefits of data use, and by encouraging critical reflection. This paper discusses the theoretical foundation, development process, and initial user experiences with PLUTO. Key challenges include translating qualitative assessments of benefits and risks into actionable quantitative metrics while maintaining inclusivity and transparency. Initial feedback highlights PLUTO's potential to foster responsible decision-making and shared accountability in data practices.

cs.HC↗

arXiv2025

Do Vision-Language Models See Visualizations Like Humans? Alignment in Chart Categorization

Vision-language models (VLMs) hold promise for enhancing visualization tools, but effective human-AI collaboration hinges on a shared perceptual understanding of visual content. Prior studies assessed VLM visualization literacy through interpretive tasks, revealing an over-reliance on textual cues rather than genuine visual analysis. Our study investigates a more foundational skill underpinning such literacy: the ability of VLMs to recognize a chart's core visual properties as humans do. We task 13 diverse VLMs with classifying scientific visualizations based solely on visual stimuli, according to three criteria: purpose (e.g., schematic, GUI, visualization), encoding (e.g., bar, point, node-link), and dimensionality (e.g., 2D, 3D). Using expert labels from the human-centric VisType typology as ground truth, we find that VLMs often identify purpose and dimensionality accurately but struggle with specific encoding types. Our preliminary results show that larger models do not always equate to superior performance and highlight the need for careful integration of VLMs in visualization tasks, with human supervision to ensure reliable outcomes.

cs.HC↗

arXiv2025

A Composable Agentic System for Automated Visual Data Reporting

To address the brittleness of monolithic AI agents, our prototype for automated visual data reporting explores a Human-AI Partnership model. Its hybrid, multi-agent architecture strategically externalizes logic from LLMs to deterministic modules, leveraging the rule-based system Draco for principled visualization design. The system delivers a dual-output: an interactive Observable report with Mosaic for reader exploration, and executable Marimo notebooks for deep, analyst-facing traceability. This granular architecture yields a fully automatic yet auditable and steerable system, charting a path toward a more synergistic partnership between human experts and AI. For reproducibility, our implementation and examples are available at https://peter-gy.github.io/VISxGenAI-2025/.

cs.HC↗

arXiv2025

Encountering Friction, Understanding Crises: How Do Digital Natives Make Sense of Crisis Maps?

Crisis maps are regarded as crucial tools in crisis communication, as demonstrated during the COVID-19 pandemic and climate change crises. However, there is limited understanding of how public audiences engage with these maps and extract essential information. Our study investigates the sensemaking of young, digitally native viewers as they interact with crisis maps. We integrate frameworks from the learning sciences and human-data interaction to explore sensemaking through two empirical studies: a thematic analysis of online comments from a New York Times series on graph comprehension, and interviews with 18 participants from German-speaking regions. Our analysis categorizes sensemaking activities into established clusters: inspecting, engaging with content, and placing, and introduces responding personally to capture the affective dimension. We identify friction points connected to these clusters, including struggles with color concepts, responses to missing context, lack of personal connection, and distrust, offering insights for improving crisis communication to public audiences.

cs.HC↗

arXiv2024

Better Together? The Role of Explanations in Supporting Novices in Individual and Collective Deliberations about AI

Deploying AI systems in public institutions can have far-reaching consequences for many people, making it a matter of public interest. Providing opportunities for stakeholders to come together, understand these systems, and debate their merits and harms is thus essential. Explainable AI often focuses on individuals, but deliberation benefits from group settings, which are underexplored. To address this gap, we present findings from an interview study with 8 focus groups and 12 individuals. Our findings provide insight into how explanations support AI novices in deliberating alone and in groups. Participants used modular explanations with four information categories to solve tasks and decide about an AI system's deployment. We found that the explanations supported groups in creating shared understanding and in finding arguments for and against the system's deployment. In comparison, individual participants engaged with explanations in more depth and performed better in the study tasks, but missed an exchange with others. Based on our findings, we provide suggestions on how explanations should be designed to work in group settings and describe their potential use in real-world contexts. With this, our contributions inform XAI research that aims to enable AI novices to understand and deliberate AI systems in the public sector.

cs.HC↗

arXiv2024

A Multidimensional Assessment Method for Situated Visualization Understanding (MdamV)

How audiences read, interpret, and critique data visualizations is mainly assessed through performance tests featuring tasks like value retrieval. Yet, other factors shown to shape visualization understanding, such as numeracy, graph familiarity, and aesthetic perception, remain underrepresented in existing instruments. To address this, we design and test a Multidimensional Assessment Method of Situated Visualization Understanding (MdamV). This method integrates task-based measures with self-perceived ability ratings and open-ended critique, applied directly to the visualizations being read. Grounded in learning sciences frameworks that view understanding as a multifaceted process, MdamV spans six dimensions: Comprehending, Decoding, Aestheticizing, Critiquing, Reading, and Contextualizing. Validation was supported by a survey (N=438) representative of Austria's population (ages 18-74, male/female split), using a line chart and a bar chart on climate data. Findings show, for example, that about a quarter of respondents indicate deficits in comprehending simple data units, roughly one in five people felt unfamiliar with each chart type, and self-assessed numeracy was significantly related to data reading performance (p=0.0004). Overall, the evaluation of MdamV demonstrates the value of assessing visualization understanding beyond performance, framing it as a situated process tied to particular visualizations.

cs.HC↗

arXiv2024

Human-Data Interaction: Thinking beyond individual datasets

Having greater access to data leads to many benefits, from advancing science to promoting accountability in government to boosting innovation. However, merely providing data access does not make data easy to use; even when data is openly available online, people may struggle to work with it. In this article, we draw on prior work, including our own, and a case study of Kaggle (a large online data science community) to discuss the importance of moving away from viewing datasets as static resources. Instead, we describe the view of data as a process with its own interactional affordances that offer many different possibilities for data, as well as for social interaction. We advocate for the notion of Human Data Interactions and their potential implications for various audiences.

cs.HC↗

arXiv2024

Challenging the Human-in-the-loop in Algorithmic Decision-making

We discuss the role of humans in algorithmic decision-making (ADM) for socially relevant problems from a technical and philosophical perspective. In particular, we illustrate tensions arising from diverse expectations, values, and constraints by and on the humans involved. To this end, we assume that a strategic decision-maker (SDM) introduces ADM to optimize strategic and societal goals while the algorithms' recommended actions are overseen by a practical decision-maker (PDM) - a specific human-in-the-loop - who makes the final decisions. While the PDM is typically assumed to be a corrective, it can counteract the realization of the SDM's desired goals and societal values not least because of a misalignment of these values and unmet information needs of the PDM. This has significant implications for the distribution of power between the stakeholders in ADM, their constraints, and information needs. In particular, we emphasize the overseeing PDM's role as a potential political and ethical decision maker, who acts expected to balance strategic, value-driven objectives and on-the-ground individual decisions and constraints. We demonstrate empirically, on a machine learning benchmark dataset, the significant impact an overseeing PDM's decisions can have even if the PDM is constrained to performing only a limited amount of actions differing from the algorithms' recommendations. To ensure that the SDM's intended values are realized, the PDM needs to be provided with appropriate information conveyed through tailored explanations and its role must be characterized clearly. Our findings emphasize the need for an in-depth discussion of the role and power of the PDM and challenge the often-taken view that just including a human-in-the-loop in ADM ensures the 'correct' and 'ethical' functioning of the system.

cs.LG↗

arXiv2024

Tensions between Preference and Performance: Designing for Visual Exploration of Multi-frequency Medical Network Data

The analysis of complex high-dimensional data is a common task in many domains, resulting in bespoke visual exploration tools. Expectations and practices of domain experts as users do not always align with visualization theory. In this paper, we report on a design study in the medical domain where we developed two high-fidelity prototypes encoding EEG-derived brain network data with different types of visualizations. We evaluate these prototypes regarding effectiveness, efficiency, and preference with two groups: participants with domain knowledge (domain experts in medical research) and those without domain knowledge, both groups having little or no visualization experience. A requirement analysis and study of low-fidelity prototypes revealed a strong preference for a novel and aesthetically pleasing visualization design, as opposed to a design that is considered more optimal based on visualization theory. Our study highlights the pros and cons of both approaches, discussing trade-offs between task-specific measurements and subjective preference. While the aesthetically pleasing and novel low-fidelity prototype was favored, the results of our evaluation show that, in most cases, this was not reflected in participants' performance or subjective preference for the high-fidelity prototypes.

cs.HC↗

arXiv2024

Information That Matters: Exploring Information Needs of People Affected by Algorithmic Decisions

Every AI system that makes decisions about people has a group of stakeholders that are personally affected by these decisions. However, explanations of AI systems rarely address the information needs of this stakeholder group, who often are AI novices. This creates a gap between conveyed information and information that matters to those who are impacted by the system's decisions, such as domain experts and decision subjects. To address this, we present the "XAI Novice Question Bank," an extension of the XAI Question Bank containing a catalog of information needs from AI novices in two use cases: employment prediction and health monitoring. The catalog covers the categories of data, system context, system usage, and system specifications. We gathered information needs through task-based interviews where participants asked questions about two AI systems to decide on their adoption and received verbal explanations in response. Our analysis showed that participants' confidence increased after receiving explanations but that their understanding faced challenges. These included difficulties in locating information and in assessing their own understanding, as well as attempts to outsource understanding. Additionally, participants' prior perceptions of the systems' risks and benefits influenced their information needs. Participants who perceived high risks sought explanations about the intentions behind a system's deployment, while those who perceived low risks rather asked about the system's operation. Our work aims to support the inclusion of AI novices in explainability efforts by highlighting their information needs, aims, and challenges. We summarize our findings as five key implications that can inform the design of future explanations for lay stakeholder audiences.

cs.HC↗

arXiv2023

Data journeys in popular science: Producing climate change and COVID-19 data visualizations at Scientific American

Vast amounts of (open) data are increasingly used to make arguments about crisis topics such as climate change and global pandemics. Data visualizations are central to bringing these viewpoints to broader publics. However, visualizations often conceal the many contexts involved in their production, ranging from decisions made in research labs about collecting and sharing data to choices made in editorial rooms about which data stories to tell. In this paper, we examine how data visualizations about climate change and COVID-19 are produced in popular science magazines, using Scientific American, an established English-language popular science magazine, as a case study. To do this, we apply the analytical concept of data journeys (Leonelli, 2020) in a mixed methods study that centers on interviews with Scientific American staff and is supplemented by a visualization analysis of selected charts. In particular, we discuss the affordances of working with open data, the role of collaborative data practices, and how the magazine works to counter misinformation and increase transparency. This work provides an empirical contribution by providing insight into the data (visualization) practices of science communicators and demonstrating how the concept of data journeys can be used as an analytical framework.

cs.DL↗

arXiv2023

Subjective visualization experiences: impact of visual design and experimental design

In contrast to objectively measurable aspects (such as accuracy, reading speed, or memorability), the subjective experience of visualizations has only recently gained importance, and we have less experience how to measure it. We explore how subjective experience is affected by chart design using multiple experimental methods. We measure the effects of changes in color, orientation, and source annotation on the perceived readability and trustworthiness of simple bar charts. Three different experimental designs (single image rating, forced choice comparison, and semi-structured interviews) provide similar but different results. We find that these subjective experiences are different from what prior work on objective dimensions would predict. Seemingly inconsequential choices, like orientation, have large effects for some methods, indicating that study design alters decision-making strategies. Next to insights into the effect of chart design, we provide methodological insights, such as a suggested need to carefully isolate individual elements in charts to study subjective experiences.

cs.HC↗

arXiv2023

The Gulf of Interpretation: From Chart to Message and Back Again

Charts are used to communicate data visually, but often, we do not know whether a chart's intended message aligns with the message readers perceive. In this mixed-methods study, we investigate how data journalists encode data and how members of a broad audience engage with, experience, and understand these visualizations. We conducted workshops and interviews with school and university students, job seekers, designers, and senior citizens to collect perceived messages and feedback on eight real-world charts. We analyzed these messages and compared them to the intended message. Our results help to understand the gulf that can exist between messages (that producers encode) and viewer interpretations. In particular, we find that consumers are often overwhelmed with the amount of data provided and are easily confused with terms that are not well known. Chart producers tend to follow strong conventions on how to visually encode particular information that might not always benefit consumers.

cs.HC↗

arXiv2023

Practitioners' Perspectives on Designing Data Visualizations for the General Public

Public-facing data visualizations can play a vital role in making complex information clear and engaging, thereby encouraging informed public discourse and participation. However, existing work offers limited insight into how practitioners make design decisions based on their envisioned target audiences and across different media channels. To investigate this, we conducted semi-structured interviews with 21 professionals from journalistic settings, focusing on how they conceptualize their readers, translate these notions into design choices, and evaluate their work. We found that practitioners often rely on broad audience definitions, despite considering ``knowing their readers'' essential. Evaluation primarily relies on peer feedback or social metrics rather than user testing. From these accounts, we identify recurring strategies employed to reach general, often undefined publics. We discuss implications for audience-centered authoring tools, proposing features such as persona simulations and content-adaptive multi-format authoring, message-first rhetoric-aware workflows, and lightweight in-tool evaluation to better support the realities of public-facing design.

cs.HC↗

arXiv2023

Applying Interdisciplinary Frameworks to Understand Algorithmic Decision-Making

We argue that explanations for "algorithmic decision-making" (ADM) systems can profit by adopting practices that are already used in the learning sciences. We shortly introduce the importance of explaining ADM systems, give a brief overview of approaches drawing from other disciplines to improve explanations, and present the results of our qualitative task-based study incorporating the "six facets of understanding" framework. We close with questions guiding the discussion of how future studies can leverage an interdisciplinary approach.

cs.HC↗

arXiv2023

On the Impact of Explanations on Understanding of Algorithmic Decision-Making

Ethical principles for algorithms are gaining importance as more and more stakeholders are affected by "high-risk" algorithmic decision-making (ADM) systems. Understanding how these systems work enables stakeholders to make informed decisions and to assess the systems' adherence to ethical values. Explanations are a promising way to create understanding, but current explainable artificial intelligence (XAI) research does not always consider existent theories on how understanding is formed and evaluated. In this work, we aim to contribute to a better understanding of understanding by conducting a qualitative task-based study with 30 participants, including users and affected stakeholders. We use three explanation modalities (textual, dialogue, and interactive) to explain a "high-risk" ADM system to participants and analyse their responses both inductively and deductively, using the "six facets of understanding" framework by Wiggins & McTighe. Our findings indicate that the "six facets" framework is a promising approach to analyse participants' thought processes in understanding, providing categories for both rational and emotional understanding. We further introduce the "dialogue" modality as a valid explanation approach to increase participant engagement and interaction with the "explainer", allowing for more insight into their understanding in the process. Our analysis further suggests that individuality in understanding affects participants' perceptions of algorithmic fairness, demonstrating the interdependence between understanding and ADM assessment that previous studies have outlined. We posit that drawing from theories on learning and understanding like the "six facets" and leveraging explanation modalities can guide XAI research to better suit explanations to learning processes of individuals and consequently enable their assessment of ethical values of ADM systems.

cs.HC↗

arXiv2023

"The main message is that sustainability would help" -- Reflections on takeaway messages of climate change data visualizations

How do different audiences make sense of climate change data visualizations and what do they take away as a main message? To investigate this question, we are building on the results of a previous study, focusing on expert opinions regarding public climate change communication and the role of data visualizations. Hereby, we conducted semi-structured interviews with 17 experts in the fields of climate change, science communication, or data visualization. We also interviewed six lay persons with no professional background in either of these areas. With this analysis, we aim to shed light on how lay audiences arrive at an understanding of climate change data visualizations and what they take away as a main message. For two exemplary data visualizations, we compare their takeaway messages with messages formulated by experts. Through a thematic analysis, we observe differences regarding the included contents, the length and abstraction of messages, and the sensemaking process between and among the participant groups.

cs.HC↗