arXiv subjects

Alex Endert

Publications and source records attributed to Alex Endert.

49 records · Page 3Linked to original sources

arXiv2020

Should We Trust (X)AI? Design Dimensions for Structured Experimental Evaluations

This paper systematically derives design dimensions for the structured evaluation of explainable artificial intelligence (XAI) approaches. These dimensions enable a descriptive characterization, facilitating comparisons between different study designs. They further structure the design space of XAI, converging towards a precise terminology required for a rigorous study of XAI. Our literature review differentiates between comparative studies and application papers, revealing methodological differences between the fields of machine learning, human-computer interaction, and visual analytics. Generally, each of these disciplines targets specific parts of the XAI process. Bridging the resulting gaps enables a holistic evaluation of XAI in real-world scenarios, as proposed by our conceptual model characterizing bias sources and trust-building. Furthermore, we identify and discuss the potential for future work based on observed research gaps that should lead to better coverage of the proposed model.

cs.HC↗

arXiv2020

CAVA: A Visual Analytics System for Exploratory Columnar Data Augmentation Using Knowledge Graphs

Most visual analytics systems assume that all foraging for data happens before the analytics process; once analysis begins, the set of data attributes considered is fixed. Such separation of data construction from analysis precludes iteration that can enable foraging informed by the needs that arise in-situ during the analysis. The separation of the foraging loop from the data analysis tasks can limit the pace and scope of analysis. In this paper, we present CAVA, a system that integrates data curation and data augmentation with the traditional data exploration and analysis tasks, enabling information foraging in-situ during analysis. Identifying attributes to add to the dataset is difficult because it requires human knowledge to determine which available attributes will be helpful for the ensuing analytical tasks. CAVA crawls knowledge graphs to provide users with a a broad set of attributes drawn from external data to choose from. Users can then specify complex operations on knowledge graphs to construct additional attributes. CAVA shows how visual analytics can help users forage for attributes by letting users visually explore the set of available data, and by serving as an interface for query construction. It also provides visualizations of the knowledge graph itself to help users understand complex joins such as multi-hop aggregations. We assess the ability of our system to enable users to perform complex data combinations without programming in a user study over two datasets. We then demonstrate the generalizability of CAVA through two additional usage scenarios. The results of the evaluation confirm that CAVA is effective in helping the user perform data foraging that leads to improved analysis outcomes, and offer evidence in support of integrating data augmentation as a part of the visual analytics pipeline.

cs.IR↗

arXiv2020

Characterizing Automated Data Insights

Many researchers have explored tools that aim to recommend data insights to users. These tools automatically communicate a rich diversity of data insights and offer such insights for many different purposes. However, there is a lack of structured understanding concerning what researchers of these tools mean by "insight" and what tasks in the analysis workflow these tools aim to support. We conducted a systematic review of existing systems that seek to recommend data insights. Grounded in the review, we propose 12 types of automated insights and four purposes of automating insights. We further discuss the design opportunities emerged from our analysis.

cs.HC↗

arXiv2019

Geono-Cluster: Interactive Visual Cluster Analysis for Biologists

Biologists often perform clustering analysis to derive meaningful patterns, relationships, and structures from data instances and attributes. Though clustering plays a pivotal role in biologists' data exploration, it takes non-trivial efforts for biologists to find the best grouping in their data using existing tools. Visual cluster analysis is currently performed either programmatically or through menus and dialogues in many tools, which require parameter adjustments over several steps of trial-and-error. In this paper, we introduce Geono-Cluster, a novel visual analysis tool designed to support cluster analysis for biologists who do not have formal data science training. Geono-Cluster enables biologists to apply their domain expertise into clustering results by visually demonstrating how their expected clustering outputs should look like with a small sample of data instances. The system then predicts users' intentions and generates potential clustering results. Our study follows the design study protocol to derive biologists' tasks and requirements, design the system, and evaluate the system with experts on their own dataset. Results of our study with six biologists provide initial evidence that Geono-Cluster enables biologists to create, refine, and evaluate clustering results to effectively analyze their data and gain data-driven insights. At the end, we discuss lessons learned and the implications of our study.

cs.HC↗

arXiv2019

EmoCo: Visual Analysis of Emotion Coherence in Presentation Videos

Emotions play a key role in human communication and public presentations. Human emotions are usually expressed through multiple modalities. Therefore, exploring multimodal emotions and their coherence is of great value for understanding emotional expressions in presentations and improving presentation skills. However, manually watching and studying presentation videos is often tedious and time-consuming. There is a lack of tool support to help conduct an efficient and in-depth multi-level analysis. Thus, in this paper, we introduce EmoCo, an interactive visual analytics system to facilitate efficient analysis of emotion coherence across facial, text, and audio modalities in presentation videos. Our visualization system features a channel coherence view and a sentence clustering view that together enable users to obtain a quick overview of emotion coherence and its temporal evolution. In addition, a detail view and word view enable detailed exploration and comparison from the sentence level and word level, respectively. We thoroughly evaluate the proposed system and visualization techniques through two usage scenarios based on TED Talk videos and interviews with two domain experts. The results demonstrate the effectiveness of our system in gaining insights into emotion coherence in presentations.

cs.CV↗

arXiv2019

Investigating Direct Manipulation of Graphical Encodings as a Method for User Interaction

We investigate direct manipulation of graphical encodings as a method for interacting with visualizations. There is an increasing interest in developing visualization tools that enable users to perform operations by directly manipulating graphical encodings rather than external widgets such as checkboxes and sliders. Designers of such tools must decide which direct manipulation operations should be supported, and identify how each operation can be invoked. However, we lack empirical guidelines for how people convey their intended operations using direct manipulation of graphical encodings. We address this issue by conducting a qualitative study that examines how participants perform 15 operations using direct manipulation of standard graphical encodings. From this study, we 1) identify a list of strategies people employ to perform each operation, 2) observe commonalities in strategies across operations, and 3) derive implications to help designers leverage direct manipulation of graphical encoding as a method for user interaction.

cs.HC↗

arXiv2019

A User-based Visual Analytics Workflow for Exploratory Model Analysis

Many visual analytics systems allow users to interact with machine learning models towards the goals of data exploration and insight generation on a given dataset. However, in some situations, insights may be less important than the production of an accurate predictive model for future use. In that case, users are more interested in generating of diverse and robust predictive models, verifying their performance on holdout data, and selecting the most suitable model for their usage scenario. In this paper, we consider the concept of Exploratory Model Analysis (EMA), which is defined as the process of discovering and selecting relevant models that can be used to make predictions on a data source. We delineate the differences between EMA and the well-known term exploratory data analysis in terms of the desired outcome of the analytic process: insights into the data or a set of deployable models. The contributions of this work are a visual analytics system workflow for EMA, a user study, and two use cases validating the effectiveness of the workflow. We found that our system workflow enabled users to generate complex models, to assess them for various qualities, and to select the most relevant model for their task.

cs.HC↗

arXiv2019

TopicSifter: Interactive Search Space Reduction Through Targeted Topic Modeling

Topic modeling is commonly used to analyze and understand large document collections. However, in practice, users want to focus on specific aspects or "targets" rather than the entire corpus. For example, given a large collection of documents, users may want only a smaller subset which more closely aligns with their interests, tasks, and domains. In particular, our paper focuses on large-scale document retrieval with high recall where any missed relevant documents can be critical. A simple keyword matching search is generally not effective nor efficient as 1) it is difficult to find a list of keyword queries that can cover the documents of interest before exploring the dataset, 2) some documents may not contain the exact keywords of interest but may still be highly relevant, and 3) some words have multiple meanings, which would result in irrelevant documents included in the retrieved subset. In this paper, we present TopicSifter, a visual analytics system for interactive search space reduction. Our system utilizes targeted topic modeling based on nonnegative matrix factorization and allows users to give relevance feedback in order to refine their target and guide the topic modeling to the most relevant results.

cs.IR↗

arXiv2019

Liger: Combining Interaction Paradigms for Visual Analysis

Visualization tools usually leverage a single interaction paradigm (e.g., manual view specification, visualization by demonstration, etc.), which fosters the process of visualization construction. A large body of work has investigated the effectiveness of individual interaction paradigms, building an understanding of advantages and disadvantages of each in isolation. However, how can we leverage the benefits of multiple interaction paradigms by combining them into a single tool? We currently lack a holistic view of how interaction paradigms that use the same input modality (e.g., mouse) can be combined into a single tool and how people use such tools. To investigate opportunities and challenges in combining paradigms, we first created a multi-paradigm prototype (Liger) that combines two mouse-based interaction paradigms (manual view specification and visualization by demonstration) in a unified tool. We then conducted an exploratory study with Liger, providing initial evidence that people 1) use both paradigms interchangeably, 2) seamlessly switch between paradigms based on the operation at hand, and 3) choose to successfully complete a single operation using a combination of both paradigms.

cs.HC↗

arXiv2018

Evaluation of Visualization by Demonstration and Manual View Specification

We present an exploratory study comparing the visualization construction and data exploration processes of people using two visualization tools, each implementing a different interaction paradigm. One of the visualization tools implements the manual view specification paradigm (Polestar) and another implements the visualization by demonstration paradigm (VisExemplar). Findings of our study indicate that the interaction paradigms implemented in these tools influence: 1) approaches used for constructing visualizations, 2) how users form goals, 3) how many visualization alternatives are considered and created, and 4) the feeling of control during the visualization construction process.

cs.HC↗

arXiv2018

Task-Based Effectiveness of Basic Visualizations

Visualizations of tabular data are widely used; understanding their effectiveness in different task and data contexts is fundamental to scaling their impact. However, little is known about how basic tabular data visualizations perform across varying data analysis tasks and data attribute types. In this paper, we report results from a crowdsourced experiment to evaluate the effectiveness of five visualization types --- Table, Line Chart, Bar Chart, Scatterplot, and Pie Chart --- across ten common data analysis tasks and three data attribute types using two real-world datasets. We found the effectiveness of these visualization types significantly varies across task and data attribute types, suggesting that visualization design would benefit from considering context dependent effectiveness. Based on our findings, we derive recommendations on which visualizations to choose based on different tasks. We finally train a decision tree on the data we collected to drive a recommender, showcasing how to effectively engineer experimental user data into practical visualization systems.

cs.HC↗

arXiv2017

VisAR: Bringing Interactivity to Static Data Visualizations through Augmented Reality

Static visualizations have analytic and expressive value. However, many interactive tasks cannot be completed using static visualizations. As datasets grow in size and complexity, static visualizations start losing their analytic and expressive power for interactive data exploration. Despite this limitation of static visualizations, there are still many cases where visualizations are limited to being static (e.g., visualizations on presentation slides or posters). We believe in many of these cases, static visualizations will benefit from allowing users to perform interactive tasks on them. Inspired by the introduction of numerous commercial personal augmented reality (AR) devices, we propose an AR solution that allows interactive data exploration of datasets on static visualizations. In particular, we present a prototype system named VisAR that uses the Microsoft Hololens to enable users to complete interactive tasks on static visualizations.

cs.HC↗

arXiv2016

Adding Semantic Information into Data Models by Learning Domain Expertise from User Interaction

Interactive visual analytic systems enable users to discover insights from complex data. Users can express and test hypotheses via user interaction, leveraging their domain expertise and prior knowledge to guide and steer the analytic models in the system. For example, semantic interaction techniques enable systems to learn from the user's interactions and steer the underlying analytic models based on the user's analytical reasoning. However, an open challenge is how to not only steer models based on the dimensions or features of the data, but how to add dimensions or attributes to the data based on the domain expertise of the user. In this paper, we present a technique for inferring and appending dimensions onto the dataset based on the prior expertise of the user expressed via user interactions. Our technique enables users to directly manipulate a spatial organization of data, from which both the dimensions of the data are weighted, and also dimensions created to represent the prior knowledge the user brings to the system. We describe this technique and demonstrate its utility via a use case.

cs.HC↗