arXiv subjects

Tim Miller

Publications and source records attributed to Tim Miller.

At least 19 recordsLinked to original sources

arXiv2026

Towards an Argumentative Foundation for Evaluative AI

Evaluative AI (EAI) has been recently proposed as a way to support human decision-making, not by producing a single recommendation, but by presenting competing hypotheses together with evidence for and against each. In this position paper, we advocate (computational) argumentation as a particularly suitable paradigm to provide a formal, computable foundation for forms of EAI that are explainable and contestable, setting the ground for a long-term research agenda towards distributed and human-centred EAI systems.

cs.AI↗

arXiv2026

M2-PALE: A Framework for Explaining Multi-Agent MCTS--Minimax Hybrids via Process Mining and LLMs

Monte-Carlo Tree Search (MCTS) is a fundamental sampling-based search algorithm widely used for online planning in sequential decision-making domains. Despite its success in driving recent advances in artificial intelligence, understanding the behavior of MCTS agents remains a challenge for both developers and users. This difficulty stems from the complex search trees produced through the simulation of numerous future states and their intricate relationships. A known weakness of standard MCTS is its reliance on highly selective tree construction, which may lead to the omission of crucial moves and a vulnerability to tactical traps. To resolve this, we incorporate shallow, full-width Minimax search into the rollout phase of multi-agent MCTS to enhance strategic depth. Furthermore, to demystify the resulting decision-making logic, we introduce \textsf{M2-PALE} (MCTS--Minimax Process-Aided Linguistic Explanations). This framework employs process mining techniques, specifically the Alpha Miner, iDHM, and Inductive Miner algorithms, to extract underlying behavioral workflows from agent execution traces. These process models are then synthesized by LLMs to generate human-readable causal and distal explanations. We demonstrate the efficacy of our approach in a small-scale checkers environment, establishing a scalable foundation for interpreting hybrid agents in increasingly complex strategic domains.

cs.AI↗

arXiv2026

Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systems

The use of Artificial Intelligence (AI) in high-risk, decision-making scenarios presents technical, safety, and normative challenges; problems that may only be ameliorated by human oversight. However, notions of human oversight lack a common foundational understanding: oversight architectures are not well defined, the roles involved remain unclear, and implementation steps are opaque. Hence, researchers and practitioners struggle to determine how to design, implement, and evaluate systems that enable effective human oversight. This paper advances a practical framework for effective human oversight of AI systems, based on a cross-disciplinary perspective that draws on insights from computer science, human-computer interaction, psychology, philosophy, and law. The core contributions are: (1) a foundational framework, with a working definition, architecture and processes for effective human oversight of AI systems; (2) an initial template for documenting oversight architectures and processes, applied to diverse domains; and (3) a synthesis of open research challenges that need to be considered in the emerging field of effective human oversight of AI systems.

cs.CY↗

arXiv2026

Same Performance, Hidden Bias: Evaluating Hypothesis- and Recommendation-Driven AI

The HCI community commonly evaluates decision support systems based on whether they improve task performance or promote appropriate user reliance. In this work, we look beyond decision outcomes to examine the process through which users develop decision-making strategies. Through a web-based experiment (N = 290) comparing recommendation-driven and hypothesis-driven interaction designs, and using Signal Detection Theory as a theoretical framework, we show that even when performance remains identical, recommendation-driven designs lower participants' thresholds for sufficient evidence and introduce a "hidden bias" in their judgments, resulting in a shifted distribution of errors. Furthermore, we find that experts are just as susceptible to these systemic shifts as novices. We conclude by advocating for a shift in focus: prioritizing decision processes and the preservation of stable evidence standards over performance and reliance alone.

cs.HC↗

arXiv2025

Reassessing Collaborative Writing Theories and Frameworks in the Age of LLMs: What Still Applies and What We Must Leave Behind

In this paper, we conduct a critical review of existing theories and frameworks on human-human collaborative writing to assess their relevance to the current human-AI paradigm in organizational workplace settings, and draw seven insights along with design implications for human-AI collaborative writing tools. Our main finding was that, as we delegate more writing to AI, our cognitive process shifts from the traditional planning/translating/reviewing process to a planning/waiting/reviewing process, breaking the process due to the waiting that occurs in between. To ensure that our cognitive process remains intact, we suggest a "prototyping" approach, where the tool allows for faster iterations of the cognitive process by starting with smaller chunks of text, and gradually moving on to a fully fleshed-out document. We aim to bring theoretical grounding and practical design guidance to the interaction designs of human-AI collaborative writing, with the goal of enhancing future human-AI writing software.

cs.HC↗

arXiv2025

Supporting Data-Frame Dynamics in AI-assisted Decision Making

High stakes decision-making often requires a continuous interplay between evolving evidence and shifting hypotheses, a dynamic that is not well supported by current AI decision support systems. In this paper, we introduce a mixed-initiative framework for AI assisted decision making that is grounded in the data-frame theory of sensemaking and the evaluative AI paradigm. Our approach enables both humans and AI to collaboratively construct, validate, and adapt hypotheses. We demonstrate our framework with an AI-assisted skin cancer diagnosis prototype that leverages a concept bottleneck model to facilitate interpretable interactions and dynamic updates to diagnostic hypotheses.

cs.HC↗

arXiv2025

Exploring Explainable Multi-agent MCTS-minimax Hybrids in Board Game Using Process Mining

Monte-Carlo Tree Search (MCTS) is a family of sampling-based search algorithms widely used for online planning in sequential decision-making domains and at the heart of many recent advances in artificial intelligence. Understanding the behavior of MCTS agents is difficult for developers and users due to the frequently large and complex search trees that result from the simulation of many possible futures, their evaluations, and their relationships. This paper presents our ongoing investigation into potential explanations for the decision-making and behavior of MCTS. A weakness of MCTS is that it constructs a highly selective tree and, as a result, can miss crucial moves and fall into tactical traps. Full-width minimax search constitutes the solution. We integrate shallow minimax search into the rollout phase of multi-agent MCTS and use process mining technique to explain agents' strategies in 3v3 checkers.

cs.AI↗

arXiv2024

Where Common Knowledge Cannot Be Formed, Common Belief Can -- Planning with Multi-Agent Belief Using Group Justified Perspectives

Epistemic planning is the sub-field of AI planning that focuses on changing knowledge and belief. It is important in both multi-agent domains where agents need to have knowledge/belief regarding the environment, but also the beliefs of other agents, including nested beliefs. When modeling knowledge in multi-agent settings, many models face an exponential growth challenge in terms of nested depth. A contemporary method, known as Planning with Perspectives (PWP), addresses these challenges through the use of perspectives and set operations for knowledge. The JP model defines that an agent's belief is justified if and only if the agent has seen evidence that this belief was true in the past and has not seen evidence to suggest that this has changed. The current paper extends the JP model to handle \emph{group belief}, including distributed belief and common belief. We call this the Group Justified Perspective (GJP) model. Using experimental problems crafted by adapting well-known benchmarks to a group setting, we show the efficiency and expressiveness of our GJP model at handling planning problems that cannot be handled by other epistemic planning tools.

cs.AI↗

arXiv2024

Evaluating Explanations Through LLMs: Beyond Traditional User Studies

As AI becomes fundamental in sectors like healthcare, explainable AI (XAI) tools are essential for trust and transparency. However, traditional user studies used to evaluate these tools are often costly, time consuming, and difficult to scale. In this paper, we explore the use of Large Language Models (LLMs) to replicate human participants to help streamline XAI evaluation. We reproduce a user study comparing counterfactual and causal explanations, replicating human participants with seven LLMs under various settings. Our results show that (i) LLMs can replicate most conclusions from the original study, (ii) different LLMs yield varying levels of alignment in the results, and (iii) experimental factors such as LLM memory and output variability affect alignment with human responses. These initial findings suggest that LLMs could provide a scalable and cost-effective way to simplify qualitative XAI evaluation.

cs.AI↗

arXiv2024

Towards Explainable Goal Recognition Using Weight of Evidence (WoE): A Human-Centered Approach

Goal recognition (GR) involves inferring an agent's unobserved goal from a sequence of observations. This is a critical problem in AI with diverse applications. Traditionally, GR has been addressed using 'inference to the best explanation' or abduction, where hypotheses about the agent's goals are generated as the most plausible explanations for observed behavior. Alternatively, some approaches enhance interpretability by ensuring that an agent's behavior aligns with an observer's expectations or by making the reasoning behind decisions more transparent. In this work, we tackle a different challenge: explaining the GR process in a way that is comprehensible to humans. We introduce and evaluate an explainable model for goal recognition (GR) agents, grounded in the theoretical framework and cognitive processes underlying human behavior explanation. Drawing on insights from two human-agent studies, we propose a conceptual framework for human-centered explanations of GR. Using this framework, we develop the eXplainable Goal Recognition (XGR) model, which generates explanations for both why and why not questions. We evaluate the model computationally across eight GR benchmarks and through three user studies. The first study assesses the efficiency of generating human-like explanations within the Sokoban game domain, the second examines perceived explainability in the same domain, and the third evaluates the model's effectiveness in aiding decision-making in illegal fishing detection. Results demonstrate that the XGR model significantly enhances user understanding, trust, and decision-making compared to baseline models, underscoring its potential to improve human-agent collaboration.

cs.AI↗

arXiv2024

UNCOVER: Significant Reddening in Cosmic Noon Quiescent Galaxies

We explore the physical properties of five massive quiescent galaxies at $z\sim2.5$, revealing the presence of non-negligible dust reservoirs. JWST NIRSpec observations were obtained for each target, finding no significant line emission; multiple star formation tracers independently place upper limits between $0.1-10~M_\odot / \mathrm{yr}$. Spectral energy distribution modeling with Prospector infers stellar masses between $\log_{10}[M / M_\odot] \sim 10-11$ and stellar mass-weighted ages between $1-2$ Gyr. The inferred mass-weighted effective radii ($r_{eff}\sim 0.4-1.4$ kpc) and inner $1$ kpc stellar surface densities ($\log_{10}[\Sigma / M_\odot \mathrm{kpc}^2 ]\gtrsim 9$) are typical of quiescent galaxies at $z \gtrsim 2$. The galaxies display negative color gradients (redder core and bluer outskirts); for one galaxy, this effect results from a dusty core, while for the others it may be evidence of an "inside-out" growth process. Unlike local quiescent galaxies, we identify significant reddening in these typical cosmic noon passive galaxies; all but one require $A_V \gtrsim 0.4$. This finding is in qualitative agreement with previous studies but our deep 20-band NIRCam imaging is able to significantly suppress the dust-age degeneracy and confidently determine that these galaxies are reddened. We speculate about the physical effects that may drive the decline in dust content in quiescent galaxies over cosmic time.

astro-ph.GA↗

arXiv2024

An Actionability Assessment Tool for Explainable AI

In this paper, we introduce and evaluate a tool for researchers and practitioners to assess the actionability of information provided to users to support algorithmic recourse. While there are clear benefits of recourse from the user's perspective, the notion of actionability in explainable AI research remains vague, and claims of `actionable' explainability techniques are based on the researchers' intuition. Inspired by definitions and instruments for assessing actionability in other domains, we construct a seven-question tool and evaluate its effectiveness through two user studies. We show that the tool discriminates actionability across explanation types and that the distinctions align with human judgements. We show the impact of context on actionability assessments, suggesting that domain-specific tool adaptations may foster more human-centred algorithmic systems. This is a significant step forward for research and practices into actionable explainability and algorithmic recourse, providing the first clear human-centred definition and tool for assessing actionability in explainable AI.

cs.HC↗

arXiv2024

3D-DASH: The Evolution of Size, Shape, and Intrinsic Scatter in Populations of Young and Old Quiescent Galaxies at 0.5 < z < 3

We present a study of the growth of the quiescent galaxy population between 0.5 < z < 3 by tracing the number density and structural evolution of a sample of 4518 old and 583 young quiescent galaxies with log($M_*$/$M_{\odot}$)>10.4, selected from the COSMOS2020 catalog with complementary HST/F160W imaging from the 3D-DASH survey. Among the quiescent population at z$\sim$2, roughly 50% are recently quenched galaxies; these young quiescent galaxies become increasingly rare towards lower redshift, supporting the idea that the peak epoch of massive galaxy quenching occurred at z>2. Our data show that while the effective half-light radii of quiescent galaxies generally increases with time, young quiescent galaxies are significantly smaller than their older counterparts at the same redshift. In this work we investigate the connection between this size difference and other structural properties, including axis ratios, color gradients, stellar mass, and the intrinsic scatter in effective radii. We demonstrate that the size difference is driven by the most massive sub-population (log($M_*$/$M_{\odot}$)>11) and does not persist when restricting the sample to intermediate mass galaxies (10.4<log($M_*$/$M_{\odot}$)<11). Interestingly, the intrinsic scatter in physical size shows a strong co-evolution over the investigated time period and peaks around z$\sim$2 for both populations, only diverging at z < 1. Taken together, and assuming we are not missing a significant population of lower surface brightness galaxies, while the formation and quenching mechanisms that dominate at higher redshifts yield compact remnants, multiple evolutionary pathways may explain the diverse morphologies of galaxies that quench at z<1.

astro-ph.GA↗

arXiv2024

Visual Evaluative AI: A Hypothesis-Driven Tool with Concept-Based Explanations and Weight of Evidence

This paper presents Visual Evaluative AI, a decision aid that provides positive and negative evidence from image data for a given hypothesis. This tool finds high-level human concepts in an image and generates the Weight of Evidence (WoE) for each hypothesis in the decision-making process. We apply and evaluate this tool in the skin cancer domain by building a web-based application that allows users to upload a dermatoscopic image, select a hypothesis and analyse their decisions by evaluating the provided evidence. Further, we demonstrate the effectiveness of Visual Evaluative AI on different concept-based explanation approaches.

cs.CV↗

arXiv2024

Explainable agency: human preferences for simple or complex explanations

Research in cognitive psychology has established that whether people prefer simpler explanations to complex ones is context dependent, but the question of `simple vs. complex' becomes critical when an artificial agent seeks to explain its decisions or predictions to humans. We present a model for abstracting causal reasoning chains for the purpose of explanation. This model uses a set of rules to progressively abstract different types of causal information in causal proof traces. We perform online studies using 123 Amazon MTurk participants and with five industry experts over two domains: maritime patrol and weather prediction. We found participants' satisfaction with generated explanations was based on the consistency of relationships among the causes (coherence) that explain an event; and that the important question is not whether people prefer simple or complex explanations, but what types of causal information are relevant to individuals in specific contexts.

cs.HC↗

arXiv2024

From Evidence to Decision: Exploring Evaluative AI

This paper presents a hypothesis-driven approach to improve AI-supported decision-making that is based on the Evaluative AI paradigm - a conceptual framework that proposes providing users with evidence for or against a given hypothesis. We propose an implementation of Evaluative AI by extending the Weight of Evidence framework, leading to hypothesis-driven models that support both tabular and image data. We demonstrate the application of the new decision-support approach in two domains: housing price prediction and skin cancer diagnosis. The findings show promising results in improving human decisions, as well as providing insights on the strengths and weaknesses of different decision-support approaches.

cs.AI↗

arXiv2023

Diverse, Top-k, and Top-Quality Planning Over Simulators

Diverse, top-k, and top-quality planning are concerned with the generation of sets of solutions to sequential decision problems. Previously this area has been the domain of classical planners that require a symbolic model of the problem instance. This paper proposes a novel alternative approach that uses Monte Carlo Tree Search (MCTS), enabling application to problems for which only a black-box simulation model is available. We present a procedure for extracting bounded sets of plans from pre-generated search trees in best-first order, and a metric for evaluating the relative quality of paths through a search tree. We demonstrate this approach on a path-planning problem with hidden information, and suggest adaptations to the MCTS algorithm to increase the diversity of generated plans. Our results show that our method can generate diverse and high-quality plan sets in domains where classical planners are not applicable.

cs.AI↗

arXiv2023

Human-centric Literature on Trust for SfTI Veracity Spearhead

This article summarizes the literature on trust of digital technologies from a human-centric perspective. We summarize literature on trust in face-to-face interactions from other fields, followed by a discussion of organizational trust, technology-mediated trust, trust of software products, trust of AI, and blockchain. This report was created for the Science for Technological Innovation Veracity Spearhead supported by New Zealand's National Science Challenges.

cs.CY↗