arXiv subjects

Romy Müller

Publications and source records attributed to Romy Müller.

13 recordsLinked to original sources

arXiv2026

When more precision is worse: Do people recognize inadequate scene representations in concept-based explainable AI?

Explainable artificial intelligence (XAI) aims to help uncover flaws in an AI model's internal representations. But do people draw the right conclusions from its explanations? Specifically, do they recognize an AI's inability to distinguish between relevant and irrelevant features? In the present study, a simulated AI classified images of railway trespassers as dangerous or not. To explain which features it has used, other images from the dataset were shown that activate the AI in a similar way. These concept images varied in three relevant features (i.e., a person's distance to the tracks, direction, and action) and in an irrelevant feature (i.e., scene background). When the AI uses a feature in its decision, this feature is retained in the concept images, otherwise the images randomize over it (e.g., same distance, varied backgrounds). Participants rated the AI more favorably when it retained relevant features. For the irrelevant feature, they did not mind in general, and sometimes even preferred it to be retained. This suggests that people may not recognize it when an AI model relies on irrelevant features to make its decisions.

cs.HC↗

arXiv2026

Treating symptoms or root causes: How does information about causal mechanisms affect interventions?

When deciding how to solve complex problems, it seems important not only to know whether an intervention is helpful but also to understand why. Therefore, the present study investigated whether explicit information about causal mechanisms enables people to distinguish between multiple interventions. It was hypothesised that mechanism information helps them appreciate indirect interventions that treat the root causes of a problem instead of just fixing its symptoms. This was investigated in an experimental hoof trimming scenario in which participants evaluated various interventions. To do so, they received causal diagrams with different types of causal information and levels of mechanistic detail. While detailed mechanism information and its embedding in the context of other influences made participants less sceptical towards indirect interventions, the effects were quite small. Moreover, it did not mitigate participants' robust preference for interventions that only fix a problem's symptoms. Taken together, the findings suggest that in order to help people choose sustainable interventions, it is not sufficient to make information about causal mechanisms available.

cs.HC↗

arXiv2025

When concept-based XAI is imprecise: Do people distinguish between generalisations and misrepresentations?

Concept-based explainable artificial intelligence (C-XAI) can let people see which representations an AI model has learned. This is particularly important when high-level semantic information (e.g., actions and relations) is used to make decisions about abstract categories (e.g., danger). In such tasks, AI models need to generalise beyond situation-specific details, and this ability can be reflected in C-XAI outputs that randomise over irrelevant features. However, it is unclear whether people appreciate such generalisation and can distinguish it from other, less desirable forms of imprecision in C-XAI outputs. Therefore, the present study investigated how the generality and relevance of C-XAI outputs affect people's evaluation of AI. In an experimental railway safety evaluation scenario, participants rated the performance of a simulated AI that classified traffic scenes involving people as dangerous or not. These classification decisions were explained via concepts in the form of similar image snippets. The latter differed in their match with the classified image, either regarding a highly relevant feature (i.e., people's relation to tracks) or a less relevant feature (i.e., people's action). Contrary to the hypotheses, concepts that generalised over less relevant features were rated lower than concepts that matched the classified image precisely. Moreover, their ratings were no better than those for systematic misrepresentations of the less relevant feature. Conversely, participants were highly sensitive to imprecisions in relevant features. These findings cast doubts on the assumption that people can easily infer from C-XAI outputs whether AI models have gained a deeper understanding of complex situations.

cs.HC↗

arXiv2025

How humans evaluate AI systems for person detection in automatic train operation: Not all misses are alike

If artificial intelligence (AI) is to be applied in safety-critical domains, its performance needs to be evaluated reliably. The present study aimed to understand how humans evaluate AI systems for person detection in automatic train operation. In three experiments, participants saw image sequences of people moving in the vicinity of railway tracks. A simulated AI had highlighted all detected people, sometimes correctly and sometimes not. Participants had to provide a numerical rating of the AI's performance and then verbally explain their rating. The experiments varied several factors that might influence human ratings: the types and plausibility of AI mistakes, the number of affected images, the number of people present in an image, the position of people relevant to the tracks, and the methods used to elicit human evaluations. While all these factors influenced human ratings, some effects were unexpected or deviated from normative standards. For instance, the factor with the strongest impact was people's position relative to the tracks, although participants had explicitly been instructed that the AI could not process such information. Taken together, the results suggest that humans may sometimes evaluate more than the AI's performance on the assigned task. Such mismatches between AI capabilities and human expectations should be taken into consideration when conducting safety audits of AI systems.

cs.HC↗

arXiv2024

Beyond object identification: How train drivers evaluate the risk of collision

When trains collide with obstacles, the consequences are often severe. To assess how artificial intelligence might contribute to avoiding collisions, we need to understand how train drivers do it. What aspects of a situation do they consider when evaluating the risk of collision? In the present study, we assumed that train drivers do not only identify potential obstacles but interpret what they see in order to anticipate how the situation might unfold. However, to date it is unclear how exactly this is accomplished. Therefore, we assessed which cues train drivers use and what inferences they make. To this end, image-based expert interviews were conducted with 33 train drivers. Participants saw images with potential obstacles, rated the risk of collision, and explained their evaluation. Moreover, they were asked how the situation would need to change to decrease or increase collision risk. From their verbal reports, we extracted concepts about the potential obstacles, contexts, or consequences, and assigned these concepts to various categories (e.g., people's identity, location, movement, action, physical features, and mental states). The results revealed that especially for people, train drivers reason about their actions and mental states, and draw relations between concepts to make further inferences. These inferences systematically differ between situations. Our findings emphasise the need to understand train drivers' risk evaluation processes when aiming to enhance the safety of both human and automatic train operation.

cs.HC↗

arXiv2024

How explainable AI affects human performance: A systematic review of the behavioural consequences of saliency maps

Saliency maps can explain how deep neural networks classify images. But are they actually useful for humans? The present systematic review of 68 user studies found that while saliency maps can enhance human performance, null effects or even costs are quite common. To investigate what modulates these effects, the empirical outcomes were organised along several factors related to the human tasks, AI performance, XAI methods, images to be classified, human participants and comparison conditions. In image-focused tasks, benefits were less common than in AI-focused tasks, but the effects depended on the specific cognitive requirements. Moreover, benefits were usually restricted to incorrect AI predictions in AI-focused tasks but to correct ones in image-focused tasks. XAI-related factors had surprisingly little impact. The evidence was limited for image- and human-related factors and the effects were highly dependent on the comparison conditions. These findings may support the design of future user studies.

cs.HC↗

arXiv2024

Interpretability is in the eye of the beholder: Human versus artificial classification of image segments generated by humans versus XAI

The evaluation of explainable artificial intelligence is challenging, because automated and human-centred metrics of explanation quality may diverge. To clarify their relationship, we investigated whether human and artificial image classification will benefit from the same visual explanations. In three experiments, we analysed human reaction times, errors, and subjective ratings while participants classified image segments. These segments either reflected human attention (eye movements, manual selections) or the outputs of two attribution methods explaining a ResNet (Grad-CAM, XRAI). We also had this model classify the same segments. Humans and the model largely agreed on the interpretability of attribution methods: Grad-CAM was easily interpretable for indoor scenes and landscapes, but not for objects, while the reverse pattern was observed for XRAI. Conversely, human and model performance diverged for human-generated segments. Our results caution against general statements about interpretability, as it varies with the explanation method, the explained images, and the agent interpreting them.

cs.HC↗

arXiv2024

Adapt/Exchange decisions or generic choices: Does framing influence how people integrate qualitatively different risks?

In complex systems, decision makers often have to consider qualitatively different risks when choosing between options. Do their strategies of integrating these risks depend on the framing of problem contents? In the present study, participants were either instructed that they were choosing between two ways of solving a complex problem, or between two generic options. The former was framed as a modular plant scenario that required choices between modifying parameter settings in a current module (Adapt) and replacing the module by another one (Exchange). The risk was higher for Adapt to harm the product and for Exchange to harm the plant. These risks were presented as probabilities, and participants were either told that the consequences of both risks were equally severe (content-same group), or that harming the plant was much worse (content-different group). A third group made decisions based on the same probabilities, but received a generic task framing (no-content group). We expected framing to affect risk integration, leading the content-same group to make different choices than the no-content group. Contrary to this hypothesis, these two groups were strikingly similar in their decision outcomes and strategies, but clearly differed from the content-different group. These findings question whether ecological validity can be enhanced merely by framing a task in terms of real-world problem contents.

cs.HC↗

arXiv2024

Adapt/Exchange decisions depend on structural and surface features: Effects of solution costs and presentation format

Problem solvers often need to choose between adapting a current solution and exchanging it for a new one. How do such decisions depend on structural and surface features of the task? The present study investigated the interplay between the costs of the two solutions (a structural feature) and the format in which this information was presented (a surface feature). In a computer-based modular plant scenario, participants chose between process parameter modifications (Adapt) and reconfigurations of the module setup (Exchange). Solution costs were presented either as graphs depicting parameter relations, separate numbers for each parameter, or integrated numbers for each solution. It was hypothesised that graphs induce satisficing (i.e., basing decisions only on Adapt), whereas the numeric formats foster a comparison of the solutions (i.e., basing decisions on the Adapt/Exchange ratio). The hypothesised effects were restricted to situations with medium Adapt costs. A second experiment replicated these findings while adjusting the scale of numeric formats. We conclude that Adapt/Exchange decisions are shaped by an interaction of structural and surface features.

cs.HC↗

arXiv2023

Information acquisition in Adapt/Exchange decisions: When do people check alternative solution principles?

Many problems can be solved in two ways: either by adapting an existing solution, or by exchanging it for a new one. To investigate under what conditions people consider new solutions, we traced their information acquisition processes in a simulated mechanical engineering task. Within a multi-step optimisation procedure, participants could either adapt the properties of a currently used machine component, or exchange this component for a new one. They had the opportunity to check whether the solutions met a set of requirements, which was varied systematically. We investigated whether participants would consistently check both solutions, or whether they would satisfice, ignoring the new solution as long as the current one was good enough. The results clearly refuted consistent checking, but only partly confirmed satisficing. On the one hand, participants indeed checked the new solution least often when the current one was applicable without problems. On the other hand, in this case the new solution still was not fully ignored. However, the latter finding could be traced back to a few participants who diverged from our anticipated strategy of first checking the current solution, and directly went for the new one. The results suggest that in Adapt/Exchange decisions, people do not usually check both solutions in an unbiased manner, but rely on existing solutions as long as they are good enough.

cs.HC↗

arXiv2023

The benefits and costs of explainable artificial intelligence in visual quality control: Evidence from fault detection performance and eye movements

Visual inspection tasks often require humans to cooperate with AI-based image classifiers. To enhance this cooperation, explainable artificial intelligence (XAI) can highlight those image areas that have contributed to an AI decision. However, the literature on visual cueing suggests that such XAI support might come with costs of its own. To better understand how the benefits and cost of XAI depend on the accuracy of AI classifications and XAI highlights, we conducted two experiments that simulated visual quality control in a chocolate factory. Participants had to decide whether chocolate moulds contained faulty bars or not, and were always informed whether the AI had classified the mould as faulty or not. In half of the experiment, they saw additional XAI highlights that justified this classification. While XAI speeded up performance, its effects on error rates were highly dependent on (X)AI accuracy. XAI benefits were observed when the system correctly detected and highlighted the fault, but XAI costs were evident for misplaced highlights that marked an intact area while the actual fault was located elsewhere. Eye movement analyses indicated that participants spent less time searching the rest of the mould and thus looked at the fault less often. However, we also observed large interindividual differences. Taken together, the results suggest that despite its potentials, XAI can discourage people from investing effort into their own information analysis.

cs.HC↗

arXiv2023

Do humans and Convolutional Neural Networks attend to similar areas during scene classification: Effects of task and image type

Deep Learning models like Convolutional Neural Networks (CNN) are powerful image classifiers, but what factors determine whether they attend to similar image areas as humans do? While previous studies have focused on technological factors, little is known about the role of factors that affect human attention. In the present study, we investigated how the tasks used to elicit human attention maps interact with image characteristics in modulating the similarity between humans and CNN. We varied the intentionality of human tasks, ranging from spontaneous gaze during categorization over intentional gaze-pointing up to manual area selection. Moreover, we varied the type of image to be categorized, using either singular, salient objects, indoor scenes consisting of object arrangements, or landscapes without distinct objects defining the category. The human attention maps generated in this way were compared to the CNN attention maps revealed by explainable artificial intelligence (Grad-CAM). The influence of human tasks strongly depended on image type: For objects, human manual selection produced maps that were most similar to CNN, while the specific eye movement task has little impact. For indoor scenes, spontaneous gaze produced the least similarity, while for landscapes, similarity was equally low across all human tasks. To better understand these results, we also compared the different human attention maps to each other. Our results highlight the importance of taking human factors into account when comparing the attention of humans and CNN.

cs.CV↗

arXiv2021

Data in context: How digital transformation can support human reasoning in cyber-physical production systems

In traditional production plants, current technologies do not provide sufficient context to support information integration and interpretation. Digital transformation technologies have the potential to support contextualization, but it is unclear how this can be achieved. The present article reviews psychological literature in four areas relevant to contextualization: information sampling, integration, categorization, and causal reasoning. Characteristic biases and limitations of human information processing are discussed. Based on this literature, we derive functional requirements for digital transformation technologies, focusing on the cognitive activities they should support. We then present a selection of technologies that have the potential to foster contextualization. These technologies enable the modelling of system relations, the integration of data from different sources, and the connection of the present situation with historical data. We illustrate how these technologies can support contextual reasoning and highlight challenges that should be addressed when designing human-technology cooperation in cyber-physical production systems.

cs.HC↗