arXiv subjects

Dirk Valkenborg

Publications and source records attributed to Dirk Valkenborg.

3 recordsLinked to original sources

arXiv2024

Evaluating Feature Attribution Methods in the Image Domain

Feature attribution maps are a popular approach to highlight the most important pixels in an image for a given prediction of a model. Despite a recent growth in popularity and available methods, little attention is given to the objective evaluation of such attribution maps. Building on previous work in this domain, we investigate existing metrics and propose new variants of metrics for the evaluation of attribution maps. We confirm a recent finding that different attribution metrics seem to measure different underlying concepts of attribution maps, and extend this finding to a larger selection of attribution metrics. We also find that metric results on one dataset do not necessarily generalize to other datasets, and methods with desirable theoretical properties such as DeepSHAP do not necessarily outperform computationally cheaper alternatives. Based on these findings, we propose a general benchmarking approach to identify the ideal feature attribution method for a given use case. Implementations of attribution metrics and our experiments are available online.

cs.CV↗

arXiv2020

Post Training Uncertainty Calibration of Deep Networks For Medical Image Segmentation

Neural networks for automated image segmentation are typically trained to achieve maximum accuracy, while less attention has been given to the calibration of their confidence scores. However, well-calibrated confidence scores provide valuable information towards the user. We investigate several post hoc calibration methods that are straightforward to implement, some of which are novel. They are compared to Monte Carlo (MC) dropout. They are applied to neural networks trained with cross-entropy (CE) and soft Dice (SD) losses on BraTS 2018 and ISLES 2018. Surprisingly, models trained on SD loss are not necessarily less calibrated than those trained on CE loss. In all cases, at least one post hoc method improves the calibration. There is limited consistency across the results, so we can't conclude on one method being superior. In all cases, post hoc calibration is competitive with MC dropout. Although average calibration improves compared to the base model, subject-level variance of the calibration remains similar.

eess.IV↗

arXiv2017

Assigning peaks and modeling ETD in top-down mass spectrometry

Among many techniques of modern mass spectrometry, the top down methods are becoming continuously more popular in the overall strive to describe the proteome. These techniques are based on fragmentation of ions inside mass spectrometers instead of being proteolytically digested. In some of these techniques, the fragmentation is induced by electron transfer. It can trigger several concurring reactions: electron transfer dissociation, electron transfer without dissociation, and proton transfer reaction. The evaluation of the extent of these reactions is important for the proper understanding of the functioning of the instrument and, what is even more important, to know if it can be used to reveal important structural information. We present a workflow for assigning peaks and interpreting the results of electron transfer driven reactions. We also present software written in Python and available under GNU v3 license.

stat.AP↗