arXiv subjects

Martin Welk

Publications and source records attributed to Martin Welk.

At least 19 recordsLinked to original sources

arXiv2025

XSRD-Net: EXplainable Stroke Relapse Detection

Stroke is the second most frequent cause of death world wide with an annual mortality of around 5.5 million. Recurrence rates of stroke are between 5 and 25% in the first year. As mortality rates for relapses are extraordinarily high (40%) it is of utmost importance to reduce the recurrence rates. We address this issue by detecting patients at risk of stroke recurrence at an early stage in order to enable appropriate therapy planning. To this end we collected 3D intracranial CTA image data and recorded concomitant heart diseases, the age and the gender of stroke patients between 2010 and 2024. We trained single- and multimodal deep learning based neural networks for binary relapse detection (Task 1) and for relapse free survival (RFS) time prediction together with a subsequent classification (Task 2). The separation of relapse from non-relapse patients (Task 1) could be solved with tabular data (AUC on test dataset: 0.84). However, for the main task, the regression (Task 2), our multimodal XSRD-net processed the modalities vision:tabular with 0.68:0.32 according to modality contribution measures. The c-index with respect to relapses for the multimodal model reached 0.68, and the AUC is 0.71 for the test dataset. Final, deeper interpretability analysis results could highlight a link between both heart diseases (tabular) and carotid arteries (vision) for the detection of relapses and the prediction of the RFS time. This is a central outcome that we strive to strengthen with ongoing data collection and model retraining.

cs.CV↗

arXiv2025

What are You Looking at? Modality Contribution in Multimodal Medical Deep Learning

Purpose High dimensional, multimodal data can nowadays be analyzed by huge deep neural networks with little effort. Several fusion methods for bringing together different modalities have been developed. Given the prevalence of high-dimensional, multimodal patient data in medicine, the development of multimodal models marks a significant advancement. However, how these models process information from individual sources in detail is still underexplored. Methods To this end, we implemented an occlusion-based modality contribution method that is both model- and performance-agnostic. This method quantitatively measures the importance of each modality in the dataset for the model to fulfill its task. We applied our method to three different multimodal medical problems for experimental purposes. Results Herein we found that some networks have modality preferences that tend to unimodal collapses, while some datasets are imbalanced from the ground up. Moreover, we provide fine-grained quantitative and visual attribute importance for each modality. Conclusion Our metric offers valuable insights that can support the advancement of multimodal model development and dataset creation. By introducing this method, we contribute to the growing field of interpretability in deep learning for multimodal research. This approach helps to facilitate the integration of multimodal AI into clinical practice. Our code is publicly available at https://github.com/ChristianGappGit/MC_MMD.

cs.CV↗

arXiv2024

Multimodal Medical Disease Classification with LLaMA II

Medical patient data is always multimodal. Images, text, age, gender, histopathological data are only few examples for different modalities in this context. Processing and integrating this multimodal data with deep learning based methods is of utmost interest due to its huge potential for medical procedure such as diagnosis and patient treatment planning. In this work we retrain a multimodal transformer-based model for disease classification. To this end we use the text-image pair dataset from OpenI consisting of 2D chest X-rays associated with clinical reports. Our focus is on fusion methods for merging text and vision information extracted from medical datasets. Different architecture structures with a LLaMA II backbone model are tested. Early fusion of modality specific features creates better results with the best model reaching 97.10% mean AUC than late fusion from a deeper level of the architecture (best model: 96.67% mean AUC). Both outperform former classification models tested on the same multimodal dataset. The newly introduced multimodal architecture can be applied to other multimodal datasets with little effort and can be easily adapted for further research, especially, but not limited to, the field of medical AI.

cs.AI↗

arXiv2024

Matrix-Valued LogSumExp Approximation for Colour Morphology

Mathematical morphology is a part of image processing that uses a window that moves across the image to change certain pixels according to certain operations. The concepts of supremum and infimum play a crucial role here, but it proves challenging to define them generally for higher-dimensional data, such as colour representations. Numerous approaches have therefore been taken to solve this problem with certain compromises. In this paper we will analyse the construction of a new approach, which we have already presented experimentally in paper [Kahra, M., Breu{\ss}, M., Kleefeld, A., Welk, M., DGMM 2024, pp. 325-337]. This is based on a method by Burgeth and Kleefeld [Burgeth, B., Kleefeld, A., ISMM 2013, pp. 243-254], who regard the colours as symmetric $2\times2$ matrices and compare them by means of the Loewner order in a bi-cone through different suprema. However, we will replace the supremum with the LogExp approximation for the maximum instead. This allows us to transfer the associativity of the dilation from the one-dimensional case to the higher-dimensional case. In addition, we will investigate the minimality property and specify a relaxation to ensure that our approach is continuously dependent on the input data.

cs.CV↗

arXiv2023

An Approach to Colour Morphological Supremum Formation using the LogSumExp Approximation

Mathematical morphology is a part of image processing that has proven to be fruitful for numerous applications. Two main operations in mathematical morphology are dilation and erosion. These are based on the construction of a supremum or infimum with respect to an order over the tonal range in a certain section of the image. The tonal ordering can easily be realised in grey-scale morphology, and some morphological methods have been proposed for colour morphology. However, all of these have certain limitations. In this paper we present a novel approach to colour morphology extending upon previous work in the field based on the Loewner order. We propose to consider an approximation of the supremum by means of a log-sum exponentiation introduced by Maslov. We apply this to the embedding of an RGB image in a field of symmetric $2\times2$ matrices. In this way we obtain nearly isotropic matrices representing colours and the structural advantage of transitivity. In numerical experiments we highlight some remarkable properties of the proposed approach.

cs.CV↗

arXiv2022

Tackling the Class Imbalance Problem of Deep Learning Based Head and Neck Organ Segmentation

The segmentation of organs at risk (OAR) is a required precondition for the cancer treatment with image guided radiation therapy. The automation of the segmentation task is therefore of high clinical relevance. Deep Learning (DL) based medical image segmentation is currently the most successful approach, but suffers from the over-presence of the background class and the anatomically given organ size difference, which is most severe in the head and neck (HAN) area. To tackle the HAN area specific class imbalance problem we first optimize the patch-size of the currently best performing general purpose segmentation framework, the nnU-Net, based on the introduced class imbalance measurement, and second, introduce the class adaptive Dice loss to further compensate for the highly imbalanced setting. Both the patch-size and the loss function are parameters with direct influence on the class imbalance and their optimization leads to a 3\% increase of the Dice score and 22% reduction of the 95% Hausdorff distance compared to the baseline, finally reaching $0.8\pm0.15$ and $3.17\pm1.7$ mm for the segmentation of seven HAN organs using a single and simple neural network. The patch-size optimization and the class adaptive Dice loss are both simply integrable in current DL based segmentation approaches and allow to increase the performance for class imbalanced segmentation tasks.

cs.CV↗

arXiv2021

Multivariate Medians for Image and Shape Analysis

Having been studied since long by statisticians, multivariate median concepts found their way into the image processing literature in the course of the last decades, being used to construct robust and efficient denoising filters for multivariate images such as colour images but also matrix-valued images. Based on the similarities between image and geometric data as results of the sampling of continuous physical quantities, it can be expected that the understanding of multivariate median filters for images provides a starting point for the development of shape processing techniques. This paper presents an overview of multivariate median concepts relevant for image and shape processing. It focusses on their mathematical principles and discusses important properties especially in the context of image processing.

eess.IV↗

arXiv2020

PDE Evolutions for M-Smoothers in One, Two, and Three Dimensions

Local M-smoothers are interesting and important signal and image processing techniques with many connections to other methods. In our paper we derive a family of partial differential equations (PDEs) that result in one, two, and three dimensions as limiting processes from M-smoothers which are based on local order-$p$ means within a ball the radius of which tends to zero. The order $p$ may take any nonzero value $>-1$, allowing also negative values. In contrast to results from the literature, we show in the space-continuous case that mode filtering does not arise for $p \to 0$, but for $p \to -1$. Extending our filter class to $p$-values smaller than $-1$ allows to include e.g. the classical image sharpening flow of Gabor. The PDEs we derive in 1D, 2D, and 3D show large structural similarities. Since our PDE class is highly anisotropic and may contain backward parabolic operators, designing adequate numerical methods is difficult. We present an $L^\infty$-stable explicit finite difference scheme that satisfies a discrete maximum--minimum principle, offers excellent rotation invariance, and employs a splitting into four fractional steps to allow larger time step sizes. Although it approximates parabolic PDEs, it consequently benefits from stabilisation concepts from the numerics of hyperbolic PDEs. Our 2D experiments show that the PDEs for $p<1$ are of specific interest: Their backward parabolic term creates favourable sharpening properties, while they appear to maintain the strong shape simplification properties of mean curvature motion.

eess.IV↗

arXiv2020

Stable Backward Diffusion Models that Minimise Convex Energies

The inverse problem of backward diffusion is known to be ill-posed and highly unstable. Backward diffusion processes appear naturally in image enhancement and deblurring applications. It is therefore greatly desirable to establish a backward diffusion model which implements a smart stabilisation approach that can be used in combination with an easy to handle numerical scheme. So far, existing stabilisation strategies in literature require sophisticated numerics to solve the underlying initial value problem. We derive a class of space-discrete one-dimensional backward diffusion as gradient descent of energies where we gain stability by imposing range constraints. Interestingly, these energies are even convex. Furthermore, we establish a comprehensive theory for the time-continuous evolution and we show that stability carries over to a simple explicit time discretisation of our model. Finally, we confirm the stability and usefulness of our technique in experiments in which we enhance the contrast of digital greyscale and colour images.

math.NA↗

arXiv2016

Multivariate Median Filters and Partial Differential Equations

Multivariate median filters have been proposed as generalisations of the well-established median filter for grey-value images to multi-channel images. As multivariate median, most of the recent approaches use the $L^1$ median, i.e.\ the minimiser of an objective function that is the sum of distances to all input points. Many properties of univariate median filters generalise to such a filter. However, the famous result by Guichard and Morel about approximation of the mean curvature motion PDE by median filtering does not have a comparably simple counterpart for $L^1$ multivariate median filtering. We discuss the affine equivariant Oja median and the affine equivariant transformation--retransformation $L^1$ median as alternatives to $L^1$ median filtering. We analyse multivariate median filters in a space-continuous setting, including the formulation of a space-continuous version of the transformation--retransformation $L^1$ median, and derive PDEs approximated by these filters in the cases of bivariate planar images, three-channel volume images and three-channel planar images. The PDEs for the affine equivariant filters can be interpreted geometrically as combinations of a diffusion and a principal-component-wise curvature motion contribution with a cross-effect term based on torsions of principal components. Numerical experiments are presented that demonstrate the validity of the approximation results.

cs.CV↗

arXiv2015

Graph entropies in texture segmentation of images

We study the applicability of a set of texture descriptors introduced in recent work by the author to texture-based segmentation of images. The texture descriptors under investigation result from applying graph indices from quantitative graph theory to graphs encoding the local structure of images. The underlying graphs arise from the computation of morphological amoebas as structuring elements for adaptive morphology, either as weighted or unweighted Dijkstra search trees or as edge-weighted pixel graphs within structuring elements. In the present paper we focus on texture descriptors in which the graph indices are entropy-based, and use them in a geodesic active contour framework for image segmentation. Experiments on several synthetic and one real-world image are shown to demonstrate texture segmentation by this approach. Forthermore, we undertake an attempt to analyse selected entropy-based texture descriptors with regard to what information about texture they actually encode. Whereas this analysis uses some heuristic assumptions, it indicates that the graph-based texture descriptors are related to fractal dimension measures that have been proven useful in texture analysis.

cs.CV↗

arXiv2015

Amoeba Techniques for Shape and Texture Analysis

Morphological amoebas are image-adaptive structuring elements for morphological and other local image filters introduced by Lerallut et al. Their construction is based on combining spatial distance with contrast information into an image-dependent metric. Amoeba filters show interesting parallels to image filtering methods based on partial differential equations (PDEs), which can be confirmed by asymptotic equivalence results. In computing amoebas, graph structures are generated that hold information about local image texture. This paper reviews and summarises the work of the author and his coauthors on morphological amoebas, particularly their relations to PDE filters and texture analysis. It presents some extensions and points out directions for future investigation on the subject.

cs.CV↗

arXiv2014

Analysis of Amoeba Active Contours

Subject of this paper is the theoretical analysis of structure-adaptive median filter algorithms that approximate curvature-based PDEs for image filtering and segmentation. These so-called morphological amoeba filters are based on a concept introduced by Lerallut et al. They achieve similar results as the well-known geodesic active contour and self-snakes PDEs. In the present work, the PDE approximated by amoeba active contours is derived for a general geometric situation and general amoeba metric. This PDE is structurally similar but not identical to the geodesic active contour equation. It reproduces the previous PDE approximation results for amoeba median filters as special cases. Furthermore, modifications of the basic amoeba active contour algorithm are analysed that are related to the morphological force terms frequently used with geodesic active contours. Experiments demonstrate the basic behaviour of amoeba active contours and its similarity to geodesic active contours.

cs.CV↗

arXiv2013

A Robust Variational Model for Positive Image Deconvolution

In this paper, an iterative method for robust deconvolution with positivity constraints is discussed. It is based on the known variational interpretation of the Richardson-Lucy iterative deconvolution as fixed-point iteration for the minimisation of an information divergence functional under a multiplicative perturbation model. The asymmetric penaliser function involved in this functional is then modified into a robust penaliser, and complemented with a regulariser. The resulting functional gives rise to a fixed point iteration that we call robust and regularised Richardson-Lucy deconvolution. It achieves an image restoration quality comparable to state-of-the-art robust variational deconvolution with a computational efficiency similar to that of the original Richardson-Lucy method. Experiments on synthetic and real-world image data demonstrate the performance of the proposed method.

cs.CV↗

arXiv2013

Algorithmic Optimisations for Iterative Deconvolution Methods

We investigate possibilities to speed up iterative algorithms for non-blind image deconvolution. We focus on algorithms in which convolution with the point-spread function to be deconvolved is used in each iteration, and aim at accelerating these convolution operations as they are typically the most expensive part of the computation. We follow two approaches: First, for some practically important specific point-spread functions, algorithmically efficient sliding window or list processing techniques can be used. In some constellations this allows faster computation than via the Fourier domain. Second, as iterations progress, computation of convolutions can be restricted to subsets of pixels. For moderate thinning rates this can be done with almost no impact on the reconstruction quality. Both approaches are demonstrated in the context of Richardson-Lucy deconvolution but are not restricted to this method.

cs.CV↗

arXiv2012

Fast and Robust Linear Motion Deblurring

We investigate efficient algorithmic realisations for robust deconvolution of grey-value images with known space-invariant point-spread function, with emphasis on 1D motion blur scenarios. The goal is to make deconvolution suitable as preprocessing step in automated image processing environments with tight time constraints. Candidate deconvolution methods are selected for their restoration quality, robustness and efficiency. Evaluation of restoration quality and robustness on synthetic and real-world test images leads us to focus on a combination of Wiener filtering with few iterations of robust and regularised Richardson-Lucy deconvolution. We discuss algorithmic optimisations for specific scenarios. In the case of uniform linear motion blur in coordinate direction, it is possible to achieve real-time performance (less than 50 ms) in single-threaded CPU computation on images of $256\times256$ pixels. For more general space-invariant blur settings, still favourable computation times are obtained. Exemplary parallel implementations demonstrate that the proposed method also achieves real-time performance for general 1D motion blurs in a multi-threaded CPU setting, and for general 2D blurs on a GPU.

cs.CV↗

arXiv2000

Covariant first order differential calculus on quantum Euclidean spheres

We study covariant differential calculus on the quantum spheres S_q^{N-1} which are quantum homogeneous spaces with coactions of the quantum groups O_q(N). The first part of the paper is devoted to first order differential calculus. A classification result is proved which says that for N>=6 there exist exactly two covariant first order differential calculi on S_q^{N-1} which satisfy the classification constraint that the bimodule of one-forms is generated as a free left module by the differentials of the generators of S_q^{N-1}. Both calculi exist also for 3<=N<=5. The same calculi can be constructed using a method introduced by Hermisson. In case N=3, the result is in accordance with the known result by Apel and Schmüdgen for the Podles sphere. In the second part, higher order differential calculus and symmetry are treated. The relations which hold for the two-forms in the universal higher order calculus extending one of the two first order calculi are given. A "braiding" homomorphism is found. The existence of an upper bound for the order of differential forms is discussed.

math.QA↗

arXiv1999

Covariant first order differential calculus on quantum projective spaces

We investigate covariant first order differential calculi on the quantum complex projective spaces CP_q^{N-1} which are quantum homogeneous spaces for the quantum group SU_q(N). Hereby, one more well-studied example of covariant first order differential calculus on a quantum homogeneous space is given. Since the complex projective spaces are subalgebras of the quantum spheres S_q^{2N-1} introduced by Vaksman and Soibelman, we get also an example of the relations between covariant differential calculus on two closely related quantum spaces. Two approaches are combined in obtaining covariant first order differential calculi on CP_q^{N-1}: 1. restriction of covariant first order differential calculi from S_q^{2N-1}; 2. classification of calculi under appropriate constraints, using methods from representation theory. The main result is that under three reasonable settings of dimension constraints, covariant first order differential calculi on CP_q^{N-1} exist and are (for N >= 6) uniquely determined. This is a clear difference as compared to the case of the quantum spheres where several parametrical series of calculi exist. For two of the constraint settings, the covariant first order calculi on CP_q^{N-1} are also obtained by restriction from calculi on S_q^{2N-1} as well as from calculi on the quantum group SU_q(N).

math.QA↗