arXiv subjects

Jan Klein

Publications and source records attributed to Jan Klein.

10 recordsLinked to original sources

arXiv2023

The Optimal Input-Independent Baseline for Binary Classification: The Dutch Draw

Before any binary classification model is taken into practice, it is important to validate its performance on a proper test set. Without a frame of reference given by a baseline method, it is impossible to determine if a score is `good' or `bad'. The goal of this paper is to examine all baseline methods that are independent of feature values and determine which model is the `best' and why. By identifying which baseline models are optimal, a crucial selection decision in the evaluation process is simplified. We prove that the recently proposed Dutch Draw baseline is the best input-independent classifier (independent of feature values) for all positional-invariant measures (independent of sequence order) assuming that the samples are randomly shuffled. This means that the Dutch Draw baseline is the optimal baseline under these intuitive requirements and should therefore be used in practice.

cs.LG↗

arXiv2022

Comparison of different automatic solutions for resection cavity segmentation in postoperative MRI volumes including longitudinal acquisitions

In this work, we compare five deep learning solutions to automatically segment the resection cavity in postoperative MRI. The proposed methods are based on the same 3D U-Net architecture. We use a dataset of postoperative MRI volumes, each including four MRI sequences and the ground truth of the corresponding resection cavity. Four solutions are trained with a different MRI sequence. Besides, a method designed with all the available sequences is also presented. Our experiments show that the method trained only with the T1 weighted contrast-enhanced MRI sequence achieves the best results, with a median DICE index of 0.81.

cs.CV↗

arXiv2022

The Dutch Draw: Constructing a Universal Baseline for Binary Prediction Models

Novel prediction methods should always be compared to a baseline to know how well they perform. Without this frame of reference, the performance score of a model is basically meaningless. What does it mean when a model achieves an $F_1$ of 0.8 on a test set? A proper baseline is needed to evaluate the `goodness' of a performance score. Comparing with the latest state-of-the-art model is usually insightful. However, being state-of-the-art can change rapidly when newer models are developed. Contrary to an advanced model, a simple dummy classifier could be used. However, the latter could be beaten too easily, making the comparison less valuable. This paper presents a universal baseline method for all binary classification models, named the Dutch Draw (DD). This approach weighs simple classifiers and determines the best classifier to use as a baseline. We theoretically derive the DD baseline for many commonly used evaluation measures and show that in most situations it reduces to (almost) always predicting either zero or one. Summarizing, the DD baseline is: (1) general, as it is applicable to all binary classification problems; (2) simple, as it is quickly determined without training or parameter-tuning; (3) informative, as insightful conclusions can be drawn from the results. The DD baseline serves two purposes. First, to enable comparisons across research papers by this robust and universal baseline. Secondly, to provide a sanity check during the development process of a prediction model. It is a major warning sign when a model is outperformed by the DD baseline.

cs.LG↗

arXiv2021

The Brain Tumor Sequence Registration (BraTS-Reg) Challenge: Establishing Correspondence Between Pre-Operative and Follow-up MRI Scans of Diffuse Glioma Patients

Registration of longitudinal brain MRI scans containing pathologies is challenging due to dramatic changes in tissue appearance. Although there has been progress in developing general-purpose medical image registration techniques, they have not yet attained the requisite precision and reliability for this task, highlighting its inherent complexity. Here we describe the Brain Tumor Sequence Registration (BraTS-Reg) challenge, as the first public benchmark environment for deformable registration algorithms focusing on estimating correspondences between pre-operative and follow-up scans of the same patient diagnosed with a diffuse brain glioma. The BraTS-Reg data comprise de-identified multi-institutional multi-parametric MRI (mpMRI) scans, curated for size and resolution according to a canonical anatomical template, and divided into training, validation, and testing sets. Clinical experts annotated ground truth (GT) landmark points of anatomical locations distinct across the temporal domain. Quantitative evaluation and ranking were based on the Median Euclidean Error (MEE), Robustness, and the determinant of the Jacobian of the displacement field. The top-ranked methodologies yielded similar performance across all evaluation metrics and shared several methodological commonalities, including pre-alignment, deep neural networks, inverse consistency analysis, and test-time instance optimization per-case basis as a post-processing step. The top-ranked method attained the MEE at or below that of the inter-rater variability for approximately 60% of the evaluated landmarks, underscoring the scope for further accuracy and robustness improvements, especially relative to human experts. The aim of BraTS-Reg is to continue to serve as an active resource for research, with the data and online evaluation tools accessible at https://bratsreg.github.io/.

eess.IV↗

arXiv2021

Jasmine: A New Active Learning Approach to Combat Cybercrime

Over the past decade, the advent of cybercrime has accelarated the research on cybersecurity. However, the deployment of intrusion detection methods falls short. One of the reasons for this is the lack of realistic evaluation datasets, which makes it a challenge to develop techniques and compare them. This is caused by the large amounts of effort it takes for a cyber analyst to classify network connections. This has raised the need for methods (i) that can learn from small sets of labeled data, (ii) that can make predictions on large sets of unlabeled data, and (iii) that request the label of only specially selected unlabeled data instances. Hence, Active Learning (AL) methods are of interest. These approaches choose specific unlabeled instances by a query function that are expected to improve overall classification performance. The resulting query observations are labeled by a human expert and added to the labeled set. In this paper, we propose a new hybrid AL method called Jasmine. Firstly, it determines how suitable each observation is for querying, i.e., how likely it is to enhance classification. These properties are the uncertainty score and anomaly score. Secondly, Jasmine introduces dynamic updating. This allows the model to adjust the balance between querying uncertain, anomalous and randomly selected observations. To this end, Jasmine is able to learn the best query strategy during the labeling process. This is in contrast to the other AL methods in cybersecurity that all have static, predetermined query functions. We show that dynamic updating, and therefore Jasmine, is able to consistently obtain good and more robust results than querying only uncertainties, only anomalies or a fixed combination of the two.

cs.CR↗

arXiv2020

Registration of ultrasound volumes based on Euclidean distance transform

During neurosurgical operations, surgeons can decide to acquire intraoperative data to better proceed with the removal of a tumor. A valid option is given by ultrasound (US) imaging, which can be easily obtained at subsequent surgical stages, giving therefore multiple updates of the resection cavity. To improve the efficacy of the intraoperative guidance, neurosurgeons may benefit from having a direct correspondence between anatomical structures identified at different US acquisitions. In this context, the commonly available neuronavigation systems already provide registration methods, which however are not enough accurate to overcome the anatomical changes happening during resection. Therefore, our aim with this work is to improve the registration of intraoperative US volumes. In the proposed methodology, first a distance mapping of automatically segmented anatomical structures is computed and then the transformed images are utilized in the registration step. Our solution is tested on a public dataset of 17 cases, where the average landmark registration error between volumes acquired at the beginning and at the end of neurosurgical procedures is reduced from 3.55mm to 1.27mm.

eess.IV↗

arXiv2013

A Ray-based Approach for Boundary Estimation of Fiber Bundles Derived from Diffusion Tensor Imaging

Diffusion Tensor Imaging (DTI) is a non-invasive imaging technique that allows estimation of the location of white matter tracts in-vivo, based on the measurement of water diffusion properties. For each voxel, a second-order tensor can be calculated by using diffusion-weighted sequences (DWI) that are sensitive to the random motion of water molecules. Given at least 6 diffusion-weighted images with different gradients and one unweighted image, the coefficients of the symmetric diffusion tensor matrix can be calculated. Deriving the eigensystem of the tensor, the eigenvectors and eigenvalues can be calculated to describe the three main directions of diffusion and its magnitude. Using DTI data, fiber bundles can be determined, to gain information about eloquent brain structures. Especially in neurosurgery, information about location and dimension of eloquent structures like the corticospinal tract or the visual pathways is of major interest. Therefore, the fiber bundle boundary has to be determined. In this paper, a novel ray-based approach for boundary estimation of tubular structures is presented.

cs.CV↗

arXiv2011

Benchmarking the Quality of Diffusion-Weighted Images

We present a novel method that allows for measuring the quality of diffusion-weighted MR images dependent on the image resolution and the image noise. For this purpose, we introduce a new thresholding technique so that noise and the signal can automatically be estimated from a single data set. Thus, no user interaction as well as no double acquisition technique, which requires a time-consuming proper geometrical registration, is needed. As a coarser image resolution or slice thickness leads to a higher signal-to-noise ratio (SNR), our benchmark determines a resolution-independent quality measure so that images with different resolutions can be adequately compared. To evaluate our method, a set of diffusion-weighted images from different vendors is used. It is shown that the quality can efficiently be determined and that the automatically computed SNR is comparable to the SNR which is measured manually in a manually selected region of interest.

cs.CV↗

arXiv2011

Ray-Based and Graph-Based Methods for Fiber Bundle Boundary Estimation

Diffusion Tensor Imaging (DTI) provides the possibility of estimating the location and course of eloquent structures in the human brain. Knowledge about this is of high importance for preoperative planning of neurosurgical interventions and for intraoperative guidance by neuronavigation in order to minimize postoperative neurological deficits. Therefore, the segmentation of these structures as closed, three-dimensional object is necessary. In this contribution, two methods for fiber bundle segmentation between two defined regions are compared using software phantoms (abstract model and anatomical phantom modeling the right corticospinal tract). One method uses evaluation points from sampled rays as candidates for boundary points, the other method sets up a directed and weighted (depending on a scalar measure) graph and performs a min-cut for optimal segmentation results. Comparison is done by using the Dice Similarity Coefficient (DSC), a measure for spatial overlap of different segmentation results.

cs.CV↗

arXiv2011

A Semi-Automatic Graph-Based Approach for Determining the Boundary of Eloquent Fiber Bundles in the Human Brain

Diffusion Tensor Imaging (DTI) allows estimating the position, orientation and dimension of bundles of nerve pathways. This non-invasive imaging technique takes advantage of the diffusion of water molecules and determines the diffusion coefficients for every voxel of the data set. The identification of the diffusion coefficients and the derivation of information about fiber bundles is of major interest for planning and performing neurosurgical interventions. To minimize the risk of neural deficits during brain surgery as tumor resection (e.g. glioma), the segmentation and integration of the results in the operating room is of prime importance. In this contribution, a robust and efficient graph-based approach for segmentating tubular fiber bundles in the human brain is presented. To define a cost function, the fractional anisotropy (FA) is used, derived from the DTI data, but this value may differ from patient to patient. Besides manually definining seed regions describing the structure of interest, additionally a manual definition of the cost function by the user is necessary. To improve the approach the contribution introduces a solution for automatically determining the cost function by using different 3D masks for each individual data set.

cs.CV↗