arXiv subjects

Alexander Schlaefer

Publications and source records attributed to Alexander Schlaefer.

At least 73 records · Page 4Linked to original sources

arXiv2020

4D Spatio-Temporal Deep Learning with 4D fMRI Data for Autism Spectrum Disorder Classification

Autism spectrum disorder (ASD) is associated with behavioral and communication problems. Often, functional magnetic resonance imaging (fMRI) is used to detect and characterize brain changes related to the disorder. Recently, machine learning methods have been employed to reveal new patterns by trying to classify ASD from spatio-temporal fMRI images. Typically, these methods have either focused on temporal or spatial information processing. Instead, we propose a 4D spatio-temporal deep learning approach for ASD classification where we jointly learn from spatial and temporal data. We employ 4D convolutional neural networks and convolutional-recurrent models which outperform a previous approach with an F1-score of 0.71 compared to an F1-score of 0.65.

eess.IV↗

arXiv2020

Melanoma detection with electrical impedance spectroscopy and dermoscopy using joint deep learning models

The initial assessment of skin lesions is typically based on dermoscopic images. As this is a difficult and time-consuming task, machine learning methods using dermoscopic images have been proposed to assist human experts. Other approaches have studied electrical impedance spectroscopy (EIS) as a basis for clinical decision support systems. Both methods represent different ways of measuring skin lesion properties as dermoscopy relies on visible light and EIS uses electric currents. Thus, the two methods might carry complementary features for lesion classification. Therefore, we propose joint deep learning models considering both EIS and dermoscopy for melanoma detection. For this purpose, we first study machine learning methods for EIS that incorporate domain knowledge and previously used heuristics into the design process. As a result, we propose a recurrent model with state-max-pooling which automatically learns the relevance of different EIS measurements. Second, we combine this new model with different convolutional neural networks that process dermoscopic images. We study ensembling approaches and also propose a cross-attention module guiding information exchange between the EIS and dermoscopy model. In general, combinations of EIS and dermoscopy clearly outperform models that only use either EIS or dermoscopy. We show that our attention-based, combined model outperforms other models with specificities of 34.4% (CI 31.3-38.4), 34.7% (CI 31.0-38.8) and 53.7% (CI 50.1-57.6) for dermoscopy, EIS and the combined model, respectively, at a clinically relevant sensitivity of 98%.

cs.CV↗

arXiv2020

Learning Preference-Based Similarities from Face Images using Siamese Multi-Task CNNs

Online dating has become a common occurrence over the last few decades. A key challenge for online dating platforms is to determine suitable matches for their users. A lot of dating services rely on self-reported user traits and preferences for matching. At the same time, some services largely rely on user images and thus initial visual preference. Especially for the latter approach, previous research has attempted to capture users' visual preferences for automatic match recommendation. These approaches are mostly based on the assumption that physical attraction is the key factor for relationship formation and personal preferences, interests, and attitude are largely neglected. Deep learning approaches have shown that a variety of properties can be predicted from human faces to some degree, including age, health and even personality traits. Therefore, we investigate the feasibility of bridging image-based matching and matching with personal interests, preferences, and attitude. We approach the problem in a supervised manner by predicting similarity scores between two users based on images of their faces only. The ground-truth for the similarity matching scores is determined by a test that aims to capture users' preferences, interests, and attitude that are relevant for forming romantic relationships. The images are processed by a Siamese Multi-Task deep learning architecture. We find a statistically significant correlation between predicted and target similarity scores. Thus, our results indicate that learning similarities in terms of interests, preferences, and attitude from face images appears to be feasible to some degree.

cs.CV↗

arXiv2020

High-Speed Markerless Tissue Motion Tracking Using Volumetric Optical Coherence Tomography Images

Modern optical coherence tomography (OCT) devices provide volumetric images with micrometer-scale spatial resolution and a temporal resolution beyond video rate. In this work, we analyze an OCT-based prototypical tracking system which processes 831 volumes per second, estimates translational motion, and automatically adjusts the field-of-view, which has a size of few millimeters, to follow a sample even along larger distances. The adjustment is realized by two galvo mirrors and a motorized reference arm, such that no mechanical movement of the scanning setup is necessary. Without requiring a marker or any other knowledge about the sample, we demonstrate that reliable tracking of velocities up to 25 mm/s is possible with mean tracking errors in the order of 0.25 mm. Further, we report successful tracking of lateral velocities up to 70 mm/s with errors below 0.3 mm.

eess.IV↗

arXiv2019

In-Vitro MPI-Guided IVOCT Catheter Tracking in Real Time for Motion Artifact Compensation

Purpose: Using 4D magnetic particle imaging (MPI), intravascular optical coherence tomography (IVOCT) catheters are tracked in real time in order to compensate for image artifacts related to relative motion. Our approach demonstrates the feasibility for bimodal IVOCT and MPI in-vitro experiments. Material and Methods: During IVOCT imaging of a stenosis phantom the catheter is tracked using MPI. A 4D trajectory of the catheter tip is determined from the MPI data using center of mass sub-voxel strategies. A custom built IVOCT imaging adapter is used to perform different catheter motion profiles: no motion artifacts, motion artifacts due to catheter bending, and heart beat motion artifacts. Two IVOCT volume reconstruction methods are compared qualitatively and quantitatively using the DICE metric and the known stenosis length. Results: The MPI-tracked trajectory of the IVOCT catheter is validated in multiple repeated measurements calculating the absolute mean error and standard deviation. Both volume reconstruction methods are compared and analyzed whether they are capable of compensating the motion artifacts. The novel approach of MPI-guided catheter tracking corrects motion artifacts leading to a DICE coefficient with a minimum of 86% in comparison to 58% for a standard reconstruction approach. Conclusions: IVOCT catheter tracking with MPI in real time is an auspicious method for radiation free MPI-guided IVOCT interventions. The combination of MPI and IVOCT can help to reduce motion artifacts due to catheter bending and heart beat for optimized IVOCT volume reconstructions.

physics.med-ph↗

arXiv2019

Bimodal intravascular volumetric imaging combining OCT and MPI

Intravascular optical coherence tomography (IVOCT) is a catheter based image modality allowing for high resolution imaging of vessels. It is based on a fast sequential acquisition of A-scans with an axial spatial resolution in the range of 5 to 10 μm, i.e., one order of magnitude higher than in conventional methods like intravascular ultrasound or computed tomography angiography. However, position and orientation of the catheter in patient coordinates cannot be obtained from the IVOCT measurements alone. Hence, the pose of the catheter needs to be established to correctly reconstruct the three-dimensional vessel shape. Magnetic particle imaging (MPI) is a three-dimensional tomographic, tracer-based and radiation-free image modality providing high temporal resolution with unlimited penetration depth. Volumetric MPI images are angiographic and hence suitable to complement IVOCT as a co-modality. We study simultaneous bimodal IVOCT MPI imaging with the goal of estimating the IVOCT pullback path based on the 3D MPI data. We present a setup to study and evaluate simultaneous IVOCT and MPI image acquisition of differently shaped vessel phantoms. First, the infuence of the MPI tracer concentration on the optical properties required for IVOCT is analyzed. Second, using a concentration allowing for simultaneous imaging, IVOCT and MPI image data is acquired sequentially and simultaneously. Third, the luminal centerline is established from the MPI image volumes and used to estimate the catheter pullback trajectory for IVOCT image reconstruction. The image volumes are compared to the known shape of the phantoms. We were able to identify a suitable MPI tracer concentration of 2.5 mmol/L with negligible influence on the IVOCT signal. The pullback trajectory estimated from MPI agrees well with the centerline of the phantoms. (...)

physics.med-ph↗

arXiv2019

Skin Lesion Classification Using Ensembles of Multi-Resolution EfficientNets with Meta Data

In this paper, we describe our method for the ISIC 2019 Skin Lesion Classification Challenge. The challenge comes with two tasks. For task 1, skin lesions have to be classified based on dermoscopic images. For task 2, dermoscopic images and additional patient meta data have to be used. A diverse dataset of 25000 images was provided for training, containing images from eight classes. The final test set contains an additional, unknown class. We address this challenging problem with a simple, data driven approach by including external data with skin lesions types that are not present in the training set. Furthermore, multi-class skin lesion classification comes with the problem of severe class imbalance. We try to overcome this problem by using loss balancing. Also, the dataset contains images with very different resolutions. We take care of this property by considering different model input resolutions and different cropping strategies. To incorporate meta data such as age, anatomical site, and sex, we use an additional dense neural network and fuse its features with the CNN. We aggregate all our models with an ensembling strategy where we search for the optimal subset of models. Our best ensemble achieves a balanced accuracy of 74.2% using five-fold cross-validation. On the official test set our method is ranked first for both tasks with a balanced accuracy of 63.6% for task 1 and 63.4% for task 2.

cs.CV↗

arXiv2019

Towards Deep Learning-Based EEG Electrode Detection Using Automatically Generated Labels

Electroencephalography (EEG) allows for source measurement of electrical brain activity. Particularly for inverse localization, the electrode positions on the scalp need to be known. Often, systems such as optical digitizing scanners are used for accurate localization with a stylus. However, the approach is time-consuming as each electrode needs to be scanned manually and the scanning systems are expensive. We propose using an RGBD camera to directly track electrodes in the images using deep learning methods. Studying and evaluating deep learning methods requires large amounts of labeled data. To overcome the time-consuming data annotation, we generate a large number of ground-truth labels using a robotic setup. We demonstrate that deep learning-based electrode detection is feasible with a mean absolute error of 5.69 +- 6.1mm and that our annotation scheme provides a useful environment for studying deep learning methods for electrode detection.

cs.CV↗

arXiv2019

Left Ventricle Quantification Using Direct Regression with Segmentation Regularization and Ensembles of Pretrained 2D and 3D CNNs

Cardiac left ventricle (LV) quantification provides a tool for diagnosing cardiac diseases. Automatic calculation of all relevant LV indices from cardiac MR images is an intricate task due to large variations among patients and deformation during the cardiac cycle. Typical methods are based on segmentation of the myocardium or direct regression from MR images. To consider cardiac motion and deformation, recurrent neural networks and spatio-temporal convolutional neural networks (CNNs) have been proposed. We study an approach combining state-of-the-art models and emphasizing transfer learning to account for the small dataset provided for the LVQuan19 challenge. We compare 2D spatial and 3D spatio-temporal CNNs for LV indices regression and cardiac phase classification. To incorporate segmentation information, we propose an architecture-independent segmentation-based regularization. To improve the robustness further, we employ a search scheme that identifies the optimal ensemble from a set of architecture variants. Evaluating on the LVQuan19 Challenge training dataset with 5-fold cross-validation, we achieve mean absolute errors of 111 +- 76mm^2, 1.84 +- 0.9mm and 1.22 +- 0.6mm for area, dimension and regional wall thickness regression, respectively. The error rate for cardiac phase classification is 6.7%.

eess.IV↗

arXiv2019

Spatio-Temporal Deep Learning Models for Tip Force Estimation During Needle Insertion

Purpose. Precise placement of needles is a challenge in a number of clinical applications such as brachytherapy or biopsy. Forces acting at the needle cause tissue deformation and needle deflection which in turn may lead to misplacement or injury. Hence, a number of approaches to estimate the forces at the needle have been proposed. Yet, integrating sensors into the needle tip is challenging and a careful calibration is required to obtain good force estimates. Methods. We describe a fiber-optical needle tip force sensor design using a single OCT fiber for measurement. The fiber images the deformation of an epoxy layer placed below the needle tip which results in a stream of 1D depth profiles. We study different deep learning approaches to facilitate calibration between this spatio-temporal image data and the related forces. In particular, we propose a novel convGRU-CNN architecture for simultaneous spatial and temporal data processing. Results. The needle can be adapted to different operating ranges by changing the stiffness of the epoxy layer. Likewise, calibration can be adapted by training the deep learning models. Our novel convGRU-CNN architecture results in the lowest mean absolute error of 1.59 +- 1.3 mN and a cross-correlation coefficient of 0.9997, and clearly outperforms the other methods. Ex vivo experiments in human prostate tissue demonstrate the needle's application. Conclusions. Our OCT-based fiber-optical sensor presents a viable alternative for needle tip force estimation. The results indicate that the rich spatio-temporal information included in the stream of images showing the deformation throughout the epoxy layer can be effectively used by deep learning models. Particularly, we demonstrate that the convGRU-CNN architecture performs favorably, making it a promising approach for other spatio-temporal learning problems.

eess.IV↗

arXiv2019

Deep Transfer Learning Methods for Colon Cancer Classification in Confocal Laser Microscopy Images

Purpose: The gold standard for colorectal cancer metastases detection in the peritoneum is histological evaluation of a removed tissue sample. For feedback during interventions, real-time in-vivo imaging with confocal laser microscopy has been proposed for differentiation of benign and malignant tissue by manual expert evaluation. Automatic image classification could improve the surgical workflow further by providing immediate feedback. Methods: We analyze the feasibility of classifying tissue from confocal laser microscopy in the colon and peritoneum. For this purpose, we adopt both classical and state-of-the-art convolutional neural networks to directly learn from the images. As the available dataset is small, we investigate several transfer learning strategies including partial freezing variants and full fine-tuning. We address the distinction of different tissue types, as well as benign and malignant tissue. Results: We present a thorough analysis of transfer learning strategies for colorectal cancer with confocal laser microscopy. In the peritoneum, metastases are classified with an AUC of 97.1 and in the colon, the primarius is classified with an AUC of 73.1. In general, transfer learning substantially improves performance over training from scratch. We find that the optimal transfer learning strategy differs for models and classification tasks. Conclusions: We demonstrate that convolutional neural networks and transfer learning can be used to identify cancer tissue with confocal laser microscopy. We show that there is no generally optimal transfer learning strategy and model as well as task-specific engineering is required. Given the high performance for the peritoneum, even with a small dataset, application for intraoperative decision support could be feasible.

cs.CV↗

arXiv2019

Skin Lesion Classification Using CNNs with Patch-Based Attention and Diagnosis-Guided Loss Weighting

Objective: This work addresses two key problems of skin lesion classification. The first problem is the effective use of high-resolution images with pretrained standard architectures for image classification. The second problem is the high class imbalance encountered in real-world multi-class datasets. Methods: To use high-resolution images, we propose a novel patch-based attention architecture that provides global context between small, high-resolution patches. We modify three pretrained architectures and study the performance of patch-based attention. To counter class imbalance problems, we compare oversampling, balanced batch sampling, and class-specific loss weighting. Additionally, we propose a novel diagnosis-guided loss weighting method which takes the method used for ground-truth annotation into account. Results: Our patch-based attention mechanism outperforms previous methods and improves the mean sensitivity by 7%. Class balancing significantly improves the mean sensitivity and we show that our diagnosis-guided loss weighting method improves the mean sensitivity by 3% over normal loss balancing. Conclusion: The novel patch-based attention mechanism can be integrated into pretrained architectures and provides global context between local patches while outperforming other patch-based methods. Hence, pretrained architectures can be readily used with high-resolution images without downsampling. The new diagnosis-guided loss weighting method outperforms other methods and allows for effective training when facing class imbalance. Significance: The proposed methods improve automatic skin lesion classification. They can be extended to other clinical applications where high-resolution image data and class imbalance are relevant.

cs.CV↗

arXiv2019

Efficient Neural Architecture Search on Low-Dimensional Data for OCT Image Segmentation

Typically, deep learning architectures are handcrafted for their respective learning problem. As an alternative, neural architecture search (NAS) has been proposed where the architecture's structure is learned in an additional optimization step. For the medical imaging domain, this approach is very promising as there are diverse problems and imaging modalities that require architecture design. However, NAS is very time-consuming and medical learning problems often involve high-dimensional data with high computational requirements. We propose an efficient approach for NAS in the context of medical, image-based deep learning problems by searching for architectures on low-dimensional data which are subsequently transferred to high-dimensional data. For OCT-based layer segmentation, we demonstrate that a search on 1D data reduces search time by 87.5% compared to a search on 2D data while the final 2D models achieve similar performance.

eess.IV↗

arXiv2019

Endoscopic vs. volumetric OCT imaging of mastoid bone structure for pose estimation in minimally invasive cochlear implant surgery

Purpose: The facial recess is a delicate structure that must be protected in minimally invasive cochlear implant surgery. Current research estimates the drill trajectory by using endoscopy of the unique mastoid patterns. However, missing depth information limits available features for a registration to preoperative CT data. Therefore, this paper evaluates OCT for enhanced imaging of drill holes in mastoid bone and compares OCT data to original endoscopic images. Methods: A catheter-based OCT probe is inserted into a drill trajectory of a mastoid phantom in a translation-rotation manner to acquire the inner surface state. The images are undistorted and stitched to create volumentric data of the drill hole. The mastoid cell pattern is segmented automatically and compared to ground truth. Results: The mastoid pattern segmented on images acquired with OCT show a similarity of J = 73.6 % to ground truth based on endoscopic images and measured with the Jaccard metric. Leveraged by additional depth information, automated segmentation tends to be more robust and fail-safe compared to endoscopic images. Conclusion: The feasibility of using a clinically approved OCT probe for imaging the drill hole in cochlear implantation is shown. The resulting volumentric images provide additional information on the shape of caveties in the bone structure, which will be useful for image-to-patient registration and to estimate the drill trajectory. This will be another step towards safe minimally invasive cochlear implantation.

cs.CV↗

arXiv2019

Towards Automatic Lesion Classification in the Upper Aerodigestive Tract Using OCT and Deep Transfer Learning Methods

Early detection of cancer is crucial for treatment and overall patient survival. In the upper aerodigestive tract (UADT) the gold standard for identification of malignant tissue is an invasive biopsy. Recently, non-invasive imaging techniques such as confocal laser microscopy and optical coherence tomography (OCT) have been used for tissue assessment. In particular, in a recent study experts classified lesions in the UADT with respect to their invasiveness using OCT images only. As the results were promising, automatic classification of lesions might be feasible which could assist experts in their decision making. Therefore, we address the problem of automatic lesion classification from OCT images. This task is very challenging as the available dataset is extremely small and the data quality is limited. However, as similar issues are typical in many clinical scenarios we study to what extent deep learning approaches can still be trained and used for decision support.

cs.CV↗

arXiv2019

Feasibility of a markerless tracking system based on optical coherence tomography

Clinical tracking systems are popular but typically require specific tracking markers. During the last years, scanning speed of optical coherence tomography (OCT) has increased to A-scan rates above 1 MHz allowing to acquire volume scans of moving objects. Thorefore, we propose a markerless tracking system based on OCT to obtain small volumetric images including information of sub-surface structures at high spatio-temporal resolution. In contrast to conventional vision based approaches, this allows identifying natural landmarks even for smooth and homogeneous surfaces. We describe the optomechanical setup and process flow to evaluate OCT volumes for translations and accordingly adjust the position of the field-of-view to follow moving samples. While our current setup is still preliminary, we demonstrate tracking of motion transversal to the OCT beam of up to 20 mm/s with errors around 0.2 mm and even better for some scenarios. Tracking is evaluated on a clearly structured and on a homogeneous phantom as well as on actual tissue samples. The results show that OCT is promising for fast and precise tracking of smooth, monochromatic objects in medical scenarios.

physics.med-ph↗

arXiv2018

Feasibility of Colon Cancer Detection in Confocal Laser Microscopy Images Using Convolution Neural Networks

Histological evaluation of tissue samples is a typical approach to identify colorectal cancer metastases in the peritoneum. For immediate assessment, reliable and real-time in-vivo imaging would be required. For example, intraoperative confocal laser microscopy has been shown to be suitable for distinguishing organs and also malignant and benign tissue. So far, the analysis is done by human experts. We investigate the feasibility of automatic colon cancer classification from confocal laser microscopy images using deep learning models. We overcome very small dataset sizes through transfer learning with state-of-the-art architectures. We achieve an accuracy of 89.1% for cancer detection in the peritoneum which indicates viability as an intraoperative decision support system.

cs.CV↗

arXiv2018

Bioresorbable Scaffold Visualization in IVOCT Images Using CNNs and Weakly Supervised Localization

Bioresorbable scaffolds have become a popular choice for treatment of coronary heart disease, replacing traditional metal stents. Often, intravascular optical coherence tomography is used to assess potential malapposition after implantation and for follow-up examinations later on. Typically, the scaffold is manually reviewed by an expert, analyzing each of the hundreds of image slices. As this is time consuming, automatic stent detection and visualization approaches have been proposed, mostly for metal stent detection based on classic image processing. As bioresorbable scaffolds are harder to detect, recent approaches have used feature extraction and machine learning methods for automatic detection. However, these methods require detailed, pixel-level labels in each image slice and extensive feature engineering for the particular stent type which might limit the approaches' generalization capabilities. Therefore, we propose a deep learning-based method for bioresorbable scaffold visualization using only image-level labels. A convolutional neural network is trained to predict whether an image slice contains a metal stent, a bioresorbable scaffold, or no device. Then, we derive local stent strut information by employing weakly supervised localization using saliency maps with guided backpropagation. As saliency maps are generally diffuse and noisy, we propose a novel patch-based method with image shifting which allows for high resolution stent visualization. Our convolutional neural network model achieves a classification accuracy of 99.0 % for image-level stent classification which can be used for both high quality in-slice stent visualization and 3D rendering of the stent structure.

cs.CV↗