arXiv subjects

Remco Duits

Publications and source records attributed to Remco Duits.

At least 37 records · Page 2Linked to original sources

arXiv2020

Lesion Segmentation in Ultrasound Using Semi-pixel-wise Cycle Generative Adversarial Nets

Breast cancer is the most common invasive cancer with the highest cancer occurrence in females. Handheld ultrasound is one of the most efficient ways to identify and diagnose the breast cancer. The area and the shape information of a lesion is very helpful for clinicians to make diagnostic decisions. In this study we propose a new deep-learning scheme, semi-pixel-wise cycle generative adversarial net (SPCGAN) for segmenting the lesion in 2D ultrasound. The method takes the advantage of a fully convolutional neural network (FCN) and a generative adversarial net to segment a lesion by using prior knowledge. We compared the proposed method to a fully connected neural network and the level set segmentation method on a test dataset consisting of 32 malignant lesions and 109 benign lesions. Our proposed method achieved a Dice similarity coefficient (DSC) of 0.92 while FCN and the level set achieved 0.90 and 0.79 respectively. Particularly, for malignant lesions, our method increases the DSC (0.90) of the fully connected neural network to 0.93 significantly (p$<$0.001). The results show that our SPCGAN can obtain robust segmentation results. The framework of SPCGAN is particularly effective when sufficient training samples are not available compared to FCN. Our proposed method may be used to relieve the radiologists' burden for annotation.

cs.CV↗

arXiv2020

Roto-Translation Equivariant Convolutional Networks: Application to Histopathology Image Analysis

Rotation-invariance is a desired property of machine-learning models for medical image analysis and in particular for computational pathology applications. We propose a framework to encode the geometric structure of the special Euclidean motion group SE(2) in convolutional networks to yield translation and rotation equivariance via the introduction of SE(2)-group convolution layers. This structure enables models to learn feature representations with a discretized orientation dimension that guarantees that their outputs are invariant under a discrete set of rotations. Conventional approaches for rotation invariance rely mostly on data augmentation, but this does not guarantee the robustness of the output when the input is rotated. At that, trained conventional CNNs may require test-time rotation augmentation to reach their full capability. This study is focused on histopathology image analysis applications for which it is desirable that the arbitrary global orientation information of the imaged tissues is not captured by the machine learning models. The proposed framework is evaluated on three different histopathology image analysis tasks (mitosis detection, nuclei segmentation and tumor classification). We present a comparative analysis for each problem and show that consistent increase of performances can be achieved when using the proposed framework.

cs.CV↗

arXiv2019

Total Variation and Mean Curvature PDEs on $\mathbb{R}^d \rtimes S^{d-1}$

Total variation regularization and total variation flows (TVF) have been widely applied for image enhancement and denoising. To include a generic preservation of crossing curvilinear structures in TVF we lift images to the homogeneous space $M = \mathbb{R}^d \rtimes S^{d-1}$ of positions and orientations as a Lie group quotient in SE(d). For d = 2 this is called 'total roto-translation variation' by Chambolle & Pock. We extend this to d = 3, by a PDE-approach with a limiting procedure for which we prove convergence. We also include a Mean Curvature Flow (MCF) in our PDE model on M. This was first proposed for d = 2 by Citti et al. and we extend this to d = 3. Furthermore, for d = 2 we take advantage of locally optimal differential frames in invertible orientation scores (OS). We apply our TVF and MCF in the denoising/enhancement of crossing fiber bundles in DW-MRI. In comparison to data-driven diffusions, we see a better preservation of bundle boundaries and angular sharpness in fiber orientation densities at crossings. We support this by error comparisons on a noisy DW-MRI phantom. We also apply our TVF and MCF in enhancement of crossing elongated structures in 2D images via OS, and compare the results to nonlinear diffusions (CED-OS) via OS.

math.AP↗

arXiv2018

Fourier Transform on the Homogeneous Space of 3D Positions and Orientations for Exact Solutions to Linear Parabolic and (Hypo-)Elliptic PDEs

Fokker-Planck PDEs (incl. diffusions) for stable Lévy processes (incl. Wiener processes) on the joint space of positions and orientations play a major role in mechanics, robotics, image analysis, directional statistics and probability theory. Exact analytic designs and solutions are known in the 2D case, where they have been obtained using Fourier transform on $SE(2)$. Here we extend these approaches to 3D using Fourier transform on the Lie group $SE(3)$ of rigid body motions. More precisely, we define the homogeneous space of 3D positions and orientations $\mathbb{R}^{3}\rtimes S^{2}:=SE(3)/(\{\mathbf{0}\} \times SO(2))$ as the quotient in $SE(3)$. In our construction, two group elements are equivalent if they are equal up to a rotation around the reference axis. On this quotient we design a specific Fourier transform. We apply this Fourier transform to derive new exact solutions to Fokker-Planck PDEs of $α$-stable Lévy processes on $\mathbb{R}^{3}\rtimes S^{2}$. This reduces classical analysis computations and provides an explicit algebraic spectral decomposition of the solutions. We compare the exact probability kernel for $α= 1$ (the diffusion kernel) to the kernel for $α=\frac12$ (the Poisson kernel). We set up SDEs for the Lévy processes on the quotient and derive corresponding Monte-Carlo methods. We verify that the exact probability kernels arise as the limit of the Monte-Carlo approximations.

math.AP↗

arXiv2018

Roto-Translation Covariant Convolutional Networks for Medical Image Analysis

We propose a framework for rotation and translation covariant deep learning using $SE(2)$ group convolutions. The group product of the special Euclidean motion group $SE(2)$ describes how a concatenation of two roto-translations results in a net roto-translation. We encode this geometric structure into convolutional neural networks (CNNs) via $SE(2)$ group convolutional layers, which fit into the standard 2D CNN framework, and which allow to generically deal with rotated input samples without the need for data augmentation. We introduce three layers: a lifting layer which lifts a 2D (vector valued) image to an $SE(2)$-image, i.e., 3D (vector valued) data whose domain is $SE(2)$; a group convolution layer from and to an $SE(2)$-image; and a projection layer from an $SE(2)$-image to a 2D image. The lifting and group convolution layers are $SE(2)$ covariant (the output roto-translates with the input). The final projection layer, a maximum intensity projection over rotations, makes the full CNN rotation invariant. We show with three different problems in histopathology, retinal imaging, and electron microscopy that with the proposed group CNNs, state-of-the-art performance can be achieved, without the need for data augmentation by rotation and with increased performance compared to standard CNNs that do rely on augmentation.

cs.CV↗

arXiv2017

Optimal Paths for Variants of the 2D and 3D Reeds-Shepp Car with Applications in Image Analysis

We present a PDE-based approach for finding optimal paths for the Reeds-Shepp car. In our model we minimize a (data-driven) functional involving both curvature and length penalization, with several generalizations. Our approach encompasses the two and three dimensional variants of this model, state dependent costs, and moreover, the possibility of removing the reverse gear of the vehicle. We prove both global and local controllability results of the models. Via eikonal equations on the manifold $\mathbb{R}^d \times \mathbb{S}^{d-1}$ we compute distance maps w.r.t. highly anisotropic Finsler metrics, which approximate the singular (quasi)-distances underlying the model. This is achieved using a Fast-Marching (FM) method, building on work by Mirebeau. The FM method is based on specific discretization stencils which are adapted to the preferred directions of the Finsler metric and obey a generalized acuteness property. The shortest paths can be found with a gradient descent method on the distance map, which we formalize in a theorem. We justify the use of our approximating metrics by proving convergence results. Our curve optimization model in $\mathbb{R}^{d} \times \mathbb{S}^{d-1}$ with data-driven cost allows to extract complex tubular structures from medical images, e.g. crossings, and incomplete data due to occlusions or low contrast. Our work extends the results of Sanguinetti et al. on numerical sub-Riemannian eikonal equations and the Reeds-Shepp Car to 3D, with comparisons to exact solutions by Duits et al. Numerical experiments show the high potential of our method in two applications: vessel tracking in retinal images for the case $d=2$, and brain connectivity measures from diffusion weighted MRI-data for the case $d=3$, extending the work of Bekkers et al. We demonstrate how the new model without reverse gear better handles bifurcations.

math.NA↗

arXiv2017

Template Matching via Densities on the Roto-Translation Group

We propose a template matching method for the detection of 2D image objects that are characterized by orientation patterns. Our method is based on data representations via orientation scores, which are functions on the space of positions and orientations, and which are obtained via a wavelet-type transform. This new representation allows us to detect orientation patterns in an intuitive and direct way, namely via cross-correlations. Additionally, we propose a generalized linear regression framework for the construction of suitable templates using smoothing splines. Here, it is important to recognize a curved geometry on the position-orientation domain, which we identify with the Lie group SE(2): the roto-translation group. Templates are then optimized in a B-spline basis, and smoothness is defined with respect to the curved geometry. We achieve state-of-the-art results on three different applications: detection of the optic nerve head in the retina (99.83% success rate on 1737 images), of the fovea in the retina (99.32% success rate on 1616 images), and of the pupil in regular camera images (95.86% on 1521 images). The high performance is due to inclusion of both intensity and orientation features with effective geometric priors in the template matching. Moreover, our method is fast due to a cross-correlation based matching approach.

cs.CV↗

arXiv2016

Retrieving challenging vessel connections in retinal images by line co-occurrence statistics

Natural images contain often curvilinear structures, which might be disconnected, or partly occluded. Recovering the missing connection of disconnected structures is an open issue and needs appropriate geometric reasoning. We propose to find line co-occurrence statistics from the centerlines of blood vessels in retinal images and show its remarkable similarity to a well-known probabilistic model for the connectivity pattern in the primary visual cortex. Furthermore, the probabilistic model is trained from the data via statistics and used for automated grouping of interrupted vessels in a spectral clustering based approach. Several challenging image patches are investigated around junction points, where successful results indicate the perfect match of the trained model to the profiles of blood vessels in retinal images. Also, comparisons among several statistical models obtained from different datasets reveals their high similarity i.e., they are independent of the dataset. On top of that, the best approximation of the statistical model with the symmetrized extension of the probabilistic model on the projective line bundle is found with a least square error smaller than 2%. Apparently, the direction process on the projective line bundle is a good continuation model for vessels in retinal images.

cs.CV↗

arXiv2016

On sub-Riemannian geodesics in $SE(3)$ whose spatial projections do not have cusps

We consider the problem $\mathbf{P_{curve}}$ of minimizing $\int \limits_0^L \sqrt{ξ^2 + κ^2(s)} \, {\rm d}s$ for a curve $\mathbf{x}$ on $\mathbb R$ with fixed boundary points and directions. Here the total length $L\geq 0$ is free, $s$ denotes the arclength parameter, $κ$ denotes the absolute curvature of $\mathbf{x}$, and $ξ>0$ is constant. We lift problem $\mathbf{P_{curve}}$ on $\mathbb R^3$ to a sub-Riemannian problem $\mathbf{P_{mec}}$ on $\operatorname{SE(3)}\nolimits/(\{\mathbf{0}\}\times \operatorname{SO(2)}\nolimits)$. Here, for admissible boundary conditions, the spatial projections of sub-Riemannian geodesics do not exhibit cusps and they solve problem $\mathbf{P_{curve}}$. We apply the Pontryagin Maximum Principle (PMP) and prove Liouville integrability of the Hamiltonian system. We derive explicit analytic formulas for such sub-Riemannian geodesics, relying on the co-adjoint orbit structure, an underlying Cartan connection, and the matrix representation of $\operatorname{SE(3)}\nolimits$ arising in the Cartan-matrix. These formulas allow us to extract geometrical properties of the sub-Riemannian geodesics with cuspless projection, such as planarity conditions, explicit bounds on their torsion, and their symmetries. Furthermore, they allow us to parameterize all admissible boundary conditions reachable by geodesics with cuspless spatial projection. Such projections lay in the upper half space. We prove this for most cases, and the rest is checked numerically. Finally, we employ the formulas to numerically solve the boundary value problem, and visualize the set of admissible boundary conditions.

math.OC↗

arXiv2016

Numerical Approaches for Linear Left-invariant Diffusions on SE(2), their Comparison to Exact Solutions, and their Applications in Retinal Imaging

Left-invariant PDE-evolutions on the roto-translation group $SE(2)$ (and their resolvent equations) have been widely studied in the fields of cortical modeling and image analysis. They include hypo-elliptic diffusion (for contour enhancement) proposed by Citti & Sarti, and Petitot, and they include the direction process (for contour completion) proposed by Mumford. This paper presents a thorough study and comparison of the many numerical approaches, which, remarkably, is missing in the literature. Existing numerical approaches can be classified into 3 categories: Finite difference methods, Fourier based methods (equivalent to $SE(2)$-Fourier methods), and stochastic methods (Monte Carlo simulations). There are also 3 types of exact solutions to the PDE-evolutions that were derived explicitly (in the spatial Fourier domain) in previous works by Duits and van Almsick in 2005. Here we provide an overview of these 3 types of exact solutions and explain how they relate to each of the 3 numerical approaches. We compute relative errors of all numerical approaches to the exact solutions, and the Fourier based methods show us the best performance with smallest relative errors. We also provide an improvement of Mathematica algorithms for evaluating Mathieu-functions, crucial in implementations of the exact solutions. Furthermore, we include an asymptotical analysis of the singularities within the kernels and we propose a probabilistic extension of underlying stochastic processes that overcomes the singular behavior in the origin of time-integrated kernels. Finally, we show retinal imaging applications of combining left-invariant PDE-evolutions with invertible orientation scores.

math.NA↗

arXiv2015

Sub-Riemannian Fast Marching in SE(2)

We propose a Fast Marching based implementation for computing sub-Riemanninan (SR) geodesics in the roto-translation group SE(2), with a metric depending on a cost induced by the image data. The key ingredient is a Riemannian approximation of the SR-metric. Then, a state of the art Fast Marching solver that is able to deal with extreme anisotropies is used to compute a SR-distance map as the solution of a corresponding eikonal equation. Subsequent backtracking on the distance map gives the geodesics. To validate the method, we consider the uniform cost case in which exact formulas for SR-geodesics are known and we show remarkable accuracy of the numerically computed SR-spheres. We also show a dramatic decrease in computational time with respect to a previous PDE-based iterative approach. Regarding image analysis applications, we show the potential of considering these data adaptive geodesics for a fully automated retinal vessel tree segmentation.

math.NA↗

arXiv2015

Invertible Orientation Scores of 3D Images

The enhancement and detection of elongated structures in noisy image data is relevant for many biomedical applications. To handle complex crossing structures in 2D images, 2D orientation scores were introduced, which already showed their use in a variety of applications. Here we extend this work to 3D orientation scores. First, we construct the orientation score from a given dataset, which is achieved by an invertible coherent state type of transform. For this transformation we introduce 3D versions of the 2D cake-wavelets, which are complex wavelets that can simultaneously detect oriented structures and oriented edges. For efficient implementation of the different steps in the wavelet creation we use a spherical harmonic transform. Finally, we show some first results of practical applications of 3D orientation scores.

math.NA↗

arXiv2015

A PDE Approach to Data-driven Sub-Riemannian Geodesics in SE(2)

We present a new flexible wavefront propagation algorithm for the boundary value problem for sub-Riemannian (SR) geodesics in the roto-translation group $SE(2) = \mathbb{R}^2 \rtimes S^1$ with a metric tensor depending on a smooth external cost $\mathcal{C}:SE(2) \to [δ,1]$, $δ>0$, computed from image data. The method consists of a first step where a SR-distance map is computed as a viscosity solution of a Hamilton-Jacobi-Bellman (HJB) system derived via Pontryagin's Maximum Principle (PMP). Subsequent backward integration, again relying on PMP, gives the SR-geodesics. For $\mathcal{C}=1$ we show that our method produces the global minimizers. Comparison with exact solutions shows a remarkable accuracy of the SR-spheres and the SR-geodesics. We present numerical computations of Maxwell points and cusp points, which we again verify for the uniform cost case $\mathcal{C}=1$. Regarding image analysis applications, tracking of elongated structures in retinal and synthetic images show that our line tracking generically deals with crossings. We show the benefits of including the sub-Riemannian geometry.

math.GR↗

arXiv2014

Vesselness via Multiple Scale Orientation Scores

The multi-scale Frangi vesselness filter is an established tool in (retinal) vascular imaging. However, it cannot cope with crossings or bifurcations, since it only looks for elongated structures. Therefore, we disentangle crossing structures in the image via (multiple scale) invertible orientation scores. The described vesselness filter via scale-orientation scores performs considerably better at enhancing vessels throughout crossings and bifurcations than the Frangi version. Both methods are evaluated on a public dataset. Performance is measured by comparing ground truth data to the segmentation results obtained by basic thresholding and morphological component analysis of the filtered images.

cs.CV↗

arXiv2013

A Multi-Orientation Analysis Approach to Retinal Vessel Tracking

This paper presents a method for retinal vasculature extraction based on biologically inspired multi-orientation analysis. We apply multi-orientation analysis via so-called invertible orientation scores, modeling the cortical columns in the visual system of higher mammals. This allows us to generically deal with many hitherto complex problems inherent to vessel tracking, such as crossings, bifurcations, parallel vessels, vessels of varying widths and vessels with high curvature. Our approach applies tracking in invertible orientation scores via a novel geometrical principle for curve optimization in the Euclidean motion group SE(2). The method runs fully automatically and provides a detailed model of the retinal vasculature, which is crucial as a sound basis for further quantitative analysis of the retina, especially in screening applications.

cs.CV↗

arXiv2013

Curve cuspless reconstruction via sub-Riemannian geometry

We consider the problem of minimizing $\int_{0}^L \sqrt{ξ^2 +K^2(s)}\, ds $ for a planar curve having fixed initial and final positions and directions. The total length $L$ is free. Here $s$ is the variable of arclength parametrization, $K(s)$ is the curvature of the curve and $ξ>0$ a parameter. This problem comes from a model of geometry of vision due to Petitot, Citti and Sarti. We study existence of local and global minimizers for this problem. We prove that if for a certain choice of boundary conditions there is no global minimizer, then there is neither a local minimizer nor a geodesic. We finally give properties of the set of boundary conditions for which there exists a solution to the problem.

math.OC↗

arXiv2011

Diffusion, Convection and Erosion on SE(3)/({0} \times SO(2)) and their Application to the Enhancement of Crossing Fibers

In this article we study both left-invariant (convection-)diffusions and left-invariant Hamilton-Jacobi equations on the space SE(3)/({0} \times SO(2)) of 3D-positions and orientations naturally embedded in the group SE(3) of 3D-rigid body movements. The general motivation for these (convection-)diffusions and erosions is to obtain crossing-preserving fiber enhancement on probability densities defined on the space of positions and orientations. The linear left-invariant (convection-)diffusions are forward Kolmogorov equations of Brownian motions on SE(3)/({0}\timesSO(2)) and can be solved by convolution with the corresponding Green's functions or by a finite difference scheme. The left-invariant Hamilton-Jacobi equations are Bellman equations of cost processes on SE(3)/({0}\timesSO(2)) and they are solved by a morphological convolution with the corresponding Green's functions. Furthermore, we consider pseudo-linear scale spaces on the space of positions and orientations that combines dilation and diffusion in a single evolution. In our design and analysis for appropriate linear, non-linear, morphological and pseudo-linear scale spaces on SE(3)/({0}\timesSO(2)) we employ the underlying differential geometry on SE(3), where the frame of left-invariant vector fields serves as a moving frame of reference. Furthermore, we will present new and simpler finite difference schemes for our diffusions, which are clear improvements of our previous finite difference schemes. We apply our theory to the enhancement of fibres in magnetic resonance imaging (MRI) techniques for imaging water diffusion processes in brain white matter. We provide experiments of our crossing-preserving evolutions on neural images of a human brain containing crossing fibers.

math.AP↗

arXiv2011

Evolution Equations on Gabor Transforms and their Applications

We introduce a systematic approach to the design, implementation and analysis of left-invariant evolution schemes acting on Gabor transform, primarily for applications in signal and image analysis. Within this approach we relate operators on signals to operators on Gabor transforms. In order to obtain a translation and modulation invariant operator on the space of signals, the corresponding operator on the reproducing kernel space of Gabor transforms must be left invariant, i.e. it should commute with the left regular action of the reduced Heisenberg group H_r. By using the left-invariant vector fields on H_r in the generators of our evolution equations on Gabor transforms, we naturally employ the essential group structure on the domain of a Gabor transform. Here we distinguish between two tasks. Firstly, we consider non-linear adaptive left-invariant convection (reassignment) to sharpen Gabor transforms, while maintaining the original signal. Secondly, we consider signal enhancement via left-invariant diffusion on the corresponding Gabor transform. We provide numerical experiments and analytical evidence for our methods and we consider an explicit medical imaging application.

math.AP↗