arXiv subjects

Markus Haltmeier

Publications and source records attributed to Markus Haltmeier.

At least 55 records · Page 3Linked to original sources

arXiv2023

Data-proximal complementary $\ell^1$-TV reconstruction for limited data CT

In a number of tomographic applications, data cannot be fully acquired, resulting in a severely underdetermined image reconstruction. In such cases, conventional methods lead to reconstructions with significant artifacts. To overcome these artifacts, regularization methods are applied that incorporate additional information. An important example is TV reconstruction, which is known to be efficient at compensating for missing data and reducing reconstruction artifacts. At the same time, however, tomographic data is also contaminated by noise, which poses an additional challenge. The use of a single regularizer must therefore account for both the missing data and the noise. However, a particular regularizer may not be ideal for both tasks. For example, the TV regularizer is a poor choice for noise reduction across multiple scales, in which case $\ell^1$ curvelet regularization methods are well suited. To address this issue, in this paper we introduce a novel variational regularization framework that combines the advantages of different regularizers. The basic idea of our framework is to perform reconstruction in two stages, where the first stage mainly aims at accurate reconstruction in the presence of noise, and the second stage aims at artifact reduction. Both reconstruction stages are connected by a data proximity condition. The proposed method is implemented and tested for limited-view CT using a combined curvelet-TV approach. We define and implement a curvelet transform adapted to the limited-view problem and illustrate the advantages of our approach in numerical experiments.

math.NA↗

arXiv2023

Variational multichannel multiclass segmentation using unsupervised lifting with CNNs

We propose an unsupervised image segmentation approach, that combines a variational energy functional and deep convolutional neural networks. The variational part is based on a recent multichannel multiphase Chan-Vese model, which is capable to extract useful information from multiple input images simultaneously. We implement a flexible multiclass segmentation method that divides a given image into $K$ different regions. We use convolutional neural networks (CNNs) targeting a pre-decomposition of the image. By subsequently minimising the segmentation functional, the final segmentation is obtained in a fully unsupervised manner. Special emphasis is given to the extraction of informative feature maps serving as a starting point for the segmentation. The initial results indicate that the proposed method is able to decompose and segment the different regions of various types of images, such as texture and medical images and compare its performance with another multiphase segmentation method.

cs.CV↗

arXiv2023

Convergence analysis of critical point regularization with non-convex regularizers

One of the key assumptions in the stability and convergence analysis of variational regularization is the ability of finding global minimizers. However, such an assumption is often not feasible when the regularizer is a black box or non-convex making the search for global minimizers of the involved Tikhonov functional a challenging task. This is in particular the case for the emerging class of learned regularizers defined by neural networks. Instead, standard minimization schemes are applied which typically only guarantee that a critical point is found. To address this issue, in this paper we study stability and convergence properties of critical points of Tikhonov functionals with a possible non-convex regularizer. To this end, we introduce the concept of relative sub-differentiability and study its basic properties. Based on this concept, we develop a convergence analysis assuming relative sub-differentiability of the regularizer. The rationale behind the proposed concept is that critical points of the Tikhonov functional are also relative critical points and that for the latter a convergence theory can be developed. For the case where the noise level tends to zero, we derive a limiting problem representing first-order optimality conditions of a related restricted optimization problem. Besides this, we also give a comparison with classical methods and show that the class of ReLU-networks are appropriate choices for the regularization functional. Finally, we provide numerical simulations that support our theoretical findings and the need for the sort of analysis that we provide in this paper.

math.OC↗

arXiv2023

Uncertainty-Aware Null Space Networks for Data-Consistent Image Reconstruction

Reconstructing an image from noisy and incomplete measurements is a central task in several image processing applications. In recent years, state-of-the-art reconstruction methods have been developed based on recent advances in deep learning. Especially for highly underdetermined problems, maintaining data consistency is a key goal. This can be achieved either by iterative network architectures or by a subsequent projection of the network reconstruction. However, for such approaches to be used in safety-critical domains such as medical imaging, the network reconstruction should not only provide the user with a reconstructed image, but also with some level of confidence in the reconstruction. In order to meet these two key requirements, this paper combines deep null-space networks with uncertainty quantification. Evaluation of the proposed method includes image reconstruction from undersampled Radon measurements on a toy CT dataset and accelerated MRI reconstruction on the fastMRI dataset. This work is the first approach to solving inverse problems that additionally models data-dependent uncertainty by estimating an input-dependent scale map, providing a robust assessment of reconstruction quality.

cs.CV↗

arXiv2023

Translation invariant diagonal frame decomposition of inverse problems and their regularization

Solving inverse problems is central to a variety of important applications, such as biomedical image reconstruction and non-destructive testing. These problems are characterized by the sensitivity of direct solution methods with respect to data perturbations. To stabilize the reconstruction process, regularization methods have to be employed. Well-known regularization methods are based on frame expansions, such as the wavelet-vaguelette (WVD) decomposition, which are well adapted to the underlying signal class and the forward model and furthermore allow efficient implementation. However, it is well known that the lack of translational invariance of wavelets and related systems leads to specific artifacts in the reconstruction. To overcome this problem, in this paper we introduce and analyze the translation invariant diagonal frame decomposition (TI-DFD) of linear operators as a novel concept generalizing the SVD. We characterize ill-posedness via the TI-DFD and prove that a TI-DFD combined with a regularizing filter leads to a convergent regularization method with optimal convergence rates. As illustrative example, we construct a wavelet-based TI-DFD for one-dimensional integration, where we also investigate our approach numerically. The results indicate that filtered TI-DFDs eliminate the typical wavelet artifacts when using standard wavelets and provide a fast, accurate, and stable solution scheme for inverse problems.

math.NA↗

arXiv2023

Convergence rates for critical point regularization

Tikhonov regularization involves minimizing the combination of a data discrepancy term and a regularizing term, and is the standard approach for solving inverse problems. The use of non-convex regularizers, such as those defined by trained neural networks, has been shown to be effective in many cases. However, finding global minimizers in non-convex situations can be challenging, making existing theory inapplicable. A recent development in regularization theory relaxes this requirement by providing convergence based on critical points instead of strict minimizers. This paper investigates convergence rates for the regularization with critical points using Bregman distances. Furthermore, we show that when implementing near-minimization through an iterative algorithm, a finite number of iterations is sufficient without affecting convergence rates.

math.OC↗

arXiv2022

Plug-and-Play image reconstruction is a convergent regularization method

Non-uniqueness and instability are characteristic features of image reconstruction processes. As a result, it is necessary to develop regularization methods that can be used to compute reliable approximate solutions. A regularization method provides of a family of stable reconstructions that converge to an exact solution of the noise-free problem as the noise level tends to zero. The standard regularization technique is defined by variational image reconstruction, which minimizes a data discrepancy augmented by a regularizer. The actual numerical implementation makes use of iterative methods, often involving proximal mappings of the regularizer. In recent years, Plug-and-Play image reconstruction (PnP) has been developed as a new powerful generalization of variational methods based on replacing proximal mappings by more general image denoisers. While PnP iterations yield excellent results, neither stability nor convergence in the sense of regularization has been studied so far. In this work, we extend the idea of PnP by considering families of PnP iterations, each being accompanied by its own denoiser. As our main theoretical result, we show that such PnP reconstructions lead to stable and convergent regularization methods. This shows for the first time that PnP is mathematically equally justified for robust image reconstruction as variational methods

math.NA↗

arXiv2022

Unsupervised Joint Image Transfer and Uncertainty Quantification Using Patch Invariant Networks

Unsupervised image transfer enables intra- and inter-modality image translation in applications where a large amount of paired training data is not abundant. To ensure a structure-preserving mapping from the input to the target domain, existing methods for unpaired image transfer are commonly based on cycle-consistency, causing additional computational resources and instability due to the learning of an inverse mapping. This paper presents a novel method for uni-directional domain mapping that does not rely on any paired training data. A proper transfer is achieved by using a GAN architecture and a novel generator loss based on patch invariance. To be more specific, the generator outputs are evaluated and compared at different scales, also leading to an increased focus on high-frequency details as well as an implicit data augmentation. This novel patch loss also offers the possibility to accurately predict aleatoric uncertainty by modeling an input-dependent scale map for the patch residuals. The proposed method is comprehensively evaluated on three well-established medical databases. As compared to four state-of-the-art methods, we observe significantly higher accuracy on these datasets, indicating great potential of the proposed method for unpaired image transfer with uncertainty taken into account. Implementation of the proposed framework is released here: \url{https://github.com/anger-man/unsupervised-image-transfer-and-uq}.

cs.CV↗

arXiv2022

Regularization of Inverse Problems by Filtered Diagonal Frame Decomposition

The characteristic feature of inverse problems is their instability with respect to data perturbations. In order to stabilize the inversion process, regularization methods have to be developed and applied. In this work we introduce and analyze the concept of filtered diagonal frame decomposition which extends the standard filtered singular value decomposition to the frame case. Frames as generalized singular system allows to better adapt to a given class of potential solutions. In this paper, we show that filtered diagonal frame decomposition yield a convergent regularization method. Moreover, we derive convergence rates under source type conditions and prove order optimality under the assumption that the considered frame is a Riesz-basis.

math.NA↗

arXiv2022

Surface Topography Characterization Using a Simple Optical Device and Artificial Neural Networks

State-of-the-art methods for quantifying wear in cylinder liners of large internal combustion engines require disassembly and cutting of the liner. This is followed by laboratory-based high-resolution microscopic surface depth measurement that quantitatively evaluates wear based on bearing load curves (Abbott-Firestone curves). Such methods are destructive, time-consuming and costly. The goal of the research presented is to develop nondestructive yet reliable methods for quantifying the surface topography. A novel machine learning framework is proposed that allows prediction of the bearing load curves from RGB images of the liner surface that can be collected with a handheld microscope. A joint deep learning approach involving two neural network modules optimizes the prediction quality of surface roughness parameters as well and is trained using a custom-built database containing 422 aligned depth profile and reflection image pairs of liner surfaces. The observed success suggests its great potential for on-site wear assessment of engines during service.

cs.CV↗

arXiv2022

Unpaired Single-Image Depth Synthesis with cycle-consistent Wasserstein GANs

Real-time estimation of actual environment depth is an essential module for various autonomous system tasks such as localization, obstacle detection and pose estimation. During the last decade of machine learning, extensive deployment of deep learning methods to computer vision tasks yielded successful approaches for realistic depth synthesis out of a simple RGB modality. While most of these models rest on paired depth data or availability of video sequences and stereo images, there is a lack of methods facing single-image depth synthesis in an unsupervised manner. Therefore, in this study, latest advancements in the field of generative neural networks are leveraged to fully unsupervised single-image depth synthesis. To be more exact, two cycle-consistent generators for RGB-to-depth and depth-to-RGB transfer are implemented and simultaneously optimized using the Wasserstein-1 distance. To ensure plausibility of the proposed method, we apply the models to a self acquised industrial data set as well as to the renown NYU Depth v2 data set, which allows comparison with existing approaches. The observed success in this study suggests high potential for unpaired single-image depth estimation in real world applications.

cs.CV↗

arXiv2022

Convolutional Dictionary Learning by End-To-End Training of Iterative Neural Networks

Sparsity-based methods have a long history in the field of signal processing and have been successfully applied to various image reconstruction problems. The involved sparsifying transformations or dictionaries are typically either pre-trained using a model which reflects the assumed properties of the signals or adaptively learned during the reconstruction - yielding so-called blind Compressed Sensing approaches. However, by doing so, the transforms are never explicitly trained in conjunction with the physical model which generates the signals. In addition, properly choosing the involved regularization parameters remains a challenging task. Another recently emerged training-paradigm for regularization methods is to use iterative neural networks (INNs) - also known as unrolled networks - which contain the physical model. In this work, we construct an INN which can be used as a supervised and physics-informed online convolutional dictionary learning algorithm. We evaluated the proposed approach by applying it to a realistic large-scale dynamic MR reconstruction problem and compared it to several other recently published works. We show that the proposed INN improves over two conventional model-agnostic training methods and yields competitive results also compared to a deep INN. Further, it does not require to choose the regularization parameters and - in contrast to deep INNs - each network component is entirely interpretable.

eess.IV↗

arXiv2022

Unsupervised Single-shot Depth Estimation using Perceptual Reconstruction

Real-time estimation of actual object depth is an essential module for various autonomous system tasks such as 3D reconstruction, scene understanding and condition assessment. During the last decade of machine learning, extensive deployment of deep learning methods to computer vision tasks has yielded approaches that succeed in achieving realistic depth synthesis out of a simple RGB modality. Most of these models are based on paired RGB-depth data and/or the availability of video sequences and stereo images. The lack of sequences, stereo data and RGB-depth pairs makes depth estimation a fully unsupervised single-image transfer problem that has barely been explored so far. This study builds on recent advances in the field of generative neural networks in order to establish fully unsupervised single-shot depth estimation. Two generators for RGB-to-depth and depth-to-RGB transfer are implemented and simultaneously optimized using the Wasserstein-1 distance, a novel perceptual reconstruction term and hand-crafted image filters. We comprehensively evaluate the models using industrial surface depth data as well as the Texas 3D Face Recognition Database, the CelebAMask-HQ database of human portraits and the SURREAL dataset that records body depth. For each evaluation dataset the proposed method shows a significant increase in depth accuracy compared to state-of-the-art single-image transfer methods.

cs.CV↗

arXiv2022

Convergence rates for the joint solution of inverse problems with compressed sensing data

Compressed sensing (CS) is a powerful tool for reducing the amount of data to be collected while maintaining high spatial resolution. Such techniques work well in practice and at the same time are supported by solid theory. Standard CS results assume measurements to be made directly on the targeted signal. In many practical applications, however, CS information can only be taken from indirect data $h_\star = W x_\star$ related to the original signal by an additional forward operator. If inverting the forward operator is ill-posed, then existing CS theory is not applicable. In this paper, we address this issue and present two joint reconstruction approaches, namely relaxed $\ell^1$ co-regularization and strict $\ell^1$ co-regularization, for CS from indirect data. As main results, we derive error estimates for recovering $x_\star$ and $h_\star$. In particular, we derive a linear convergence rate in the norm for the latter. To obtain these results, solutions are required to satisfy a source condition and the CS measurement operator is required to satisfy a restricted injectivity condition. We further show that these conditions are not only sufficient but even necessary to obtain linear convergence.

cs.IT↗

arXiv2022

Photoacoustic inversion formulas using mixed data on finite time intervals

We study the inverse source problem in photoacoustic tomography (PAT) for mixed data, which denote a weighted linear combination of the acoustic pressure and its normal derivative on an observation surface. We consider in particular the case where the data are only available on finite time intervals, which accounts for real-world usage of PAT where data are only feasible within a certain time interval. Extending our previous work, we derive explicit formulas up to a smoothing integral on convex domains with a smooth boundary, yielding exact reconstruction for circular or elliptical domains. We also present numerical reconstructions of our new exact inversion formulas on finite time intervals and compare them with the reconstructions of our previous formulas for unlimited time wave measurements.

math.AP↗

arXiv2022

Convolutional Analysis Operator Learning by End-To-End Training of Iterative Neural Networks

The concept of sparsity has been extensively applied for regularization in image reconstruction. Typically, sparsifying transforms are either pre-trained on ground-truth images or adaptively trained during the reconstruction. Thereby, learning algorithms are designed to minimize some target function which encodes the desired properties of the transform. However, this procedure ignores the subsequently employed reconstruction algorithm as well as the physical model which is responsible for the image formation process. Iterative neural networks - which contain the physical model - can overcome these issues. In this work, we demonstrate how convolutional sparsifying filters can be efficiently learned by end-to-end training of iterative neural networks. We evaluated our approach on a non-Cartesian 2D cardiac cine MRI example and show that the obtained filters are better suitable for the corresponding reconstruction algorithm than the ones obtained by decoupled pre-training.

eess.IV↗

arXiv2022

Feature reconstruction from incomplete tomographic data without detour

In this paper, we consider the problem of feature reconstruction from incomplete x-ray CT data. Such problems occurs, e.g., as a result of dose reduction in the context medical imaging. Since image reconstruction from incomplete data is a severely ill-posed problem, the reconstructed images may suffer from characteristic artefacts or missing features, and significantly complicate subsequent image processing tasks (e.g., edge detection or segmentation). In this paper, we introduce a novel framework for the robust reconstruction of convolutional image features directly from CT data, without the need of computing a reconstruction firs. Within our framework we use non-linear (variational) regularization methods that can be adapted to a variety of feature reconstruction tasks and to several limited data situations . In our numerical experiments, we consider several instances of edge reconstructions from angularly undersampled data and show that our approach is able to reliably reconstruct feature maps in this case.

eess.IV↗

arXiv2021

Analysis for Full Field Photoacoustic Tomography with Variable Sound Speed

Photoacoustic tomography (PAT) is a non-invasive imaging modality that requires recovering the initial data of the wave equation from certain measurements of the solution outside the object. In the standard PAT measurement setup, the used data consist of time-dependent signals measured on an observation surface. In contrast, the measured data from the recently invented full-field detection technique provide the solution of the wave equation on a spatial domain at a single instant in time. While reconstruction using classical PAT data has been extensively studied, not much is known for the full field PAT problem. In this paper, we build mathematical foundations of the latter problem for variable sound speed and settle its uniqueness and stability. Moreover, we introduce an exact inversion method using time-reversal and study its convergence. Our results demonstrate the suitability of both the full field approach and the proposed time-reversal technique for high resolution photoacoustic imaging.

math.AP↗