arXiv subjects

Markus Haltmeier

Publications and source records attributed to Markus Haltmeier.

At least 73 records · Page 4Linked to original sources

arXiv2021

Discretization of learned NETT regularization for solving inverse problems

Deep learning based reconstruction methods deliver outstanding results for solving inverse problems and are therefore becoming increasingly important. A recently invented class of learning-based reconstruction methods is the so-called NETT (for Network Tikhonov Regularization), which contains a trained neural network as regularizer in generalized Tikhonov regularization. The existing analysis of NETT considers fixed operator and fixed regularizer and analyzes the convergence as the noise level in the data approaches zero. In this paper, we extend the frameworks and analysis considerably to reflect various practical aspects and take into account discretization of the data space, the solution space, the forward operator and the neural network defining the regularizer. We show the asymptotic convergence of the discretized NETT approach for decreasing noise levels and discretization errors. Additionally, we derive convergence rates and present numerical results for a limited data problem in photoacoustic tomography.

math.NA↗

arXiv2021

Combining reconstruction and edge detection in computed tomography

We present two methods that combine image reconstruction and edge detection in computed tomography (CT) scans. Our first method is as an extension of the prominent filtered backprojection algorithm. In our second method we employ $\ell^{1}$-regularization for stable calculation of the gradient. As opposed to the first method, we show that this approach is able to compensate for undersampled CT data.

math.NA↗

arXiv2021

A Variational View on Statistical Multiscale Estimation

We present a unifying view on various statistical estimation techniques including penalization, variational and thresholding methods. These estimators will be analyzed in the context of statistical linear inverse problems including nonparametric and change point regression, and high dimensional linear models as examples. Our approach reveals many seemingly unrelated estimation schemes as special instances of a general class of variational multiscale estimators, named MIND (MultIscale Nemirovskii--Dantzig). These estimators result from minimizing certain regularization functionals under convex constraints that can be seen as multiple statistical tests for local hypotheses. For computational purposes, we recast MIND in terms of simpler unconstraint optimization problems via Lagrangian penalization as well as Fenchel duality. Performance of several MINDs is demonstrated on numerical examples.

stat.ME↗

arXiv2021

Deep Structure Learning using Feature Extraction in Trained Projection Space

Over the last decade of machine learning, convolutional neural networks have been the most striking successes for feature extraction of rich sensory and high-dimensional data. While learning data representations via convolutions is already well studied and efficiently implemented in various deep learning libraries, one often faces limited memory capacity and insufficient number of training data, especially for high-dimensional and large-scale tasks. To overcome these limitations, we introduce a network architecture using a self-adjusting and data dependent version of the Radon-transform (linear data projection), also known as x-ray projection, to enable feature extraction via convolutions in lower-dimensional space. The resulting framework, named PiNet, can be trained end-to-end and shows promising performance on volumetric segmentation tasks. We test proposed model on public datasets to show that our approach achieves comparable results only using fractional amount of parameters. Investigation of memory usage and processing time confirms PiNet's superior efficiency compared to other segmentation models.

eess.IV↗

arXiv2021

Augmented NETT Regularization of Inverse Problems

We propose aNETT (augmented NETwork Tikhonov) regularization as a novel data-driven reconstruction framework for solving inverse problems. An encoder-decoder type network defines a regularizer consisting of a penalty term that enforces regularity in the encoder domain, augmented by a penalty that penalizes the distance to the data manifold. We present a rigorous convergence analysis including stability estimates and convergence rates. For that purpose, we prove the coercivity of the regularizer used without requiring explicit coercivity assumptions for the networks involved. We propose a possible realization together with a network architecture and a modular training strategy. Applications to sparse-view and low-dose CT show that aNETT achieves results comparable to state-of-the-art deep-learning-based reconstruction methods. Unlike learned iterative methods, aNETT does not require repeated application of the forward and adjoint models, which enables the use of aNETT for inverse problems with numerically expensive forward models. Furthermore, we show that aNETT trained on coarsely sampled data can leverage an increased sampling rate without the need for retraining.

math.NA↗

arXiv2021

An End-To-End-Trainable Iterative Network Architecture for Accelerated Radial Multi-Coil 2D Cine MR Image Reconstruction

Purpose: Iterative Convolutional Neural Networks (CNNs) which resemble unrolled learned iterative schemes have shown to consistently deliver state-of-the-art results for image reconstruction problems across different imaging modalities. However, because these methodes include the forward model in the architecture, their applicability is often restricted to either relatively small reconstruction problems or to problems with operators which are computationally cheap to compute. As a consequence, they have so far not been applied to dynamic non-Cartesian multi-coil reconstruction problems. Methods: In this work, we propose a CNN-architecture for image reconstruction of accelerated 2D radial cine MRI with multiple receiver coils. The network is based on a computationally light CNN-component and a subsequent conjugate gradient (CG) method which can be jointly trained end-to-end using an efficient training strategy. We investigate the proposed training-strategy and compare our method to other well-known reconstruction techniques with learned and non-learned regularization methods. Results: Our proposed method outperforms all other methods based on non-learned regularization. Further, it performs similar or better than a CNN-based method employing a 3D U-Net and a method using adaptive dictionary learning. In addition, we empirically demonstrate that even by training the network with only iteration, it is possible to increase the length of the network at test time and further improve the results. Conclusions: End-to-end training allows to highly reduce the number of trainable parameters of and stabilize the reconstruction network. Further, because it is possible to change the length of the network at test time, the need to find a compromise between the complexity of the CNN-block and the number of iterations in each CG-block becomes irrelevant.

cs.LG↗

arXiv2021

Recovering the Initial Data of the Wave Equation from Neumann Traces

We study the problem of recovering the initial data (f, 0) of the standard wave equation from the Neumann trace (the normal derivative) of the solution on the boundary of convex domains in arbitrary spatial dimension. Among others, this problem is relevant for tomographic image reconstruction including photoacoustic tomography. We establish explicit inversion formulas of the back-projection type that recover the initial data up to an additive term defined by a smoothing integral operator. In the case that the boundary of the domain is an ellipsoid, the integral operator vanishes, and hence we obtain an analytic formula for recovering the initial data from Neumann traces of the wave equation on ellipsoids.

math.AP↗

arXiv2021

Multi-Scale Factorization of the Wave Equation with Application to Compressed Sensing Photoacoustic Tomography

Performing a large number of spatial measurements enables high-resolution photoacoustic imaging without specific prior information. However, the acquisition of spatial measurements is time-consuming, costly, and technically challenging. By exploiting nonlinear prior information, compressed sensing techniques in combination with sophisticated reconstruction algorithms allow reducing the number of measurements while maintaining high spatial resolution. To this end, in this work we propose a multiscale factorization for the wave equation that decomposes the measured data into a low-frequency factor and sparse high-frequency factors. By extending the acoustic reciprocity principle, we transfer sparsity in the measurement domain into spatial sparsity of the initial pressure, which allows the use of sparse reconstruction techniques. Numerical results are presented that demonstrate the feasibility of the proposed framework.

math.NA↗

arXiv2020

Linking information theory and thermodynamics to spatial resolution in photothermal and photoacoustic imaging

In this tutorial, we combine the different scientific fields of information theory, thermodynamics, regularization theory and non-destructive imaging, especially for photoacoustic and photothermal imaging. The goal is to get a better understanding of how information gaining for subsurface imaging works and how the spatial resolution limit can be overcome by using additional information. Here, the resolution limit in photoacoustic and photothermal imaging is derived from the irreversibility of attenuation of the pressure wave and of heat diffusion during propagation of the signals from the imaged subsurface structures to the sample surface, respectively. The acoustic or temperature signals are converted into so-called virtual waves, which are their reversible counterparts and which can be used for image reconstruction by well-known ultrasound reconstruction methods. The conversion into virtual waves is an ill-posed inverse problem which needs regularization. The reason for that is the information loss during signal propagation to the sample surface, which turns out to be equal to the entropy production. As the entropy production from acoustic attenuation is usually small compared to the entropy production from heat diffusion, the spatial resolution in acoustic imaging is higher than in thermal imaging. Therefore, it is especially necessary to overcome this resolution limit for thermographic imaging by using additional information. Incorporating sparsity and non-negativity in iterative regularization methods gives a significant resolution enhancement, which was experimentally demonstrated by one-dimensional imaging of thin layers with varying depth or by three-dimensional imaging, either from a single detection plane or from three perpendicular detection planes on the surface of a sample cube.

physics.app-ph↗

arXiv2020

Regularization of Inverse Problems by Neural Networks

Inverse problems arise in a variety of imaging applications including computed tomography, non-destructive testing, and remote sensing. The characteristic features of inverse problems are the non-uniqueness and instability of their solutions. Therefore, any reasonable solution method requires the use of regularization tools that select specific solutions and at the same time stabilize the inversion process. Recently, data-driven methods using deep learning techniques and neural networks demonstrated to significantly outperform classical solution methods for inverse problems. In this chapter, we give an overview of inverse problems and demonstrate the necessity of regularization concepts for their solution. We show that neural networks can be used for the data-driven solution of inverse problems and review existing deep learning methods for inverse problems. In particular, we view these deep learning methods from the perspective of regularization theory, the mathematical foundation of stable solution methods for inverse problems. This chapter is more than just a review as many of the presented theoretical results extend existing ones.

math.NA↗

arXiv2020

Sparse aNETT for Solving Inverse Problems with Deep Learning

We propose a sparse reconstruction framework (aNETT) for solving inverse problems. Opposed to existing sparse reconstruction techniques that are based on linear sparsifying transforms, we train an autoencoder network $D \circ E$ with $E$ acting as a nonlinear sparsifying transform and minimize a Tikhonov functional with learned regularizer formed by the $\ell^q$-norm of the encoder coefficients and a penalty for the distance to the data manifold. We propose a strategy for training an autoencoder based on a sample set of the underlying image class such that the autoencoder is independent of the forward operator and is subsequently adapted to the specific forward model. Numerical results are presented for sparse view CT, which clearly demonstrate the feasibility, robustness and the improved generalization capability and stability of aNETT over post-processing networks.

math.NA↗

arXiv2020

Data-consistent neural networks for solving nonlinear inverse problems

Data assisted reconstruction algorithms, incorporating trained neural networks, are a novel paradigm for solving inverse problems. One approach is to first apply a classical reconstruction method and then apply a neural network to improve its solution. Empirical evidence shows that such two-step methods provide high-quality reconstructions, but they lack a convergence analysis. In this paper we formalize the use of such two-step approaches with classical regularization theory. We propose data-consistent neural networks that we combine with classical regularization methods. This yields a data-driven regularization method for which we provide a full convergence analysis with respect to noise. Numerical simulations show that compared to standard two-step deep learning methods, our approach provides better stability with respect to structural changes in the test set, while performing similarly on test data similar to the training set. Our method provides a stable solution of inverse problems that exploits both the known nonlinear forward model as well as the desired solution manifold from data.

math.NA↗

arXiv2020

A Joint Deep Learning Approach for Automated Liver and Tumor Segmentation

Hepatocellular carcinoma (HCC) is the most common type of primary liver cancer in adults, and the most common cause of death of people suffering from cirrhosis. The segmentation of liver lesions in CT images allows assessment of tumor load, treatment planning, prognosis and monitoring of treatment response. Manual segmentation is a very time-consuming task and in many cases, prone to inaccuracies and automatic tools for tumor detection and segmentation are desirable. In this paper, we compare two network architectures, one that is composed of one neural network and manages the segmentation task in one step and one that consists of two consecutive fully convolutional neural networks. The first network segments the liver whereas the second network segments the actual tumor inside the liver. Our networks are trained on a subset of the LiTS (Liver Tumor Segmentation) Challenge and evaluated on data.

cs.CV↗

arXiv2020

The conical Radon transform with vertices on triple lines

We study the inversion of the conical Radon which integrates a function in three-dimensional space from integrals over circular cones. The conical Radon recently got significant attention due to its relevance in various imaging applications such as Compton camera imaging and single scattering optical tomography. The unrestricted conical Radon transform is over-determined because the manifold of all cones depends on six variables: the center position, the axis orientation and the opening angle of the cone. In this work, we consider a particular restricted transform using triple line sensors where integrals over a three-dimensional set of cones are collected, determined by a one-dimensional vertex set, a one-dimensional set of central axes, and the one-dimensional set of opening angle. As the main result in this paper, we derive an analytic inversion formula for the restricted conical Radon transform. Along that way we define a certain ray transform adapted to the triple line sensor for which we establish an analytic inversion formula.

math.NA↗

arXiv2020

Unsupervised Adaptive Neural Network Regularization for Accelerated Radial Cine MRI

In this work, we propose an iterative reconstruction scheme (ALONE - Adaptive Learning Of NEtworks) for 2D radial cine MRI based on ground truth-free unsupervised learning of shallow convolutional neural networks. The network is trained to approximate patches of the current estimate of the solution during the reconstruction. By imposing a shallow network topology and constraining the $L_2$-norm of the learned filters, the network's representation power is limited in order not to be able to recover noise. Therefore, the network can be interpreted to perform a low dimensional approximation of the patches for stabilizing the inversion process. We compare the proposed reconstruction scheme to two ground truth-free reconstruction methods, namely a well known Total Variation (TV) minimization and an unsupervised adaptive Dictionary Learning (DIC) method. The proposed method outperforms both methods with respect to all reported quantitative measures. Further, in contrast to DIC, where the sparse approximation of the patches involves the solution of a complex optimization problem, ALONE only requires a forward pass of all patches through the shallow network and therefore significantly accelerates the reconstruction.

eess.IV↗

arXiv2020

Deep synthesis regularization of inverse problems

Recently, a large number of efficient deep learning methods for solving inverse problems have been developed and show outstanding numerical performance. For these deep learning methods, however, a solid theoretical foundation in the form of reconstruction guarantees is missing. In contrast, for classical reconstruction methods, such as convex variational and frame-based regularization, theoretical convergence and convergence rate results are well established. In this paper, we introduce deep synthesis regularization (DESYRE) using neural networks as nonlinear synthesis operator bridging the gap between these two worlds. The proposed method allows to exploit the deep learning benefits of being well adjustable to available training data and on the other hand comes with a solid mathematical foundation. We present a complete convergence analysis with convergence rates for the proposed deep synthesis regularization. We present a strategy for constructing a synthesis network as part of an analysis-synthesis sequence together with an appropriate training strategy. Numerical results show the plausibility of our approach.

math.NA↗

arXiv2020

Neural Networks-based Regularization for Large-Scale Medical Image Reconstruction

In this paper we present a generalized Deep Learning-based approach for solving ill-posed large-scale inverse problems occuring in medical image reconstruction. Recently, Deep Learning methods using iterative neural networks and cascaded neural networks have been reported to achieve state-of-the-art results with respect to various quantitative quality measures as PSNR, NRMSE and SSIM across different imaging modalities. However, the fact that these approaches employ the forward and adjoint operators repeatedly in the network architecture requires the network to process the whole images or volumes at once, which for some applications is computationally infeasible. In this work, we follow a different reconstruction strategy by decoupling the regularization of the solution from ensuring consistency with the measured data. The regularization is given in the form of an image prior obtained by the output of a previously trained neural network which is used in a Tikhonov regularization framework. By doing so, more complex and sophisticated network architectures can be used for the removal of the artefacts or noise than it is usually the case in iterative networks. Due to the large scale of the considered problems and the resulting computational complexity of the employed networks, the priors are obtained by processing the images or volumes as patches or slices. We evaluated the method for the cases of 3D cone-beam low dose CT and undersampled 2D radial cine MRI and compared it to a total variation-minimization-based reconstruction algorithm as well as to a method with regularization based on learned overcomplete dictionaries. The proposed method outperformed all the reported methods with respect to all chosen quantitative measures and further accelerates the regularization step in the reconstruction by several orders of magnitude.

eess.IV↗

arXiv2019

Sparse regularization of inverse problems by operator-adapted frame thresholding

We analyze sparse frame based regularization of inverse problems by means of a diagonal frame decomposition (DFD) for the forward operator, which generalizes the SVD. The DFD allows to define a non-iterative (direct) operator-adapted frame thresholding approach which we show to provide a convergent regularization method with linear convergence rates. These results will be compared to the well-known analysis and synthesis variants of sparse $\ell^1$-regularization which are usually implemented thorough iterative schemes. If the frame is a basis (non-redundant case), the three versions of sparse regularization, namely synthesis and analysis variants of $\ell^1$ regularization as well as the DFD thresholding are equivalent. However, in the redundant case, those three approaches are pairwise different.

math.NA↗