arXiv subjects

Martin Holler

Publications and source records attributed to Martin Holler.

29 records · Page 2Linked to original sources

arXiv2023

Latent-space disentanglement with untrained generator networks for the isolation of different motion types in video data

Isolating different types of motion in video data is a highly relevant problem in video analysis. Applications can be found, for example, in dynamic medical or biological imaging, where the analysis and further processing of the dynamics of interest is often complicated by additional, unwanted dynamics, such as motion of the measurement subject. In this work, it is empirically shown that a representation of video data via untrained generator networks, together with a specific technique for latent space disentanglement that uses minimal, one-dimensional information on some of the underlying dynamics, allows to efficiently isolate different, highly non-linear motion types. In particular, such a representation allows to freeze any selection of motion types, and to obtain accurate independent representations of other dynamics of interest. Obtaining such a representation does not require any pre-training on a training data set, i.e., all parameters of the generator network are learned directly from a single video.

eess.IV↗

arXiv2023

Learning-informed parameter identification in nonlinear time-dependent PDEs

We introduce and analyze a method of learning-informed parameter identification for partial differential equations (PDEs) in an all-at-once framework. The underlying PDE model is formulated in a rather general setting with three unknowns: physical parameter, state and nonlinearity. Inspired by advances in machine learning, we approximate the nonlinearity via a neural network, whose parameters are learned from measurement data. The later is assumed to be given as noisy observations of the unknown state, and both the state and the physical parameters are identified simultaneously with the parameters of the neural network. Moreover, diverging from the classical approach, the proposed all-at-once setting avoids constructing the parameter-to-state map by explicitly handling the state as additional variable. The practical feasibility of the proposed method is confirmed with experiments using two different algorithmic settings: A function-space algorithm based on analytic adjoints as well as a purely discretized setting using standard machine learning algorithms.

math.OC↗

arXiv2023

A Note on the Regularity of Images Generated by Convolutional Neural Networks

The regularity of images generated by convolutional neural networks, such as the U-net, generative networks, or the deep image prior, is analyzed. In a resolution-independent, infinite dimensional setting, it is shown that such images, represented as functions, are always continuous and, in some circumstances, even continuously differentiable, contradicting the widely accepted modeling of sharp edges in images via jump discontinuities. While such statements require an infinite dimensional setting, the connection to (discretized) neural networks used in practice is made by considering the limit as the resolution approaches infinity. As practical consequence, the results of this paper in particular provide analytical evidence that basic L2 regularization of network weights might lead to over-smoothed outputs.

cs.CV↗

arXiv2023

Accelerometry-based classification of circulatory states during out-of-hospital cardiac arrest

Objective: Exploit accelerometry data for an automatic, reliable, and prompt detection of spontaneous circulation during cardiac arrest, as this is both vital for patient survival and practically challenging. Methods: We developed a machine learning algorithm to automatically predict the circulatory state during cardiopulmonary resuscitation from 4-second-long snippets of accelerometry and electrocardiogram (ECG) data from pauses of chest compressions of real-world defibrillator records. The algorithm was trained based on 422 cases from the German Resuscitation Registry, for which ground truth labels were created by a manual annotation of physicians. It uses a kernelized Support Vector Machine classifier based on 49 features, which partially reflect the correlation between accelerometry and electrocardiogram data. Results: Evaluating 50 different test-training data splits, the proposed algorithm exhibits a balanced accuracy of 81.2%, a sensitivity of 80.6%, and a specificity of 81.8%, whereas using only ECG leads to a balanced accuracy of 76.5%, a sensitivity of 80.2%, and a specificity of 72.8%. Conclusion: The first method employing accelerometry for pulse/no-pulse decision yields a significant increase in performance compared to single ECG-signal usage. Significance: This shows that accelerometry provides relevant information for pulse/no-pulse decisions. In application, such an algorithm may be used to simplify retrospective annotation for quality management and, moreover, to support clinicians to assess circulatory state during cardiac arrest treatment.

eess.SP↗

arXiv2022

Fast and memory-efficient reconstruction of sparse Poisson data in listmode with non-smooth priors with application to time-of-flight PET

Complete time of flight (TOF) sinograms of state-of-the-art TOF PET scanners have a large memory footprint. Currently, they contain ~4e9 data bins which amount to ~17GB in 32bit floating point precision. Using iterative algorithms to reconstruct such enormous TOF sinograms becomes increasingly challenging due to the memory requirements and the computation time needed to evaluate the forward model for every data bin. This is especially true for more advanced optimization algorithms such as the SPDHG algorithm which allows for the use of non-smooth priors using subsets with guaranteed convergence. SPDHG requires the storage of additional sinograms in memory, which severely limits its application to data sets from state-of-the-art TOF PET systems. Motivated by the generally sparse nature of the TOF sinograms, we propose and analyze a new listmode (LM) extension of the SPDHG algorithm for reconstruction of sparse data following a Poisson distribution. The new algorithm is evaluated based on 2D and 3D simulations, and a real dataset acquired on a recent TOF PET/CT system. The performance of the newly proposed LM SPDHG algorithm is compared against the conventional sinogram SPDHG and the listmode EM-TV algorithm. We show that the speed of convergence of LM-SPDHG is equivalent the original SPDHG using binned data. However, we find that for a TOF PET system with 400ps TOF resolution, LM-SPDHG reduces the required memory from ~56GB to 0.7GB for a short dynamic frame with 1e7 counts and to 12.4GB for a long static acquisition with 5e8 counts. In contrast to SPDHG, the reduced memory requirements of LM-SPDHG enable a pure GPU implementation on state-of-the-art GPUs which will substantially accelerate reconstruction times. This in turn will allow the application of LM-SPDHG in routine clinical practice where short reconstruction times are crucial.

physics.med-ph↗

arXiv2021

A Generative Variational Model for Inverse Problems in Imaging

This paper is concerned with the development, analysis and numerical realization of a novel variational model for the regularization of inverse problems in imaging. The proposed model is inspired by the architecture of generative convolutional neural networks; it aims to generate the unknown from variables in a latent space via multi-layer convolutions and non-linear penalties, and penalizes an associated cost. In contrast to conventional neural-network-based approaches, however, the convolution kernels are learned directly from the measured data such that no training is required. The present work provides a mathematical analysis of the proposed model in a function space setting, including proofs for regularity and existence/stability of solutions, and convergence for vanishing noise. Moreover, in a discretized setting, a numerical algorithm for solving various types of inverse problems with the proposed model is derived. Numerical results are provided for applications in inpainting, denoising, deblurring under noise, super-resolution and JPEG decompression with multiple test images.

math.OC↗

arXiv2021

Regularization Graphs -- A unified framework for variational regularization of inverse problems

We introduce and study a mathematical framework for a broad class of regularization functionals for ill-posed inverse problems: Regularization Graphs. Regularization graphs allow to construct functionals using as building blocks linear operators and convex functionals, assembled by means of operators that can be seen as generalizations of classical infimal convolution operators. This class of functionals exhaustively covers existing regularization approaches and it is flexible enough to craft new ones in a simple and constructive way. We provide well-posedness and convergence results with the proposed class of functionals in a general setting. Further, we consider a bilevel optimization approach to learn optimal weights for such regularization graphs from training data. We demonstrate that this approach is capable of optimizing the structure and the complexity of a regularization graph, allowing, for example, to automatically select a combination of regularizers that is optimal for given training data.

math.OC↗

arXiv2019

Higher-order total variation approaches and generalisations

Over the last decades, the total variation (TV) evolved to one of the most broadly-used regularisation functionals for inverse problems, in particular for imaging applications. When first introduced as a regulariser, higher-order generalisations of TV were soon proposed and studied with increasing interest, which led to a variety of different approaches being available today. We review several of these approaches, discussing aspects ranging from functional-analytic foundations to regularisation theory for linear inverse problems in Banach space, and provide a unified framework concerning well-posedness and convergence for vanishing noise level for respective Tikhonov regularisation. This includes general higher orders of TV, additive and infimal-convolution multi-order total variation, total generalised variation (TGV), and beyond. Further, numerical optimisation algorithms are developed and discussed that are suitable for solving the Tikhonov minimisation problem for all presented models. Focus is laid in particular on covering the whole pipeline starting at the discretisation of the problem and ending at concrete, implementable iterative procedures. A major part of this review is finally concerned with presenting examples and applications where higher-order TV approaches turned out to be beneficial. These applications range from classical inverse problems in imaging such as denoising, deconvolution, compressed sensing, optical-flow estimation and decompression, to image reconstruction in medical imaging and beyond, including magnetic resonance imaging (MRI), computed tomography (CT), magnetic-resonance positron emission tomography (MR-PET), and electron tomography.

math.OC↗

arXiv2019

Non-smooth variational regularization for processing manifold-valued data

Many methods for processing scalar and vector valued images, volumes and other data in the context of inverse problems are based on variational formulations. Such formulations require appropriate regularization functionals that model expected properties of the object to reconstruct. Prominent examples of regularization functionals in a vector-space context are the total variation (TV) and the Mumford-Shah functional, as well as higher-order schemes such as total generalized variation models. Driven by applications where the signals or data live in nonlinear manifolds, there has been quite some interest in developing analogous methods for nonlinear, manifold-valued data recently. In this chapter, we consider various variational regularization methods for manifold-valued data. In particular, we consider TV minimization as well as higher order models such as total generalized variation (TGV). Also, we discuss (discrete) Mumford-Shah models and related methods for piecewise constant data. We develop discrete energies for denoising and report on algorithmic approaches to minimize them. Further, we also deal with the extension of such methods to incorporate indirect measurement terms, thus addressing the inverse problem setup. Finally, we discuss wavelet sparse regularization for manifold-valued data.

math.NA↗

arXiv2018

Coupled regularization with multiple data discrepancies

We consider a class of regularization methods for inverse problems where a coupled regularization is employed for the simultaneous reconstruction of data from multiple sources. Applications for such a setting can be found in multi-spectral or multi-modality inverse problems, but also in inverse problems with dynamic data. We consider this setting in a rather general framework and derive stability and convergence results, including convergence rates. In particular, we show how parameter choice strategies adapted to the interplay of different data channels allow to improve upon convergence rates that would be obtained by treating all channels equally. Motivated by concrete applications, our results are obtained under rather general assumptions that allow to include the Kullback-Leibler divergence as data discrepancy term. To simplify their application to concrete settings, we further elaborate several practically relevant special cases in detail. To complement the analytical results, we also provide an algorithmic framework and source code that allows to solve a class of jointly regularized inverse problems with any number of data discrepancies. As concrete applications, we show numerical results for multi-contrast MR and joint MR-PET reconstruction.

math.OC↗

arXiv2017

A function space framework for structural total variation regularization with applications in inverse problems

In this work, we introduce a function space setting for a wide class of structural/weighted total variation (TV) regularization methods motivated by their applications in inverse problems. In particular, we consider a regularizer that is the appropriate lower semi-continuous envelope (relaxation) of a suitable total variation type functional initially defined for sufficiently smooth functions. We study examples where this relaxation can be expressed explicitly, and we also provide refinements for weighted total variation for a wide range of weights. Since an integral characterization of the relaxation in function space is, in general, not always available, we show that, for a rather general linear inverse problems setting, instead of the classical Tikhonov regularization problem, one can equivalently solve a saddle-point problem where no a priori knowledge of an explicit formulation of the structural TV functional is needed. In particular, motivated by concrete applications, we deduce corresponding results for linear inverse problems with norm and Poisson log-likelihood data discrepancy terms. Finally, we provide proof-of-concept numerical examples where we solve the saddle-point problem for weighted TV denoising as well as for MR guided PET image reconstruction.

math.OC↗