arXiv subjects

Martin Burger

Publications and source records attributed to Martin Burger.

At least 55 records · Page 3Linked to original sources

arXiv2022

Dynamic optimal transport on networks

We study a dynamic optimal transport problem on a network. Despite the cost for transport along the edges, an additional cost, scaled with a parameter $κ$, has to be paid for interchanging mass between edges and vertices. We show existence f minimisers using duality and discuss the relationship of the model to other metrics such as Fisher-Rao and the classical Wasserstein metric. Finally, we examine the limiting behaviour of the model in terms of the parameter $κ$.

math.AP↗

arXiv2022

On multi-species diffusion with size exclusion

We revisit a classical continuum model for the diffusion of multiple species with size-exclusion constraint, which leads to a degenerate nonlinear cross-diffusion system. The purpose of this article is twofold: first, it aims at a systematic study of the question of existence of weak solutions and their long-time asymptotic behaviour. Second, it provides a weak-strong stability estimate for a wide range of coefficients, which had been missing so far. In order to achieve the results mentioned above, we exploit the formal gradient-flow structure of the model with respect to a logarithmic entropy, which leads to best estimates in the full-interaction case, where all cross-diffusion coefficients are non-zero. Those are crucial to obtain the minimal Sobolev regularity needed for a weak-strong stability result. For meaningful cases when some of the coefficients vanish, we provide a novel existence result based on approximation by the full-interaction case.

math.AP↗

arXiv2022

Well-posedness of an integro-differential model for active Brownian particles

We propose a general strategy for solving nonlinear integro-differential evolution problems with periodic boundary conditions, where no direct maximum/minimum principle is available. This is motivated by the study of recent macroscopic models for active Brownian particles with repulsive interactions, consisting of advection-diffusion processes in the space of particle position and orientation. We focus on one of such models, namely a semilinear parabolic equation with a nonlinear active drift term, whereby the velocity depends on the particle orientation and angle-independent overall particle density (leading to a nonlocal term by integrating out the angular variable). The main idea of the existence analysis is to exploit a-priori estimates from (approximate) entropy dissipation. The global existence and uniqueness of weak solutions is shown using a two-step Galerkin approximation with appropriate cutoff in order to obtain nonnegativity, an upper bound on the overall density and preserve a-priori estimates. Our anyalysis naturally includes the case of finite systems, corresponding to the case of a finite number of directions. The Duhamel principle is then used to obtain additional regularity of the solution, namely continuity in time-space. Motivated by the class of initial data relevant for the application, which includes perfectly aligned particles (same orientation), we extend the well-posedness result to very weak solutions allowing distributional initial data with low regularity.

math.AP↗

arXiv2022

Region-of-Interest Prioritised Sampling for Constrained Autonomous Exploration Systems

Goal oriented autonomous operation of space rovers has been known to increase scientific output of a mission. In this work we present an algorithm, called the RoI Prioritised Sampling (RPS), that prioritises Region-of-Interests (RoIs) in an exploration scenario in order to utilise the limited resources of the imaging instrument on the rover effectively. This prioritisation is based on an estimator that evaluates the change in information content at consecutive spatial scales of the RoIs without calculating the finer scale reconstruction. The estimator, called the Refinement Indicator (RI), is motivated and derived. Multi-scale acquisition approaches, based on classical and multilevel compressed sensing, with respect to the single pixel camera architecture are discussed. The performance of the algorithm is verified on remote sensing images and compared with the state-of-the-art multi-resolution reconstruction algorithms. At the considered sub-sampling rates the RPS is shown to better utilise the system resources for reconstructing the RoIs.

eess.IV↗

arXiv2022

A Bregman Learning Framework for Sparse Neural Networks

We propose a learning framework based on stochastic Bregman iterations, also known as mirror descent, to train sparse neural networks with an inverse scale space approach. We derive a baseline algorithm called LinBreg, an accelerated version using momentum, and AdaBreg, which is a Bregmanized generalization of the Adam algorithm. In contrast to established methods for sparse training the proposed family of algorithms constitutes a regrowth strategy for neural networks that is solely optimization-based without additional heuristics. Our Bregman learning framework starts the training with very few initial parameters, successively adding only significant ones to obtain a sparse and expressive network. The proposed approach is extremely easy and efficient, yet supported by the rich mathematical theory of inverse scale space methods. We derive a statistically profound sparse parameter initialization strategy and provide a rigorous stochastic convergence analysis of the loss decay and additional convergence proofs in the convex regime. Using only 3.4% of the parameters of ResNet-18 we achieve 90.2% test accuracy on CIFAR-10, compared to 93.6% using the dense network. Our algorithm also unveils an autoencoder architecture for a denoising task. The proposed framework also has a huge potential for integrating sparse backpropagation and resource-friendly training.

cs.LG↗

arXiv2021

Variational Regularization in Inverse Problems and Machine Learning

This paper discusses basic results and recent developments on variational regularization methods, as developed for inverse problems. In a typical setup we review basic properties needed to obtain a convergent regularization scheme and further discuss the derivation of quantitative estimates respectively needed ingredients such as Bregman distances for convex functionals. In addition to the approach developed for inverse problems we will also discuss variational regularization in machine learning and work out some connections to the classical regularization theory. In particular we will discuss a reinterpretation of machine learning problems in the framework of regularization theory and a reinterpretation of variational methods for inverse problems in the framework of risk minimization. Moreover, we establish some previously unknown connections between error estimates in Bregman distances and generalization errors.

cs.LG↗

arXiv2021

Convex regularization in statistical inverse learning problems

We consider a statistical inverse learning problem, where the task is to estimate a function $f$ based on noisy point evaluations of $Af$, where $A$ is a linear operator. The function $Af$ is evaluated at i.i.d. random design points $u_n$, $n=1,...,N$ generated by an unknown general probability distribution. We consider Tikhonov regularization with general convex and $p$-homogeneous penalty functionals and derive concentration rates of the regularized solution to the ground truth measured in the symmetric Bregman distance induced by the penalty functional. We derive concrete rates for Besov norm penalties and numerically demonstrate the correspondence with the observed rates in the context of X-ray tomography.

stat.ML↗

arXiv2021

Learning convex regularizers satisfying the variational source condition for inverse problems

Variational regularization has remained one of the most successful approaches for reconstruction in imaging inverse problems for several decades. With the emergence and astonishing success of deep learning in recent years, a considerable amount of research has gone into data-driven modeling of the regularizer in the variational setting. Our work extends a recently proposed method, referred to as adversarial convex regularization (ACR), that seeks to learn data-driven convex regularizers via adversarial training in an attempt to combine the power of data with the classical convex regularization theory. Specifically, we leverage the variational source condition (SC) during training to enforce that the ground-truth images minimize the variational loss corresponding to the learned convex regularizer. This is achieved by adding an appropriate penalty term to the ACR training objective. The resulting regularizer (abbreviated as ACR-SC) performs on par with the ACR, but unlike ACR, comes with a quantitative convergence rate estimate.

cs.LG↗

arXiv2021

Gradient Flows and Nonlinear Power Methods for the Computation of Nonlinear Eigenfunctions

This chapter describes how gradient flows and nonlinear power methods in Banach spaces can be used to solve nonlinear eigenvector-dependent eigenvalue problems, and how convergence of (discretized) approximations can be verified. We review several flows from literature, which were proposed to compute nonlinear eigenfunctions, and show that they all relate to normalized gradient flows. Furthermore, we show that the implicit Euler discretization of gradient flows gives rise to a nonlinear power method of the proximal operator and prove their convergence to nonlinear eigenfunctions. Finally, we prove that $Γ$-convergence of functionals implies convergence of their ground states, which is important for discrete approximations.

math.NA↗

arXiv2021

Dynamically Sampled Nonlocal Gradients for Stronger Adversarial Attacks

The vulnerability of deep neural networks to small and even imperceptible perturbations has become a central topic in deep learning research. Although several sophisticated defense mechanisms have been introduced, most were later shown to be ineffective. However, a reliable evaluation of model robustness is mandatory for deployment in safety-critical scenarios. To overcome this problem we propose a simple yet effective modification to the gradient calculation of state-of-the-art first-order adversarial attacks. Normally, the gradient update of an attack is directly calculated for the given data point. This approach is sensitive to noise and small local optima of the loss function. Inspired by gradient sampling techniques from non-convex optimization, we propose Dynamically Sampled Nonlocal Gradient Descent (DSNGD). DSNGD calculates the gradient direction of the adversarial attack as the weighted average over past gradients of the optimization history. Moreover, distribution hyperparameters that define the sampling operation are automatically learned during the optimization scheme. We empirically show that by incorporating this nonlocal gradient information, we are able to give a more accurate estimation of the global descent direction on noisy and non-convex loss surfaces. In addition, we show that DSNGD-based attacks are on average 35% faster while achieving 0.9% to 27.1% higher success rates compared to their gradient descent-based counterparts.

cs.LG↗

arXiv2021

Reconstruction Methods in THz Single-pixel Imaging

The aim of this paper is to discuss some advanced aspects of image reconstruction in single-pixel cameras, focusing in particular on detectors in the THz regime. We discuss the reconstruction problem from a computational imaging perspective and provide a comparison of the effects of several state-of-the art regularization techniques. Moreover, we focus on some advanced aspects arising in practice with THz cameras, which lead to nonlinear reconstruction problems: the calibration of the beam reminiscent of the Retinex problem in imaging and phase recovery problems. Finally we provide an outlook to future challenges in the area.

math.OC↗

arXiv2021

The Inverse Problem of Magnetorelaxometry Imaging

The aim of this paper is to provide a solid mathematical discussion of the inverse problem in Magnetorelaxometry Imaging (MRXI), a currently developed technique for quantitative biomedical imaging using magnetic nanoparticles. We provide a detailed discussion of the mathematical modeling of the forward problems including possible ways to activate and measure, leading to a severely ill-posed linear inverse problem. Moreover, we formulate an idealized version of the inverse problem for infinitesimal small activation coils, which allows for a more detailed analysis of uniqueness issues. We propose a variational regularization approach to compute stable approximations of the solution and discuss its discretization and numerical solution. Results on synthetic are presented and improvements to methods used previously in practice are demonstrated. Finally we give an outlook to further questions and in particular experimental design.

math.NA↗

arXiv2021

Nonlinear Spectral Decompositions by Gradient Flows of One-Homogeneous Functionals

This paper establishes a theory of nonlinear spectral decompositions by considering the eigenvalue problem related to an absolutely one-homogeneous functional in an infinite-dimensional Hilbert space. This approach is both motivated by works for the total variation, where interesting results on the eigenvalue problem and the relation to the total variation flow have been proven previously, and by recent results on finite-dimensional polyhedral semi-norms, where gradient flows can yield spectral decompositions into eigenvectors. We provide a geometric characterization of eigenvectors via a dual unit ball and prove them to be subgradients of minimal norm. This establishes the connection to gradient flows, whose time evolution is a decomposition of the initial condition into subgradients of minimal norm. If these are eigenvectors, this implies an interesting orthogonality relation and the equivalence of the gradient flow to a variational regularization method and an inverse scale space flow. Indeed we verify that all scenarios where these equivalences were known before by other arguments - such as one-dimensional total variation, multidimensional generalizations to vector fields, or certain polyhedral semi-norms - yield spectral decompositions, and we provide further examples. We also investigate extinction times and extinction profiles, which we characterize as eigenvectors in a very general setting, generalizing several results from literature.

math.AP↗

arXiv2021

A Diffuse Interface Model for Cell Blebbing Including Membrane-Cortex Coupling with Linker Dynamics

The aim of this paper is to develop suitable models for the phenomenon of cell blebbing, which allow for computational predictions of mechanical effects including the crucial interaction of the cell membrane and the actin cortex. For this sake we resort to a two phase-field model that uses diffuse descriptions of both the membrane and the cortex, which in particular allows for a suitable description of the interaction via linker protein densities. Besides the detailed modelling we discuss some energetic aspects of the models and present a numerical scheme, which allows to carry out several computational studies. In those we demonstrate that several effects found in experiments can be reproduced, in particular bleb formation by cortex rupture, which was not possible by previous models without the linker dynamics.

q-bio.CB↗

arXiv2021

Neural Architecture Search via Bregman Iterations

We propose a novel strategy for Neural Architecture Search (NAS) based on Bregman iterations. Starting from a sparse neural network our gradient-based one-shot algorithm gradually adds relevant parameters in an inverse scale space manner. This allows the network to choose the best architecture in the search space which makes it well-designed for a given task, e.g., by adding neurons or skip connections. We demonstrate that using our approach one can unveil, for instance, residual autoencoders for denoising, deblurring, and classification tasks. Code is available at https://github.com/TimRoith/BregmanLearning.

cs.LG↗

arXiv2021

Kinetic Equations for Processes on Co-evolving Networks

The aim of this paper is to derive macroscopic equations for processes on large co-evolving networks, examples being opinion polarization with the emergence of filter bubbles or other social processes such as norm development. This leads to processes on graphs (or networks), where both the states of particles in nodes as well as the weights between them are updated in time. In our derivation we follow the basic paradigm of statistical mechanics: We start from paradigmatic microscopic models and derive a Liouville-type equation in a high-dimensional space including not only the node states in the network (corresponding to positions in mechanics), but also the edge weights between them. We then derive a natural (finite) marginal hierarchy and pass to an infinite limit. We will discuss the closure problem for this hierarchy and see that a simple mean-field solution can only arise if the weight distributions between nodes of equal states are concentrated. In a more interesting general case we propose a suitable closure at the level of a two-particle distribution (including the weight between them) and discuss some properties of the arising kinetic equations. Moreover, we highlight some structure-preserving properties of this closure and discuss its analysis in a minimal model. We discuss the application of our theory to some agent-based models in literature and discuss some open mathematical issues.

math.AP↗

arXiv2021

Identifying Untrustworthy Predictions in Neural Networks by Geometric Gradient Analysis

The susceptibility of deep neural networks to untrustworthy predictions, including out-of-distribution (OOD) data and adversarial examples, still prevent their widespread use in safety-critical applications. Most existing methods either require a re-training of a given model to achieve robust identification of adversarial attacks or are limited to out-of-distribution sample detection only. In this work, we propose a geometric gradient analysis (GGA) to improve the identification of untrustworthy predictions without retraining of a given model. GGA analyzes the geometry of the loss landscape of neural networks based on the saliency maps of their respective input. To motivate the proposed approach, we provide theoretical connections between gradients' geometrical properties and local minima of the loss function. Furthermore, we demonstrate that the proposed method outperforms prior approaches in detecting OOD data and adversarial attacks, including state-of-the-art and adaptive attacks.

cs.LG↗

arXiv2020

Variational regularisation for inverse problems with imperfect forward operators and general noise models

We study variational regularisation methods for inverse problems with imperfect forward operators whose errors can be modelled by order intervals in a partial order of a Banach lattice. We carry out analysis with respect to existence and convex duality for general data fidelity terms and regularisation functionals. Both for a-priori and a-posteriori parameter choice rules, we obtain convergence rates of the regularized solutions in terms of Bregman distances. Our results apply to fidelity terms such as Wasserstein distances, f-divergences, norms, as well as sums and infimal convolutions of those.

math.NA↗