arXiv subjects

Dirk A. Lorenz

Publications and source records attributed to Dirk A. Lorenz.

At least 37 records · Page 2Linked to original sources

arXiv2019

Orlicz-space regularization for optimal transport and algorithms for quadratic regularization

We investigate the continuous optimal transport problem in the so-called Kantorovich form, i.e. given two Radon measures on two compact sets, we seek an optimal transport plan which is another Radon measure on the product of the sets that has these two measures as marginals and minimizes a certain cost function. We consider regularization of the problem with so-called Young's functions, which forces the optimal transport plan to be a function in the corresponding Orlicz space rather than a Radon measure. We derive the predual problem and show strong duality and existence of primal solutions to the regularized problem. Existence of (pre-)dual solutions will be shown for the special case of $L^p$ regularization for $p\geq2$. Then we derive four algorithms to solve the dual problem of the quadratically regularized problem: A cyclic projection method, a dual gradient decent, a simple fixed point method, and Nesterov's accelerated gradient, all of which have a very low cost per iteration.

math.OC↗

arXiv2019

Quadratically regularized optimal transport

We investigate the problem of optimal transport in the so-called Kantorovich form, i.e. given two Radon measures on two compact sets, we seek an optimal transport plan which is another Radon measure on the product of the sets that has these two measures as marginals and minimizes a certain cost function. We consider quadratic regularization of the problem, which forces the optimal transport plan to be a square integrable function rather than a Radon measure. We derive the dual problem and show strong duality and existence of primal and dual solutions to the regularized problem. Then we derive two algorithms to solve the dual problem of the regularized problem: A Gauss-Seidel method and a semismooth quasi-Newton method and investigate both methods numerically. Our experiments show that the methods perform well even for small regularization parameters. Quadratic regularization is of interest since the resulting optimal transport plans are sparse, i.e. they have a small support (which is not the case for the often used entropic regularization where the optimal transport plan always has full measure).

math.OC↗

arXiv2018

Non-stationary Douglas-Rachford and alternating direction method of multipliers: adaptive stepsizes and convergence

We revisit the classical Douglas-Rachford (DR) method for finding a zero of the sum of two maximal monotone operators. Since the practical performance of the DR method crucially depends on the stepsizes, we aim at developing an adaptive stepsize rule. To that end, we take a closer look at a linear case of the problem and use our findings to develop a stepsize strategy that eliminates the need for stepsize tuning. We analyze a general non-stationary DR scheme and prove its convergence for a convergent sequence of stepsizes with summable increments. This, in turn, proves the convergence of the method with the new adaptive stepsize rule. We also derive the related non-stationary alternating direction method of multipliers (ADMM) from such a non-stationary DR method. We illustrate the efficiency of the proposed methods on several numerical examples.

math.OC↗

arXiv2018

Sarrus rules and dihedral groups

This paper is devoted to the analysis of a false generalization of the rule of Sarrus and its properties that can be derived with the help of dihedral groups. Further, we discuss a Sarrus-like scheme that could be helpful for students to memorize the calculation of a $4\times 4$ determinant.

math.CO↗

arXiv2018

Denoising of image gradients and total generalized variation denoising

We revisit total variation denoising and study an augmented model where we assume that an estimate of the image gradient is available. We show that this increases the image reconstruction quality and derive that the resulting model resembles the total generalized variation denoising method, thus providing a new motivation for this model. Further, we propose to use a constraint denoising model and develop a variational denoising model that is basically parameter free, i.e. all model parameters are estimated directly from the noisy image. Moreover, we use Chambolle-Pock's primal dual method as well as the Douglas-Rachford method for the new models. For the latter one has to solve large discretizations of partial differential equations. We propose to do this in an inexact manner using the preconditioned conjugate gradients method and derive preconditioners for this. Numerical experiments show that the resulting method has good denoising properties and also that preconditioning does increase convergence speed significantly. Finally we analyze the duality gap of different formulations of the TGV denoising problem and derive a simple stopping criterion.

math.OC↗

arXiv2018

The Randomized Kaczmarz Method with Mismatched Adjoint

This paper investigates the randomized version of the Kaczmarz method to solve linear systems in the case where the adjoint of the system matrix is not exact---a situation we refer to as "mismatched adjoint". We show that the method may still converge both in the over- and underdetermined consistent case under appropriate conditions, and we calculate the expected asymptotic rate of linear convergence. Moreover, we analyze the inconsistent case and obtain results for the method with mismatched adjoint as for the standard method. Finally, we derive a method to compute optimized probabilities for the choice of the rows and illustrate our findings with numerical example.

math.NA↗

arXiv2016

An extended Perona-Malik model based on probabilistic models

The Perona-Malik model has been very successful at restoring images from noisy input. In this paper, we reinterpret the Perona-Malik model in the language of Gaussian scale mixtures and derive some extensions of the model. Specifically, we show that the expectation-maximization (EM) algorithm applied to Gaussian scale mixtures leads to the lagged-diffusivity algorithm for computing stationary points of the Perona-Malik diffusion equations. Moreover, we show how mean field approximations to these Gaussian scale mixtures lead to a modification of the lagged-diffusivity algorithm that better captures the uncertainties in the restoration. Since this modification can be hard to compute in practice we propose relaxations to the mean field objective to make the algorithm computationally feasible. Our numerical experiments show that this modified lagged-diffusivity algorithm often performs better at restoring textured areas and fuzzy edges than the unmodified algorithm. As a second application of the Gaussian scale mixture framework, we show how an efficient sampling procedure can be obtained for the probabilistic model, making the computation of the conditional mean and other expectations algorithmically feasible. Again, the resulting algorithm has a strong resemblance to the lagged-diffusivity algorithm. Finally, we show that a probabilistic version of the Mumford-Shah segementation model can be obtained in the same framework with a discrete edge-prior.

cs.CV↗

arXiv2016

A Primal-Dual Homotopy Algorithm for $\ell_{1}$-Minimization with $\ell_{\infty}$-Constraints

In this paper we propose a primal-dual homotopy method for $\ell_1$-minimization problems with infinity norm constraints in the context of sparse reconstruction. The natural homotopy parameter is the value of the bound for the constraints and we show that there exists a piecewise linear solution path with finitely many break points for the primal problem and a respective piecewise constant path for the dual problem. We show that by solving a small linear program, one can jump to the next primal break point and then, solving another small linear program, a new optimal dual solution is calculated which enables the next such jump in the subsequent iteration. Using a theorem of the alternative, we show that the method never gets stuck and indeed calculates the whole path in a finite number of steps. Numerical experiments demonstrate the effectiveness of our algorithm. In many cases, our method significantly outperforms commercial LP solvers; this is possible since our approach employs a sequence of considerably simpler auxiliary linear programs that can be solved efficiently with specialized active-set strategies.

math.OC↗

arXiv2016

Linear convergence of the Randomized Sparse Kaczmarz Method

The randomized version of the Kaczmarz method for the solution of linear systems is known to converge linearly in expectation. In this work we extend this result and show that the recently proposed Randomized Sparse Kaczmarz method for recovery of sparse solutions, as well as many variants, also converges linearly in expectation. The result is achieved in the framework of split feasibility problems and their solution by randomized Bregman projections with respect to strongly convex functions. To obtain the expected convergence rates we prove extensions of error bounds for projections. The convergence result is shown to hold in more general settings involving smooth convex functions, piecewise linear-quadratic functions and also the regularized nuclear norm, which is used in the area of low rank matrix problems. Numerical experiments indicate that the Randomized Sparse Kaczmarz method provides advantages over both the non-randomized and the non-sparse Kaczmarz methods for the solution of over- and under-determined linear systems.

math.OC↗

arXiv2016

Rank-optimal weighting or "How to be best in the OECD Better Life Index?"

We present a method of rank-optimal weighting which can be used to explore the best possible position of a subject in a ranking based on a composite indicator by means of a mathematical optimization problem. As an example, we explore the dataset of the OECD Better Life Index and compute for each country a weight vector which brings it as far up in the ranking as possible with the greatest advance of the immediate rivals. The method is able to answer the question "What is the best possible rank a country can achieve with a given set of weighted indicators?" Typically, weights in composite indicators are justified normatively and not empirically. Our approach helps to give bounds on what is achievable by such normative judgments from a purely output-oriented and strongly competitive perspective. The method can serve as a basis for exact bounds in sensitivity analysis focused on ranking positions. In the OECD Better Life Index data we find that 19 out the 36 countries in the OECD Better Life Index 2014 can be brought to the top of the ranking by specific weights. We give a table of weights for each country which brings it to its highest possible position. Many countries achieve their best rank by focusing on their strong dimensions and setting the weights of many others to zero. Although setting dimensions to zero is possible in the OECD's online tool, this contradicts the idea of better life being multidimensional in essence. We discuss modifications of the optimization problem which could take this into account, e.g. by allowing only a minimal weight of one. Methods to find rank-optimal weights can be useful for various multidimensional datasets like the ones used to rank universities or employers.

math.OC↗

arXiv2016

Flexible sparse regularization

The seminal paper of Daubechies, Defrise, DeMol made clear that $\ell^p$ spaces with $p\in [1,2)$ and $p$-powers of the corresponding norms are appropriate settings for dealing with reconstruction of sparse solutions of ill-posed problems by regularization. It seems that the case $p=1$ provides the best results in most of the situations compared to the cases $p\in (1,2)$. An extensive literature gives great credit also to using $\ell^p$ spaces with $p\in (0,1)$ together with the corresponding quasinorms, although one has to tackle challenging numerical problems raised by the non-convexity of the quasi-norms. In any of these settings, either super, linear or sublinear, the question of how to choose the exponent $p$ has been not only a numerical issue, but also a philosophical one. In this work we introduce a more flexible way of sparse regularization by varying exponents. We introduce the corresponding functional analytic framework, that leaves the setting of normed spaces but works with so-called F-norms. One curious result is that there are F-norms which generate the $\ell^1$ space, but they are strictly convex, while the $\ell^1$-norm is just convex.

math.NA↗

arXiv2014

An inertial forward-backward algorithm for monotone inclusions

In this paper, we propose an inertial forward backward splitting algorithm to compute a zero of the sum of two monotone operators, with one of the two operators being co-coercive. The algorithm is inspired by the accelerated gradient method of Nesterov, but can be applied to a much larger class of problems including convex-concave saddle point problems and general monotone inclusions. We prove convergence of the algorithm in a Hilbert space setting and show that several recently proposed first-order methods can be obtained as special cases of the general algorithm. Numerical results show that the proposed algorithm converges faster than existing methods, while keeping the computational cost of each iteration basically unchanged.

cs.CV↗

arXiv2014

Testable uniqueness conditions for empirical assessment of undersampling levels in total variation-regularized x-ray CT

We study recoverability in fan-beam computed tomography (CT) with sparsity and total variation priors: how many underdetermined linear measurements suffice for recovering images of given sparsity? Results from compressed sensing (CS) establish such conditions for, e.g., random measurements, but not for CT. Recoverability is typically tested by checking whether a computed solution recovers the original. This approach cannot guarantee solution uniqueness and the recoverability decision therefore depends on the optimization algorithm. We propose new computational methods to test recoverability by verifying solution uniqueness conditions. Using both reconstruction and uniqueness testing we empirically study the number of CT measurements sufficient for recovery on new classes of sparse test images. We demonstrate an average-case relation between sparsity and sufficient sampling and observe a sharp phase transition as known from CS, but never established for CT. In addition to assessing recoverability more reliably, we show that uniqueness tests are often the faster option.

math.OC↗

arXiv2014

Imaging with Kantorovich-Rubinstein discrepancy

We propose the use of the Kantorovich-Rubinstein norm from optimal transport in imaging problems. In particular, we discuss a variational regularisation model endowed with a Kantorovich-Rubinstein discrepancy term and total variation regularization in the context of image denoising and cartoon-texture decomposition. We point out connections of this approach to several other recently proposed methods such as total generalized variation and norms capturing oscillating patterns. We also show that the respective optimization problem can be turned into a convex-concave saddle point problem with simple constraints and hence, can be solved by standard tools. Numerical examples exhibit interesting features and favourable performance for denoising and cartoon-texture decomposition.

cs.CV↗

arXiv2014

A sparse Kaczmarz solver and a linearized Bregman method for online compressed sensing

An algorithmic framework to compute sparse or minimal-TV solutions of linear systems is proposed. The framework includes both the Kaczmarz method and the linearized Bregman method as special cases and also several new methods such as a sparse Kaczmarz solver. The algorithmic framework has a variety of applications and is especially useful for problems in which the linear measurements are slow and expensive to obtain. We present examples for online compressed sensing, TV tomographic reconstruction and radio interferometry.

math.OC↗

arXiv2013

The Linearized Bregman Method via Split Feasibility Problems: Analysis and Generalizations

The linearized Bregman method is a method to calculate sparse solutions to systems of linear equations. We formulate this problem as a split feasibility problem, propose an algorithmic framework based on Bregman projections and prove a general convergence result for this framework. Convergence of the linearized Bregman method will be obtained as a special case. Our approach also allows for several generalizations such as other objective functions, incremental iterations, incorporation of non-gaussian noise models or box constraints.

math.OC↗

arXiv2013

Computing and Analyzing Recoverable Supports for Sparse Reconstruction

Designing computational experiments involving $\ell_1$ minimization with linear constraints in a finite-dimensional, real-valued space for receiving a sparse solution with a precise number $k$ of nonzero entries is, in general, difficult. Several conditions were introduced which guarantee that, for small $k$ and for certain matrices, simply placing entries with desired characteristics on a randomly chosen support will produce vectors which can be recovered by $\ell_1$ minimization. In this work, we consider the case of large $k$ and propose both a methodology to quickly check whether a given vector is recoverable, and to construct vectors with the largest possible support. Moreover, we gain new insights in the recoverability in a non-asymptotic regime. The theoretical results are illustrated with computational experiments.

math.OC↗

arXiv2013

An Infeasible-Point Subgradient Method Using Adaptive Approximate Projections

We propose a new subgradient method for the minimization of nonsmooth convex functions over a convex set. To speed up computations we use adaptive approximate projections only requiring to move within a certain distance of the exact projections (which decreases in the course of the algorithm). In particular, the iterates in our method can be infeasible throughout the whole procedure. Nevertheless, we provide conditions which ensure convergence to an optimal feasible point under suitable assumptions. One convergence result deals with step size sequences that are fixed a priori. Two other results handle dynamic Polyak-type step sizes depending on a lower or upper estimate of the optimal objective function value, respectively. Additionally, we briefly sketch two applications: Optimization with convex chance constraints, and finding the minimum l1-norm solution to an underdetermined linear system, an important problem in Compressed Sensing.

math.OC↗