Searcharxiv⌕ Search

arXiv subjects

Heinz H. Bauschke

Publications and source records attributed to Heinz H. Bauschke.

At least 91 records · Page 5Linked to original sources
arXiv2016

On Douglas-Rachford operators that fail to be proximal mappings

The problem of finding a zero of the sum of two maximally monotone operators is of central importance in optimization. One successful method to find such a zero is the Douglas-Rachford algorithm which iterates a firmly nonexpansive operator constructed from the resolvents of the given monotone operators. In the context of finding minimizers of convex functions, the resolvents are actually proximal mappings. Interestingly, as pointed out by Eckstein in 1989, the Douglas-Rachford operator itself may fail to be a proximal mapping. We consider the class of symmetric linear relations that are maximally monotone and prove the striking result that the Douglas-Rachford operator is generically not a proximal mapping.

math.OC↗
arXiv2015

The Douglas-Rachford algorithm in the affine-convex case

The Douglas-Rachford algorithm is a simple yet effective method for solving convex feasibility problems. However, if the underlying constraints are inconsistent, then the convergence theory is incomplete. We provide convergence results when one constraint is an affine subspace. As a consequence, we extend a result by Spingarn from halfspaces to general closed convex sets admitting least-squares solutions.

math.OC↗
arXiv2015

The resolvent average of monotone operators: dominant and recessive properties

Within convex analysis, a rich theory with various applications has been evolving since the proximal average of convex functions was first introduced over a decade ago. When one considers the subdifferential of the proximal average, a natural averaging operation of the subdifferentials of the averaged functions emerges. In the present paper we extend the reach of this averaging operation to the framework of monotone operator theory in Hilbert spaces, transforming it into the resolvent average. The theory of resolvent averages contains many desirable properties. In particular, we study a detailed list of properties of monotone operators and classify them as dominant or recessive with respect to the resolvent average. As a consequence, we recover a significant part of the theory of proximal averages. Furthermore, we shed new light on the proximal average and present novel results and desirable properties the proximal average possesses which have not been previously available.

math.OC↗
arXiv2015

On the order of the operators in the Douglas-Rachford algorithm

The Douglas-Rachford algorithm is a popular method for finding zeros of sums of monotone operators. By its definition, the Douglas-Rachford operator is not symmetric with respect to the order of the two operators. In this paper we provide a systematic study of the two possible Douglas-Rachford operators. We show that the reflectors of the underlying operators act as bijections between the fixed points sets of the two Douglas-Rachford operators. Some elegant formulae arise under additional assumptions. Various examples illustrate our results.

math.OC↗
arXiv2015

On Slater's condition and finite convergence of the Douglas-Rachford algorithm

The Douglas-Rachford algorithm is a classical and very successful method for solving optimization and feasibility problems. In this paper, we provide novel conditions sufficient for finite convergence in the context of convex feasibility problems. Our analysis builds upon, and considerably extends, pioneering work by Spingarn. Specifically, we obtain finite convergence in the presence of Slater's condition in the affine-polyhedral and in a hyperplanar-epigraphical case. Various examples illustrate our results. Numerical experiments demonstrate the competitiveness of the Douglas-Rachford algorithm for solving linear equations with a positivity constraint when compared to the method of alternating projections and the method of reflection-projection.

math.OC↗
arXiv2015

The Douglas-Rachford algorithm for two (not necessarily intersecting) affine subspaces

The Douglas--Rachford algorithm is a classical and very successful splitting method for finding the zeros of the sums of monotone operators. When the underlying operators are normal cone operators, the algorithm solves a convex feasibility problem. In this paper, we provide a detailed study of the Douglas--Rachford iterates and the corresponding {shadow sequence} when the sets are affine subspaces that do not necessarily intersect. We prove strong convergence of the shadows to the nearest generalized solution. Our results extend recent work from the consistent to the inconsistent case. Various examples are provided to illustrates the results.

math.OC↗
arXiv2015

Proximal point algorithm, Douglas-Rachford algorithm and alternating projections: a case study

Many iterative methods for solving optimization or feasibility problems have been invented, and often convergence of the iterates to some solution is proven. Under favourable conditions, one might have additional bounds on the distance of the iterate to the solution leading thus to worst case estimates, i.e., how fast the algorithm must converge. Exact convergence estimates are typically hard to come by. In this paper, we consider the complementary problem of finding best case estimates, i.e., how slow the algorithm has to converge, and we also study exact asymptotic rates of convergence. Our investigation focuses on convex feasibility in the Euclidean plane, where one set is the real axis while the other is the epigraph of a convex function. This case study allows us to obtain various convergence rate results. We focus on the popular method of alternating projections and the Douglas-Rachford algorithm. These methods are connected to the proximal point algorithm which is also discussed. Our findings suggest that the Douglas-Rachford algorithm outperforms the method of alternating projections in the absence of constraint qualifications. Various examples illustrate the theory.

math.OC↗
arXiv2014

Stadium norm and Douglas-Rachford splitting: a new approach to road design optimization

The basic optimization problem of road design is quite challenging due to a objective function that is the sum of nonsmooth functions and the presence of set constraints. In this paper, we model and solve this problem by employing the Douglas-Rachford splitting algorithm. This requires a careful study of new proximity operators related to minimizing area and to the stadium norm. We compare our algorithm to a state-of-the-art projection algorithm. Our numerical results illustrate the potential of this algorithm to significantly reduce cost in road design.

math.OC↗
arXiv2014

On the convexity of piecewise-defined functions

Functions that are piecewise defined are a common sight in mathematics while convexity is a property especially desired in optimization. Suppose now a piecewise-defined function is convex on each of its defining components - when can we conclude that the entire function is convex? In this paper we provide several convenient, verifiable conditions guaranteeing convexity (or the lack thereof). Several examples are presented to illustrate our results.

math.CA↗
arXiv2014

On the finite convergence of a projected cutter method

The subgradient projection iteration is a classical method for solving a convex inequality. Motivated by works of Polyak and of Crombez, we present and analyze a more general method for finding a fixed point of a cutter, provided that the fixed point set has nonempty interior. Our assumptions on the parameters are more general than existing ones. Various limiting examples and comparisons are provided.

math.OC↗
arXiv2014

On the range of the Douglas-Rachford operator

The problem of finding a minimizer of the sum of two convex functions - or, more generally, that of finding a zero of the sum of two maximally monotone operators - is of central importance in variational analysis. Perhaps the most popular method of solving this problem is the Douglas-Rachford splitting method. Surprisingly, little is known about the range of the Douglas-Rachford operator. In this paper, we set out to study this range systematically. We prove that for 3* monotone operators a very pleasing formula can be found that reveals the range to be nearly equal to a simple set involving the domains and ranges of the underlying operators. A similar formula holds for the range of the corresponding displacement mapping. We discuss applications to subdifferential operators, to the infimal displacement vector, and to firmly nonexpansive mappings. Various examples and counter-examples are presented, including some concerning the celebrated Brezis-Haraux theorem.

math.OC↗
arXiv2014

Optimal rates of convergence of matrices with applications

We present a systematic study on the linear convergence rates of the powers of (real or complex) matrices. We derive a characterization when the optimal convergence rate is attained. This characterization is given in terms of semi-simpleness of all eigenvalues having the second-largest modulus after 1. We also provide applications of our general results to analyze the optimal convergence rates for several relaxed alternating projection methods and the generalized Douglas-Rachford splitting methods for finding the projection on the intersection of two subspaces. Numerical experiments confirm our convergence analysis.

math.OC↗
arXiv2014

On subgradient projectors

The subgradient projector is of considerable importance in convex optimization because it plays the key role in Polyak's seminal work - and the many papers it spawned - on subgradient projection algorithms for solving convex feasibility problems. In this paper, we offer a systematic study of the subgradient projector. Fundamental properties such as continuity, nonexpansiveness, and monotonicity are investigated. We also discuss the Yamagishi-Yamada operator. Numerous examples illustrate our results.

math.OC↗
arXiv2014

Linear and strong convergence of algorithms involving averaged nonexpansive operators

We introduce regularity notions for averaged nonexpansive operators. Combined with regularity notions of their fixed point sets, we obtain linear and strong convergence results for quasicyclic, cyclic, and random iterations. New convergence results on the Borwein-Tam method (BTM) and on the cylically anchored Douglas-Rachford algorithm (CADRA) are also presented. Finally, we provide a numerical comparison of BTM, CADRA and the classical method of cyclic projections for solving convex feasibility problems.

math.OC↗
arXiv2013

The rate of linear convergence of the Douglas-Rachford algorithm for subspaces is the cosine of the Friedrichs angle

The Douglas-Rachford splitting algorithm is a classical optimization method that has found many applications. When specialized to two normal cone operators, it yields an algorithm for finding a point in the intersection of two convex sets. This method for solving feasibility problems has attracted a lot of attention due to its good performance even in nonconvex settings. In this paper, we consider the Douglas-Rachford algorithm for finding a point in the intersection of two subspaces. We prove that the method converges strongly to the projection of the starting point onto the intersection. Moreover, if the sum of the two subspaces is closed, then the convergence is linear with the rate being the cosine of the Friedrichs angle between the subspaces. Our results improve upon existing results in three ways: First, we identify the location of the limit and thus reveal the method as a best approximation algorithm; second, we quantify the rate of convergence, and third, we carry out our analysis in general (possibly infinite-dimensional) Hilbert space. We also provide various examples as well as a comparison with the classical method of alternating projections.

math.OC↗
arXiv2013

The method of cyclic intrepid projections: convergence analysis and numerical experiments

The convex feasibility problem asks to find a point in the intersection of a collection of nonempty closed convex sets. This problem is of basic importance in mathematics and the physical sciences, and projection (or splitting) methods solve it by employing the projection operators associated with the individual sets to generate a sequence which converges to a solution. Motivated by an application in road design, we present the method of cyclic intrepid projections (CycIP) and provide a rigorous convergence analysis. We also report on very promising numerical experiments in which CycIP is compared to a commercial state-of-the-art optimization solver.

math.OC↗