arXiv subjects

Heinz H. Bauschke

Publications and source records attributed to Heinz H. Bauschke.

At least 55 records · Page 3Linked to original sources

arXiv2021

The splitting algorithms by Ryu and by Malitsky-Tam applied to normal cones of linear subspaces converge strongly to the projection onto the intersection

Finding a zero of a sum of maximally monotone operators is a fundamental problem in modern optimization and nonsmooth analysis. Assuming that resolvents of the operators are available, this problem can be tackled with the Douglas-Rachford algorithm. However, when dealing with three or more operators, one must work in a product space with as many factors as there are operators. In groundbreaking recent work by Ryu and by Malitsky and Tam, it was shown that the number of factors can be reduced by one. These splitting methods guarantee weak convergence to some solution of the underlying sum problem; strong convergence holds in the presence of uniform monotonicity. In this paper, we provide a case study when the operators involved are normal cone operators of subspaces and the solution set is thus the intersection of the subspaces. Even though these operators lack strict convexity, we show that striking conclusions are available in this case: strong (instead of weak) convergence and the solution obtained is (not arbitrary but) the projection onto the intersection. Numerical experiments to illustrate our results are also provided

math.OC↗

arXiv2021

Directional asymptotics of Fejér monotone sequences

The notion of Fejér monotonicity is instrumental in unifying the convergence proofs of many iterative methods, such as the Krasnoselskii-Mann iteration, the proximal point method, the Douglas-Rachford splitting algorithm, and many others. In this paper, we present directionally asymptotical results of strongly convergent subsequences of Fejér monotone sequences. We also provide examples to show that the sets of directionally asymptotic cluster points can be large and that weak convergence is needed in infinite-dimensional spaces.

math.OC↗

arXiv2021

On the Douglas-Rachford algorithm for solving possibly inconsistent optimization problems

More than 40 years ago, Lions and Mercier introduced in a seminal paper the Douglas-Rachford algorithm. Today, this method is well recognized as a classical and highly successful splitting method to find minimizers of the sum of two (not necessarily smooth) convex functions. While the underlying theory has matured, one case remains a mystery: the behaviour of the shadow sequence when the given functions have disjoint domains. Building on previous work, we establish for the first time weak and value convergence of the shadow sequence generated by the Douglas-Rachford algorithm in a setting of unprecedented generality. The weak limit point is shown to solve the associated normal problem which is a minimal perturbation of the original optimization problem. We also present new results on the geometry of the minimal displacement vector.

math.OC↗

arXiv2021

The difference vectors for convex sets and a resolution of the geometry conjecture

The geometry conjecture, which was posed nearly a quarter of a century ago, states that the fixed point set of the composition of projectors onto nonempty closed convex sets in Hilbert space is actually equal to the intersection of certain translations of the underlying sets. In this paper, we provide a complete resolution of the geometry conjecture. Our proof relies on monotone operator theory. We revisit previously known results and provide various illustrative examples. Comments on the numerical computation of the quantities involved are also presented.

math.OC↗

arXiv2021

On angles between convex sets in Hilbert spaces

The notion of the angle between two subspaces has a long history, dating back to Friedrichs's work in 1937 and Dixmier's work on the minimal angle in 1949. In 2006, Deutsch and Hundal studied extensions to convex sets in order to analyze convergence rates for the cyclic projections algorithm. In this work, we characterize the positivity of the minimal angle between two convex cones. We show the existence of, and necessary conditions for, optimal solutions of minimal angle problems associated with two convex subsets as well. Moreover, we generalize a result by Deutsch on minimal angles from linear subspaces to cones. This generalization yields sufficient conditions for the closedness of the sum of two closed convex cones. This also relates to conditions proposed by Beutner and by Seeger and Sossa. Furthermore, we investigate the relation between the intersection of two cones (at least one of which is nonlinear) and the intersection of the polar and dual cones of the underlying cones. It turns out that the two angles involved cannot be positive simultaneously. Various examples illustrate the sharpness of our results.

math.OC↗

arXiv2021

Attouch-Théra Duality, Generalized Cycles and Gap Vectors

Using the Attouch-Théra duality, we study the cycles, gap vectors and fixed point sets of compositions of proximal mappings. Sufficient conditions are given for the existence of cycles and gap vectors. A primal-dual framework provides an exact relationship between the cycles and gap vectors. We also introduce the generalized cycle and gap vectors to tackle the case when the classical ones do not exist. Examples are given to illustrate our results.

math.OC↗

arXiv2021

Resolvents and Yosida approximations of displacement mappings of isometries

Maximally monotone operators are fundamental objects in modern optimization. The main classes of monotone operators are subdifferential operators and matrices with a positive semidefinite symmetric part. In this paper, we study a nice class of monotone operators: displacement mappings of isometries of finite order. We derive explicit formulas for resolvents, Yosida approximations, and (set-valued and MoorePenrose) inverses. We illustrate our results by considering certain rational rotators and circular shift operators.

math.FA↗

arXiv2020

Edelstein's Astonishing Affine Isometry

In 1964, Michael Edelstein presented an amazing affine isometry acting on the space of square-summable sequences. This operator has no fixed points, but a suborbit that converges to 0 while another escapes in norm to infinity! We revisit, extend and sharpen his construction. Moreover, we sketch a connection to modern optimization and monotone operator theory.

math.FA↗

arXiv2020

On the behaviour of the Douglas-Rachford algorithm for minimizing a convex function subject to a linear constraint

The Douglas-Rachford algorithm (DRA) is a powerful optimization method for minimizing the sum of two convex (not necessarily smooth) functions. The vast majority of previous research dealt with the case when the sum has at least one minimizer. In the absence of minimizers, it was recently shown that for the case of two indicator functions, the DRA converges to a best approximation solution. In this paper, we present a new convergence result on the the DRA applied to the problem of minimizing a convex function subject to a linear constraint. Indeed, a normal solution may be found even when the domain of the objective function and the linear subspace constraint have no point in common. As an important application, a new parallel splitting result is provided. We also illustrate our results through various examples.

math.OC↗

arXiv2020

On the linear convergence of circumcentered isometry methods

The circumcentered Douglas--Rachford method (C--DRM), introduced by Behling, Bello Cruz and Santos, is an acceleration of the well-known Douglas-Rachford method (DRM) for finding the best approximation onto the intersection of finitely many affine subspaces. Inspired by the C--DRM, we introduced the more flexible circumcentered reflection method (CRM) and circumcentered isometry method (CIM). The CIM essentially chooses the closest point to the solution among all of the points in an associated affine hull as its iterate and is a generalization of the CRM. The circumcentered--reflection method introduced by Behling et al. to generalize the C--DRM is a special class of our CRM. We consider the CIM induced by a set of finitely many isometries for finding the best approximation onto the intersection of fixed point sets of the isometries which turns out to be an intersection of finitely many affine subspaces. We extend our previous linear convergence results on CRMs in finite-dimensional spaces from reflections to isometries. In order to better accelerate the symmetric method of alternating projections (MAP), the accelerated symmetric MAP first applies another operator to the initial point. (Similarly, to accelerate the DRM, the C--DRM first applies another operator to the initial point as well.) Motivated by these facts, we show results on the linear convergence of CIMs in Hilbert spaces with first applying another operator to the initial point. In particular, under some restrictions, our results imply that some CRMs attain the known linear convergence rate of the accelerated symmetric MAP in Hilbert spaces. We also exhibit a class of CRMs converging to the best approximation in Hilbert spaces with a convergence rate no worse than the sharp convergence rate of MAP. The fact that some CRMs attain the linear convergence rate of MAP or accelerated symmetric MAP is entirely new.

math.OC↗

arXiv2020

Best approximation mappings in Hilbert spaces

The notion of best approximation mapping (BAM) with respect to a closed affine subspace in finite-dimensional space was introduced by Behling, Bello Cruz and Santos to show the linear convergence of the block-wise circumcentered-reflection method. The best approximation mapping possesses two critical properties of the circumcenter mapping for linear convergence. Because the iteration sequence of BAM linearly converges, the BAM is interesting in its own right. In this paper, we naturally extend the definition of BAM from closed affine subspace to nonempty closed convex set and from $\mathbb{R}^{n}$ to general Hilbert space. We discover that the convex set associated with the BAM must be the fixed point set of the BAM. Hence, the iteration sequence generated by a BAM linearly converges to the nearest fixed point of the BAM. Connections between BAMs and other mappings generating convergent iteration sequences are considered. Behling et al.\ proved that the finite composition of BAMs associated with closed affine subspaces is still a BAM in $\mathbb{R}^{n}$. We generalize their result from $\mathbb{R}^{n}$ to general Hilbert space and also construct a new constant associated with the composition of BAMs. This provides a new proof of the linear convergence of the method of alternating projections. Moreover, compositions of BAMs associated with general convex sets are investigated. In addition, we show that convex combinations of BAMs associated with affine subspaces are BAMs. Last but not least, we connect BAM with circumcenter mapping in Hilbert spaces.

math.OC↗

arXiv2020

Fixed points of compositions of nonexpansive mappings: finitely many linear reflectors

Nonexpansive mappings play a central role in modern optimization and monotone operator theory because their fixed points can describe solutions to optimization or critical point problems. It is known that when the mappings are sufficiently "nice", then the fixed point set of the composition coincides with the intersection of the individual fixed point sets. In this paper, we explore the situation for compositions of linear reflectors. We provide positive results, upper bounds, and limiting examples. We also discuss classical reflectors in the Euclidean plane.

math.FA↗

arXiv2020

Circumcentered methods induced by isometries

Motivated by the circumcentered Douglas--Rachford method recently introduced by Behling, Bello Cruz and Santos to accelerate the Douglas--Rachford method, we study the properness of the circumcenter mapping and the circumcenter method induced by isometries. Applying the demiclosedness principle for circumcenter mappings, we present weak convergence results for circumcentered isometry methods, which include the Douglas--Rachford method (DRM) and circumcentered reflection methods as special instances. We provide sufficient conditions for the linear convergence of circumcentered isometry/reflection methods. We explore the convergence rate of circumcentered reflection methods by considering the required number of iterations and as well as run time as our performance measures. Performance profiles on circumcentered reflection methods, DRM and method of alternating projections for finding the best approximation to the intersection of linear subspaces are presented.

math.OC↗

arXiv2020

On Dykstra's algorithm: finite convergence, stalling, and the method of alternating projections

A popular method for finding the projection onto the intersection of two closed convex subsets in Hilbert space is Dykstra's algorithm. In this paper, we provide sufficient conditions for Dykstra's algorithm to converge rapidly, in finitely many steps. We also analyze the behaviour of Dykstra's algorithm applied to a line and a square. This case study reveals stark similarities to the method of alternating projections. Moreover, we show that Dykstra's algorithm may stall for an arbitrarily long time. Finally, we present some open problems.

math.OC↗

arXiv2019

Multi-marginal maximal monotonicity and convex analysis

Monotonicity and convex analysis arise naturally in the framework of multi-marginal optimal transport theory. However, a comprehensive multi-marginal monotonicity and convex analysis theory is still missing. To this end we study extensions of classical monotone operator theory and convex analysis into the multi-marginal setting. We characterize multi-marginal c-monotonicity in terms of classical monotonicity and firmly nonexpansive mappings. We provide Minty type, continuity and conjugacy criteria for multi-marginal maximal monotonicity. We extend the partition of the identity into a sum of firmly nonexpansive mappings and Moreau's decomposition of the quadratic function into envelopes and proximal mappings into the multi-marginal settings. We illustrate our discussion with examples and provide applications for the determination of multi-marginal maximal monotonicity and multi-marginal conjugacy. We also point out several open questions.

math.FA↗

arXiv2019

Maximally monotone operators with ranges whose closures are not convex and an answer to a recent question by Stephen Simons

In his recent Proceedings of the AMS paper "Gossez's skew linear map and its pathological maximally monotone multifunctions", Stephen Simons proved that the closure of the range of the sum of the Gossez operator and a multiple of the duality map is nonconvex whenever the scalar is between 0 and 4. The problem of the convexity of that range when the scalar is equal to 4 was explicitly stated. In this paper, we answer this question in the negative for any scalar greater than or equal to 4. We derive this result from an abstract framework that allows us to also obtain a corresponding result for the Fitzpatrick-Phelps integral operator.

math.FA↗

arXiv2019

Applying FISTA to optimization problems (with or) without minimizers

Beck and Teboulle's FISTA method for finding a minimizer of the sum of two convex functions, one of which has a Lipschitz continuous gradient whereas the other may be nonsmooth, is arguably the most important optimization algorithm of the past decade. While research activity on FISTA has exploded ever since, the mathematically challenging case when the original optimization problem has no minimizer has found only limited attention. In this work, we systematically study FISTA and its variants. We present general results that are applicable, regardless of the existence of minimizers.

math.OC↗

arXiv2019

Generalized monotone operators and their averaged resolvents

The correspondence between the monotonicity of a (possibly) set-valued operator and the firm nonexpansiveness of its resolvent is a key ingredient in the convergence analysis of many optimization algorithms. Firmly nonexpansive operators form a proper subclass of the more general - but still pleasant from an algorithmic perspective - class of averaged operators. In this paper, we introduce the new notion of conically nonexpansive operators which generalize nonexpansive mappings. We characterize averaged operators as being resolvents of comonotone operators under appropriate scaling. As a consequence, we characterize the proximal point mappings associated with hypoconvex functions as cocoercive operators, or equivalently; as displacement mappings of conically nonexpansive operators. Several examples illustrate our analysis and demonstrate tightness of our results.

math.OC↗