arXiv subjects
Publications and source records attributed to Walaa M. Moursi.
In 1964, Michael Edelstein presented an amazing affine isometry acting on the space of square-summable sequences. This operator has no fixed points, but a suborbit that converges to 0 while another escapes in norm to infinity! We revisit, extend and sharpen his construction. Moreover, we sketch a connection to modern optimization and monotone operator theory.
The Douglas-Rachford algorithm (DRA) is a powerful optimization method for minimizing the sum of two convex (not necessarily smooth) functions. The vast majority of previous research dealt with the case when the sum has at least one minimizer. In the absence of minimizers, it was recently shown that for the case of two indicator functions, the DRA converges to a best approximation solution. In this paper, we present a new convergence result on the the DRA applied to the problem of minimizing a convex function subject to a linear constraint. Indeed, a normal solution may be found even when the domain of the objective function and the linear subspace constraint have no point in common. As an important application, a new parallel splitting result is provided. We also illustrate our results through various examples.
We consider the problem of efficiently computing the derivative of the solution map of a convex cone program, when it exists. We do this by implicitly differentiating the residual map for its homogeneous self-dual embedding, and solving the linear systems of equations required using an iterative method. This allows us to efficiently compute the derivative operator, and its adjoint, evaluated at a vector. These correspond to computing an approximate new solution, given a perturbation to the cone program coefficients (i.e., perturbation analysis), and to computing the gradient of a function of the solution with respect to the coefficients. Our method scales to large problems, with numbers of coefficients in the millions. We present an open-source Python implementation of our method that solves a cone program and returns the derivative and its adjoint as abstract linear maps; our implementation can be easily integrated into software systems for automatic differentiation.
Many iterative optimization algorithms involve compositions of special cases of Lipschitz continuous operators, namely firmly nonexpansive, averaged and nonexpansive operators. The structure and properties of the compositions are of particular importance in the proofs of convergence of such algorithms. In this paper, we systematically study the compositions of further special cases of Lipschitz continuous operators. Applications of our results include compositions of scaled conically nonexpansive mappings, as well as the Douglas--Rachford and forward-backward operators, when applied to solve certain structured monotone inclusion and optimization problems. Several examples illustrate and tighten our conclusions.
In his recent Proceedings of the AMS paper "Gossez's skew linear map and its pathological maximally monotone multifunctions", Stephen Simons proved that the closure of the range of the sum of the Gossez operator and a multiple of the duality map is nonconvex whenever the scalar is between 0 and 4. The problem of the convexity of that range when the scalar is equal to 4 was explicitly stated. In this paper, we answer this question in the negative for any scalar greater than or equal to 4. We derive this result from an abstract framework that allows us to also obtain a corresponding result for the Fitzpatrick-Phelps integral operator.
The correspondence between the monotonicity of a (possibly) set-valued operator and the firm nonexpansiveness of its resolvent is a key ingredient in the convergence analysis of many optimization algorithms. Firmly nonexpansive operators form a proper subclass of the more general - but still pleasant from an algorithmic perspective - class of averaged operators. In this paper, we introduce the new notion of conically nonexpansive operators which generalize nonexpansive mappings. We characterize averaged operators as being resolvents of comonotone operators under appropriate scaling. As a consequence, we characterize the proximal point mappings associated with hypoconvex functions as cocoercive operators, or equivalently; as displacement mappings of conically nonexpansive operators. Several examples illustrate our analysis and demonstrate tightness of our results.
Projection methods are popular algorithms for iteratively solving feasibility problems in Euclidean or even Hilbert spaces. They employ (selections of) nearest point mappings to generate sequences that are designed to approximate a point in the intersection of a collection of constraint sets. Theoretical properties of projection methods are fairly well understood when the underlying constraint sets are convex; however, convergence results for the nonconvex case are more complicated and typically only local. In this paper, we explore the perhaps simplest instance of a feasibility algorithm, namely when each constraint set consists of only finitely many points. We numerically investigate four constellations: either few or many constraint sets, with either few or many points. Each constellation is tackled by four popular projection methods each of which features a tuning parameter. We examine the behaviour for a single and for a multitude of orbits, and we also consider local and global behaviour. Our findings demonstrate the importance of the choice of the algorithm and that of the tuning parameter.
Monotone operators and (firmly) nonexpansive mappings are fundamental objects in modern analysis and computational optimization. Five years ago, it was shown that if finitely many firmly nonexpansive mappings have or "almost have" fixed points, then the same is true for compositions and convex combinations. More recently, sharp information about the minimal displacement vector of compositions and of convex combinations of firmly nonexpansive mappings was obtained in terms of the displacement vectors of the underlying operators. Using a new proof technique based on the Brezis-Haraux theorem and reflected resolvents, we extend these results from firmly nonexpansive to general averaged nonexpansive mappings. Various examples illustrate the tightness of our results.
This paper provides a comprehensive discussion of the equivalences between splitting methods. These equivalences have been studied over the past few decades and, in fact, have proven to be very useful. In this paper, we survey known results and also present new ones. In particular, we provide simplified proofs of the equivalence of the ADMM and the Douglas-Rachford method and the equivalence of the ADMM with intermediate update of multipliers and the Peaceman-Rachford method. Other splitting methods are also considered.
Aragón Artacho and Campoy recently proposed a new method for computing the projection onto the intersection of two closed convex sets in Hilbert space; moreover, they proposed in 2018 a generalization from normal cone operators to maximally monotone operators. In this paper, we complete this analysis by demonstrating that the underlying curve converges to the nearest zero of the sum of the two operators. We also provide a new interpretation of the underlying operators in terms of the resolvent and the proximal average.
Monotone operator theory and fixed point theory for nonexpansive mappings are central areas in modern nonlinear analysis and optimization. Although these areas are fairly well developed, almost all examples published are based on subdifferential operators, linear relations, or combinations thereof. In this paper, we construct an intriguing maximally monotone operator induced by a certain nonexpansive retraction. We analyze this operator, which does not appear to be assembled from subdifferential operators or linear relations, in some detail. Particular emphasis is placed on duality and strong monotonicity.
The Douglas-Rachford method is a popular splitting technique for finding a zero of the sum of two subdifferential operators of proper closed convex functions; more generally two maximally monotone operators. Recent results concerned with linear rates of convergence of the method require additional properties of the underlying monotone operators, such as strong monotonicity and cocoercivity. In this paper, we study the case when one operator is Lipschitz continuous but not necessarily a subdifferential operator and the other operator is strongly monotone. This situation arises in optimization methods which involve primal-dual approaches. We provide new linear convergence results in this setting.
Maximally monotone operators and firmly nonexpansive mappings play key roles in modern optimization and nonlinear analysis. Five years ago, it was shown that if finitely many firmly nonexpansive operators are all asymptotically regular (i.e., the have or "almost have" fixed points), then the same is true for compositions and convex combinations. In this paper, we derive bounds on the magnitude of the minimal displacement vectors of compositions and of convex combinations in terms of the displacement vectors of the underlying operators. Our results completely generalize earlier works. Moreover, we present various examples illustrating that our bounds are sharp.
The forward-backward splitting technique is a popular method for solving monotone inclusions that has applications in optimization. In this paper we explore the behaviour of the algorithm when the inclusion problem has no solution. We present a new formula to define the normal solutions using the forward-backward operator. We also provide a formula for the range of the displacement map of the forward-backward operator. Several examples illustrate our theory.
The Douglas-Rachford algorithm is a very popular splitting technique for finding a zero of the sum of two maximally monotone operators. However, the behaviour of the algorithm remains mysterious in the general inconsistent case, i.e., when the sum problem has no zeros. More than a decade ago, however, it was shown that in the (possibly inconsistent) convex feasibility setting, the shadow sequence remains bounded and it is weak cluster points solve a best approximation problem. In this paper, we advance the understanding of the inconsistent case significantly by providing a complete proof of the full weak convergence in the convex feasibility setting. In fact, a more general sufficient condition for the weak convergence in the general case is presented. Several examples illustrate the results.
The Douglas-Rachford splitting algorithm was originally proposed in 1956 to solve a system of linear equations arising from the discretization of a partial differential equation. In 1979, Lions and Mercier brought forward a very powerful extension of this method suitable to solve optimization problems. In this paper, we revisit the original affine setting. We provide a powerful convergence result for finding a zero of the sum of two maximally monotone affine relations. As a by product of our analysis, we obtain results concerning the convergence of iterates of affine nonexpansive mappings as well as Attouch-Théra duality. Numerous examples are presented.
Nearly convex sets play important roles in convex analysis, optimization and theory of monotone operators. We give a systematic study of nearly convex sets, and construct examples of subdifferentials of lower semicontinuous convex functions whose domain or ranges are nonconvex.
The Douglas-Rachford algorithm is a simple yet effective method for solving convex feasibility problems. However, if the underlying constraints are inconsistent, then the convergence theory is incomplete. We provide convergence results when one constraint is an affine subspace. As a consequence, we extend a result by Spingarn from halfspaces to general closed convex sets admitting least-squares solutions.