Searcharxiv⌕ Search

arXiv subjects

Heinz H. Bauschke

Publications and source records attributed to Heinz H. Bauschke.

At least 37 records · Page 2Linked to original sources
arXiv2025

On the boundedness of the sequence generated by minibatch stochastic gradient descent

Stochastic Gradient Descent (SGD) with Polyak's stepsize has recently gained renewed attention in stochastic optimization. Recently, Orvieto, Lacoste-Julien, and Loizou introduced a decreasing variant of Polyak's stepsize, where convergence relies on a boundedness assumption of the iterates. They established that this assumption holds under strong convexity. In this paper, we extend their result by proving that boundedness also holds for a broader class of objective functions, including coercive functions. We also present a case in which boundedness may or may not hold.

math.OC↗
arXiv2025

On the Bredies-Chenchene-Lorenz-Naldi algorithm

Monotone inclusion problems occur in many areas of optimization and variational analysis. Splitting methods, which utilize resolvents or proximal mappings of the underlying operators, are often applied to solve these problems. In 2022, Bredies, Chenchene, Lorenz, and Naldi introduced a new elegant algorithmic framework that encompasses various well known algorithms including Douglas-Rachford and Chambolle-Pock. They obtained powerful weak and strong convergence results, where the latter type relies on additional strong monotonicity assumptions. In this paper, we complement the analysis by Bredies et al. by relating the projections of the fixed point sets of the underlying operators that generate the (reduced and original) preconditioned proximal point sequences. We also obtain strong convergence results in the case of linear relations. Various examples are provided to illustrate the applicability of our results.

math.OC↗
arXiv2024

Projecting onto rectangular hyperbolic paraboloids in Hilbert space

In $\mathbb{R}^3$, a hyperbolic paraboloid is a classical saddle-shaped quadric surface. Recently, Elser has modeled problems arising in Deep Learning using rectangular hyperbolic paraboloids in $\mathbb{R}^n$. Motivated by his work, we provide a rigorous analysis of the associated projection. In some cases, finding this projection amounts to finding a certain root of a quintic or cubic polynomial. We also observe when the projection is not a singleton and point out connections to graphical and set convergence.

math.OC↗
arXiv2024

On a result by Baillon, Bruck, and Reich

It is well known that the iterates of an averaged nonexpansive mapping may only converge weakly to fixed point. A celebrated result by Baillon, Bruck, and Reich from 1978 yields strong convergence in the presence of linearity. In this paper, we extend this result to allow for flexible relaxation parameters. Examples are also provided to illustrate the results.

math.FA↗
arXiv2023

How averaged is the composition of two linear projections?

Projection operators are fundamental algorithmic operators in Analysis and Optimization. It is well known that these operators are firmly nonexpansive; however, their composition is generally only averaged and no longer firmly nonexpansive. In this note, we introduce the modulus of averagedness and provide an exact result for the composition of two linear projection operators. As a consequence, we deduce that the Ogura-Yamada bound for the modulus of the composition is sharp.

math.OC↗
arXiv2023

Real roots of real cubics and optimization

The solution of the cubic equation has a century-long history; however, the usual presentation is geared towards applications in algebra and is somewhat inconvenient to use in optimization where frequently the main interest lies in real roots. In this note, we present the roots of the cubic in a form that makes them convenient to use and we also focus on information on the location of the real roots. Armed with this, we provide several applications in optimization where we compute Fenchel conjugates, proximal mappings and projections.

math.OC↗
arXiv2022

The splitting algorithms by Ryu, by Malitsky-Tam, and by Campoy applied to normal cones of linear subspaces converge strongly to the projection onto the intersection

Finding a zero of a sum of maximally monotone operators is a fundamental problem in modern optimization and nonsmooth analysis. Assuming that the resolvents of the operators are available, this problem can be tackled with the Douglas-Rachford algorithm. However, when dealing with three or more operators, one must work in a product space with as many factors as there are operators. In groundbreaking recent work by Ryu and by Malitsky and Tam, it was shown that the number of factors can be reduced by one. A similar reduction was achieved recently by Campoy through a clever reformulation originally proposed by Kruger. All three splitting methods guarantee weak convergence to some solution of the underlying sum problem; strong convergence holds in the presence of uniform monotonicity. In this paper, we provide a case study when the operators involved are normal cone operators of subspaces and the solution set is thus the intersection of the subspaces. Even though these operators lack strict convexity, we show that striking conclusions are available in this case: strong (instead of weak) convergence and the solution obtained is (not arbitrary but) the projection onto the intersection. Numerical experiments to illustrate our results are also provided.

math.OC↗
arXiv2022

On Carlier's inequality

The Fenchel-Young inequality is fundamental in Convex Analysis and Optimization. It states that the difference between certain function values of two vectors and their inner product is nonnegative. Recently, Carlier introduced a very nice sharpening of this inequality, providing a lower bound that depends on a positive parameter. In this note, we expand on Carlier's inequality in three ways. First, a duality statement is provided. Secondly, we discuss asymptotic behaviour as the underlying parameter approaches zero or infinity. Thirdly, relying on cyclic monotonicity and associated Fitzpatrick functions, we present a lower bound that features an infinite series of squares of norms. Several examples illustrate our results.

math.OC↗
arXiv2022

How to project onto the intersection of a closed affine subspace and a hyperplane

Let $A$ be a closed affine subspace and let $B$ be a hyperplane in a Hilbert space. Suppose we are given their associated nearest point mappings $P_A$ and $P_B$, respectively. We present a formula for the projection onto their intersection $A\cap B$. As a special case, we derive a formula for the projection onto the intersection of two hyperplanes. These formulas provides useful information even if $A\cap B$ is empty. Examples and numerical experiments are also provided.

math.OC↗
arXiv2022

The Homogenization Cone: Polar Cone and Projection

Let $C$ be a closed convex subset of a real Hilbert space containing the origin, and assume that $K$ is the homogenization cone of $C$, i.e., the smallest closed convex cone containing $C \times \{1\}$. Homogenization cones play an important role in optimization as they include, for instance, the second-order/Lorentz/"ice cream" cone. In this note, we discuss the polar cone of $K$ as well as an algorithm for finding the projection onto $K$ provided that the projection onto $C$ is available. Various examples illustrate our results.

math.OC↗
arXiv2022

The Bregman proximal average

We provide a proximal average with repect to a $1$-coercive Legendre function. In the sense of Bregman distance, the Bregman envelope of the proximal average is a convex combination of Bregman envelopes of individual functions. The Bregman proximal mapping of the average is a convex combination of convexified proximal mappings of individual functions. Techniques from variational analysis provide the keys for the Bregman proximal average.

math.OC↗
arXiv2022

Roots of the identity operator and proximal mappings: (classical and phantom) cycles and gap vectors

Recently, Simons provided a lemma for a support function of a closed convex set in a general Hilbert space and used it to prove the geometry conjecture on cycles of projections. In this paper, we extend Simons's lemma to closed convex functions, show its connections to Attouch-Thera duality, and use it to characterize (classical and phantom) cycles and gap vectors of proximal mappings.

math.FA↗
arXiv2022

The projection onto the cross

We consider the set of pairs of orthogonal vectors in Hilbert space, which is also called the cross because it is the union of the horizontal and vertical axes in the Euclidean plane when the underlying space is the real line. Crosses, which are nonconvex sets, play a significant role in various branches of nonsmooth analysis such as feasibility problems and optimization problems. In this work, we study crosses and show that in infinite-dimensional settings, they are never weakly (sequentially) closed. Nonetheless, crosses do turn out to be proximinal (i.e., they always admit projections) and we provide explicit formulas for the projection onto the cross in all cases.

math.OC↗
arXiv2021

Projecting onto rectangular matrices with prescribed row and column sums

In 1990, Romero presented a beautiful formula for the projection onto the set of rectangular matrices with prescribed row and column sums. Variants of Romero's formula have been rediscovered by Khoury and by Glunt, Hayden, and Reams, for bistochastic (square) matrices in 1998. These results have found various generalizations and applications. In this paper, we provide a formula for the more general problem of finding the projection onto the set of rectangular matrices with prescribed scaled row and column sums. Our approach is based on computing the Moore-Penrose inverse of a certain linear operator associated with the problem. In fact, our analysis holds even for Hilbert-Schmidt operators and we do not have to assume consistency. We also perform numerical experiments featuring the new projection operator.

math.OC↗
arXiv2021

Finding best approximation pairs for two intersections of closed convex sets

The problem of finding a best approximation pair of two sets, which in turn generalizes the well known convex feasibility problem, has a long history that dates back to work by Cheney and Goldstein in 1959. In 2018, Aharoni, Censor, and Jiang revisited this problem and proposed an algorithm that can be used when the two sets are finite intersections of halfspaces. Motivated by their work, we present alternative algorithms that utilize projection and proximity operators. Numerical experiments indicate that these methods are competitive and sometimes superior to the one proposed by Aharoni et al.

math.OC↗
arXiv2021

On angles between convex cones

There are two basic angles associated with a pair of linear subspaces: the Diximier angle and the Friedrichs angle. The Dixmier angle of the pair of orthogonal complements is the same as the Dixmier angle of the original pair provided that the original pair gives rise to a direct (not necessarily orthogonal) sum of the underlying Hilbert space. The Friedrichs angles of the original pair and the pair of the orthogonal complements always coincide. These two results are due to Krein, Krasnoselskii, and Milman and to Solmon, respectively. In 1995, Deutsch provided a very nice survey with complete proofs and interesting historical comments. One key result in Deutsch's survey was an inequality for Dixmier angles provided by Hundal. In this paper, we present extensions of these results to the case when the linear subspaces are only required to be convex cones. It turns out that Hundal's result has a nice conical extension while the situation is more technical for the results by Krein et al.\ and by Solmon. Our analysis is based on Deutsch's survey and our recent work on angles between convex sets. Throughout, we also provide examples illustrating the sharpness of our results.

math.FA↗