arXiv subjects
Publications and source records attributed to Xianfu Wang.
Sets of bilinear constraints are important in various machine learning models. Mathematically, they are hyperbolas in a product space. In this paper, we give a complete formula for projections onto sets of bilinear constraints or hyperbolas in a general Hilbert space.
We extend the Malitsky-Tam forward-reflected-backward (FRB) splitting method for inclusion problems of monotone operators to nonconvex minimization problems. By assuming the generalized concave Kurdyka-Łojasiewicz (KL) property of a quadratic regularization of the objective, we show that the FRB method converges globally to a stationary point of the objective and enjoys finite length property. The sharpness of our approach is guaranteed by virtue of the exact modulus associated with the generalized concave KL property. Numerical experiments suggest that FRB is competitive compared to the Douglas-Rachford method and the Boţ-Csetnek inertial Tseng's method.
In 1990, Romero presented a beautiful formula for the projection onto the set of rectangular matrices with prescribed row and column sums. Variants of Romero's formula have been rediscovered by Khoury and by Glunt, Hayden, and Reams, for bistochastic (square) matrices in 1998. These results have found various generalizations and applications. In this paper, we provide a formula for the more general problem of finding the projection onto the set of rectangular matrices with prescribed scaled row and column sums. Our approach is based on computing the Moore-Penrose inverse of a certain linear operator associated with the problem. In fact, our analysis holds even for Hilbert-Schmidt operators and we do not have to assume consistency. We also perform numerical experiments featuring the new projection operator.
The problem of finding a best approximation pair of two sets, which in turn generalizes the well known convex feasibility problem, has a long history that dates back to work by Cheney and Goldstein in 1959. In 2018, Aharoni, Censor, and Jiang revisited this problem and proposed an algorithm that can be used when the two sets are finite intersections of halfspaces. Motivated by their work, we present alternative algorithms that utilize projection and proximity operators. Numerical experiments indicate that these methods are competitive and sometimes superior to the one proposed by Aharoni et al.
We introduce a generalized version of the concave Kurdyka-Łojasiewicz (KL) property by employing nonsmooth desingularizing functions. We also present the exact modulus of the generalized concave KL property, which provides an answer to the open question regarding the optimal concave desingularizing function. The exact modulus is designed to be the smallest among all possible concave desingularizing functions. Examples are given to illustrate this pleasant property. In turn, using the exact modulus we provide the sharpest upper bound for the total length of iterates generated by the celebrated Bolte-Sabach-Teboulle PALM algorithm.
In this paper, we propose several calculus rules for the generalized concave Kurdyka-Łojasiewicz (KL) property, which generalize Li and Pong's results for KL exponents. The optimal concave desingularizing function has various forms and may be nondifferentiable. Our calculus rules do not assume desingularizing functions to have any specific form nor differentiable, while the known results do. Several examples are also given to show that our calculus rules are applicable to a broader class of functions than the known ones.
There are two basic angles associated with a pair of linear subspaces: the Diximier angle and the Friedrichs angle. The Dixmier angle of the pair of orthogonal complements is the same as the Dixmier angle of the original pair provided that the original pair gives rise to a direct (not necessarily orthogonal) sum of the underlying Hilbert space. The Friedrichs angles of the original pair and the pair of the orthogonal complements always coincide. These two results are due to Krein, Krasnoselskii, and Milman and to Solmon, respectively. In 1995, Deutsch provided a very nice survey with complete proofs and interesting historical comments. One key result in Deutsch's survey was an inequality for Dixmier angles provided by Hundal. In this paper, we present extensions of these results to the case when the linear subspaces are only required to be convex cones. It turns out that Hundal's result has a nice conical extension while the situation is more technical for the results by Krein et al.\ and by Solmon. Our analysis is based on Deutsch's survey and our recent work on angles between convex sets. Throughout, we also provide examples illustrating the sharpness of our results.
Finding a zero of a sum of maximally monotone operators is a fundamental problem in modern optimization and nonsmooth analysis. Assuming that resolvents of the operators are available, this problem can be tackled with the Douglas-Rachford algorithm. However, when dealing with three or more operators, one must work in a product space with as many factors as there are operators. In groundbreaking recent work by Ryu and by Malitsky and Tam, it was shown that the number of factors can be reduced by one. These splitting methods guarantee weak convergence to some solution of the underlying sum problem; strong convergence holds in the presence of uniform monotonicity. In this paper, we provide a case study when the operators involved are normal cone operators of subspaces and the solution set is thus the intersection of the subspaces. Even though these operators lack strict convexity, we show that striking conclusions are available in this case: strong (instead of weak) convergence and the solution obtained is (not arbitrary but) the projection onto the intersection. Numerical experiments to illustrate our results are also provided
The notion of Fejér monotonicity is instrumental in unifying the convergence proofs of many iterative methods, such as the Krasnoselskii-Mann iteration, the proximal point method, the Douglas-Rachford splitting algorithm, and many others. In this paper, we present directionally asymptotical results of strongly convergent subsequences of Fejér monotone sequences. We also provide examples to show that the sets of directionally asymptotic cluster points can be large and that weak convergence is needed in infinite-dimensional spaces.
The geometry conjecture, which was posed nearly a quarter of a century ago, states that the fixed point set of the composition of projectors onto nonempty closed convex sets in Hilbert space is actually equal to the intersection of certain translations of the underlying sets. In this paper, we provide a complete resolution of the geometry conjecture. Our proof relies on monotone operator theory. We revisit previously known results and provide various illustrative examples. Comments on the numerical computation of the quantities involved are also presented.
The notion of the angle between two subspaces has a long history, dating back to Friedrichs's work in 1937 and Dixmier's work on the minimal angle in 1949. In 2006, Deutsch and Hundal studied extensions to convex sets in order to analyze convergence rates for the cyclic projections algorithm. In this work, we characterize the positivity of the minimal angle between two convex cones. We show the existence of, and necessary conditions for, optimal solutions of minimal angle problems associated with two convex subsets as well. Moreover, we generalize a result by Deutsch on minimal angles from linear subspaces to cones. This generalization yields sufficient conditions for the closedness of the sum of two closed convex cones. This also relates to conditions proposed by Beutner and by Seeger and Sossa. Furthermore, we investigate the relation between the intersection of two cones (at least one of which is nonlinear) and the intersection of the polar and dual cones of the underlying cones. It turns out that the two angles involved cannot be positive simultaneously. Various examples illustrate the sharpness of our results.
Using the Attouch-Théra duality, we study the cycles, gap vectors and fixed point sets of compositions of proximal mappings. Sufficient conditions are given for the existence of cycles and gap vectors. A primal-dual framework provides an exact relationship between the cycles and gap vectors. We also introduce the generalized cycle and gap vectors to tackle the case when the classical ones do not exist. Examples are given to illustrate our results.
Circumcenters play an important role in the design and analysis of accelerating various iterative methods in optimization. In this work, we propose Bregman (pseudo-)circumcenters associated with finite sets. We show the existence and give explicit formulae for the unique backward and forward Bregman pseudo-circumcenters of finite sets. Moreover, we use duality to establish connections between backward and forward Bregman (pseudo-)circumcenters. Various examples are presented to illustrate the backward and forward Bregman (pseudo-)circumcenters of finite sets. Our general framework for circumcenters paves the way for the development of accelerating iterative methods by Bregman circumcenters.
Maximally monotone operators are fundamental objects in modern optimization. The main classes of monotone operators are subdifferential operators and matrices with a positive semidefinite symmetric part. In this paper, we study a nice class of monotone operators: displacement mappings of isometries of finite order. We derive explicit formulas for resolvents, Yosida approximations, and (set-valued and MoorePenrose) inverses. We illustrate our results by considering certain rational rotators and circular shift operators.
The circumcentered Douglas--Rachford method (C--DRM), introduced by Behling, Bello Cruz and Santos, is an acceleration of the well-known Douglas-Rachford method (DRM) for finding the best approximation onto the intersection of finitely many affine subspaces. Inspired by the C--DRM, we introduced the more flexible circumcentered reflection method (CRM) and circumcentered isometry method (CIM). The CIM essentially chooses the closest point to the solution among all of the points in an associated affine hull as its iterate and is a generalization of the CRM. The circumcentered--reflection method introduced by Behling et al. to generalize the C--DRM is a special class of our CRM. We consider the CIM induced by a set of finitely many isometries for finding the best approximation onto the intersection of fixed point sets of the isometries which turns out to be an intersection of finitely many affine subspaces. We extend our previous linear convergence results on CRMs in finite-dimensional spaces from reflections to isometries. In order to better accelerate the symmetric method of alternating projections (MAP), the accelerated symmetric MAP first applies another operator to the initial point. (Similarly, to accelerate the DRM, the C--DRM first applies another operator to the initial point as well.) Motivated by these facts, we show results on the linear convergence of CIMs in Hilbert spaces with first applying another operator to the initial point. In particular, under some restrictions, our results imply that some CRMs attain the known linear convergence rate of the accelerated symmetric MAP in Hilbert spaces. We also exhibit a class of CRMs converging to the best approximation in Hilbert spaces with a convergence rate no worse than the sharp convergence rate of MAP. The fact that some CRMs attain the linear convergence rate of MAP or accelerated symmetric MAP is entirely new.
The notion of best approximation mapping (BAM) with respect to a closed affine subspace in finite-dimensional space was introduced by Behling, Bello Cruz and Santos to show the linear convergence of the block-wise circumcentered-reflection method. The best approximation mapping possesses two critical properties of the circumcenter mapping for linear convergence. Because the iteration sequence of BAM linearly converges, the BAM is interesting in its own right. In this paper, we naturally extend the definition of BAM from closed affine subspace to nonempty closed convex set and from $\mathbb{R}^{n}$ to general Hilbert space. We discover that the convex set associated with the BAM must be the fixed point set of the BAM. Hence, the iteration sequence generated by a BAM linearly converges to the nearest fixed point of the BAM. Connections between BAMs and other mappings generating convergent iteration sequences are considered. Behling et al.\ proved that the finite composition of BAMs associated with closed affine subspaces is still a BAM in $\mathbb{R}^{n}$. We generalize their result from $\mathbb{R}^{n}$ to general Hilbert space and also construct a new constant associated with the composition of BAMs. This provides a new proof of the linear convergence of the method of alternating projections. Moreover, compositions of BAMs associated with general convex sets are investigated. In addition, we show that convex combinations of BAMs associated with affine subspaces are BAMs. Last but not least, we connect BAM with circumcenter mapping in Hilbert spaces.
Nonexpansive mappings play a central role in modern optimization and monotone operator theory because their fixed points can describe solutions to optimization or critical point problems. It is known that when the mappings are sufficiently "nice", then the fixed point set of the composition coincides with the intersection of the individual fixed point sets. In this paper, we explore the situation for compositions of linear reflectors. We provide positive results, upper bounds, and limiting examples. We also discuss classical reflectors in the Euclidean plane.
Motivated by the circumcentered Douglas--Rachford method recently introduced by Behling, Bello Cruz and Santos to accelerate the Douglas--Rachford method, we study the properness of the circumcenter mapping and the circumcenter method induced by isometries. Applying the demiclosedness principle for circumcenter mappings, we present weak convergence results for circumcentered isometry methods, which include the Douglas--Rachford method (DRM) and circumcentered reflection methods as special instances. We provide sufficient conditions for the linear convergence of circumcentered isometry/reflection methods. We explore the convergence rate of circumcentered reflection methods by considering the required number of iterations and as well as run time as our performance measures. Performance profiles on circumcentered reflection methods, DRM and method of alternating projections for finding the best approximation to the intersection of linear subspaces are presented.