SearcharxivSearch

arXiv subjects

Luiz-Rafael Santos

Publications and source records attributed to Luiz-Rafael Santos.

At least 19 recordsLinked to original sources

New operator designs for Halpern iterations with explicit rates under H\"older error bounds

We investigate the asymptotic behavior of Halpern-type iterations applied to quasi-nonexpansive operators arising in best approximation problems over the intersection of finitely many closed convex sets in $\mathbb{R}^n$. Assuming a local decrease condition for the underlying operator and standard requirements on the stepsizes $(\alpha_k) \subset (0,1)$, we first prove strong convergence of the Halpern sequence $x_{k+1} = \alpha_k x_0 + (1-\alpha_k) T x_k$ to the best approximation point $x^\star$ in the intersection set, that is, the metric projection of $x_0$ onto that set. Under the additional assumption that the intersection satisfies a H\"older-type error bound with exponent $\gamma \in (0,1]$, we then derive explicit convergence rates for both feasibility and norm error: the distance from $x_k$ to the intersection set decays like $\mathcal O(\alpha_k^{\gamma/(2-\gamma)})$, while the norm error $\|x_k - x^\star\|$ decays like $\mathcal O(\alpha_k^{\gamma/(4-2\gamma)})$. These results apply to most projection-type operators used in convex feasibility problems (including MAP, CRM/SCCRM, Cimmino and 3PM/A3PM) and extend classical convergence analyses of the Halpern-type iterations by providing explicit, geometry-dependent rates governed by H\"older-type error bounds. Our numerical experiments show that Halpern-type iterations combined with most of these projection-type operators are quicker than Dykstra's algorithm to find the projection of a point in an intersection of ellipsoids or in an intersection of polyhedrons.

math.OC

Basis pursuit by inconsistent alternating projections

Basis pursuit is the problem of finding a vector with smallest $\ell_1$-norm among the solutions of a given linear system of equations. It is a well-known convex relaxation of the sparse affine feasibility problem, where sparse solutions to underdetermined systems are sought. Since basis pursuit admits a linear programming reformulation, standard LP solvers are directly applicable. We instead address the basis pursuit directly in its $\ell_1$-minimization form, without LP reformulation, via a scheme that uses alternating projections in its subproblems. These subproblems are designed to be inconsistent in the sense that they relate to two non-intersecting sets. Recently in [R. Behling, Y. Bello-Cruz and L.-R. Santos, SIAM J. Optim., 31 (2021), pp. 2863-2892], inconsistency coming from infeasibility has been shown to accelerate convergence of alternating projections. We deliberately enforce this inconsistency by constructing subproblems whose feasible sets are disjoint by design. We prove that the resulting $\ell_1$-radii converge linearly to the optimal value, and that when the solution is unique, all generated sequences converge linearly to it at a rate governed by a natural error bound between the feasible set and the optimal $\ell_1$-ball. The proposed method is numerically competitive against state-of-the-art open-source solvers on synthetic and real-world instances.

math.OC

On circumcentered direct methods for monotone variational inequality problems

Circumcentered techniques have been shown to significantly accelerate projection-based methods for convex feasibility problems. Motivated by this success, we propose two direct methods with circumcenter acceleration for solving variational inequality problems involving two classes of operators: paramonotone and monotone. Both schemes rely on approximate projections onto separating halfspaces, thereby avoiding computationally expensive exact projections. We establish convergence results for both methods and conduct numerical experiments, demonstrating that the proposed algorithms outperform classical methods, such as the extragradient algorithm, by orders of magnitude in terms of computational time, particularly when the feasible set is a complex intersection of convex sets.

math.OC

Parallelizing the Circumcentered-Reflection Method

This paper introduces the Parallelized Circumcentered Reflection Method (P-CRM), a circumcentric approach that parallelizes the Circumcentered Reflection Method (CRM) for solving Convex Feasibility Problems in affine settings. Beyond feasibility, P-CRM solves the best approximation problem for any finite collection of affine subspaces; that is, it not only finds a feasible point but directly computes the projection of an initial point onto the intersection. Within a fully self-contained scheme, we also introduce the Framework for the Simultaneous Projection Method (F-SPM) which includes Cimmino's method as a special case. Theoretical results show that both P-CRM and F-SPM achieve linear convergence. Moreover, P-CRM converges at a rate that is at least as fast as, and potentially superior to, the best convergence rate of F-SPM. As a byproduct, this also yields a new and simplified convergence proof for Cimmino's method. Numerical experiments show that P-CRM is competitive compared to CRM and indicate that it offers a scalable and flexible alternative, particularly suited for large-scale problems and modern computing environments.

math.OC

Constructing Magic Squares: an integer constraint satisfaction problem and a fast approach

Magic squares are a fascinating mathematical challenge that has intrigued mathematicians for centuries. Given a positive (and possibly large) integer \( n \), one of the main challenges that still remains is to find, within a computational time, a magic square of order \( n \), that is, a square matrix of order \( n \) with unique integers from \( a_{\min} \) to \( a_{\max} \), such that the sum of each row, column, and diagonal equals a constant \( \mathcal{C}(A) \). In this work, we first present an integer constraint satisfaction problem for constructing a magic square of order \( n \). Nonetheless, the solution time of this problem grows exponentially as the order increases. To overcome this limitation, we also propose a that constructs magic squares depending on whether \( n \) is odd, singly even, or doubly even. Moreover, we provide a proof of the correctness of this novel approach. Our numerical results show that our method can construct magic squares of order up to \num{70000} in less than \num{140} seconds, demonstrating its efficiency and scalability.

math.OC

Fej\'er* monotonicity in optimization algorithms

Fej\'er monotonicity is a well-established property often observed in sequences generated by optimization algorithms. In this paper, we study an extension of this property, called Fej\'er* monotonicity, which was initially proposed in [SIAM J. Optim., 34(3), 2535-2556 (2024)]. We discuss and explore its behavior within Hilbert spaces as a tool for optimization algorithms. Additionally, we investigate weak and strong convergence properties of this novel concept. Through illustrative examples and insightful results, we contrast Fej\'er* with weaker notions of quasi-Fej\'er-type monotonicity.

math.OC

Randomized Nystr\"om Preconditioned Interior Point-Proximal Method of Multipliers

We present a new algorithm for convex separable quadratic programming (QP) called Nys-IP-PMM, a regularized interior-point solver that uses low-rank structure to accelerate solution of the Newton system. The algorithm combines the interior point proximal method of multipliers (IP-PMM) with the randomized Nystr\"om preconditioned conjugate gradient method as the inner linear system solver. Our algorithm is matrix-free: it accesses the input matrices solely through matrix-vector products, as opposed to methods involving matrix factorization. It works particularly well for separable QP instances with dense constraint matrices. We establish convergence of Nys-IP-PMM. Numerical experiments demonstrate its superior performance in terms of wallclock time compared to previous matrix-free IPM-based approaches.

math.OC

A finitely convergent circumcenter method for the Convex Feasibility Problem

In this paper, we present a variant of the circumcenter method for the Convex Feasibility Problem (CFP), ensuring finite convergence under a Slater assumption. The method replaces exact projections onto the convex sets with projections onto separating halfspaces, perturbed by positive exogenous parameters that decrease to zero along the iterations. If the perturbation parameters decrease slowly enough, such as the terms of a diverging series, finite convergence is achieved. To the best of our knowledge, this is the first circumcenter method for CFP that guarantees finite convergence.

math.OC

A successive centralized circumcenter reflection method for the convex feasibility problem

In this paper we present the successive centralization of the circumcenter reflection scheme with several control sequences for solving the convex feasibility problem in Euclidean space. Assuming that a standard error bound holds, we prove the linear convergence of the method with the most violated constraint control sequence. Under additional smoothness assumptions, we prove the superlinear convergence. Numerical experiments confirm the efficiency of our method.

math.OC

A circumcentered-reflection method for finding common fixed points of firmly nonexpansive operators

The circumcentered-reflection method (CRM) has been recently proposed as a methodology for accelerating several algorithms for solving the Convex Feasibility Problem (CFP), equivalent to finding a common fixed-point of the orthogonal projections onto a finite number of closed and convex sets. In this paper, we apply CRM to the more general Fixed Point Problem (denoted as FPP), consisting of finding a common fixed-point of operators belonging to a larger family of operators, namely firmly nonexpansive operators. We prove than in this setting, CRM is globally convergent to a common fixed-point (supposing at least one exists). We also establish linear convergence of the sequence generated by CRM applied to FPP, under a not too demanding error bound assumption, and provide an estimate of the asymptotic constant. We provide solid numerical evidence of the superiority of CRM when compared to the classical Parallel Projections Method (PPM). Additionally, we present certain results of convex combination of orthogonal projections, of some interest on its own.

math.OC

Circumcentric directions of cones

Generalized circumcenters have been recently introduced and employed to speed up classical projection-type methods for solving feasibility problems. In this note, circumcenters are enforced in a new setting; they are proven to provide inward directions to sets given by convex inequalities. In particular, we show that circumcentric directions of finitely generated cones belong to the interior of their polars. We also derive a measure of interiorness of the circumcentric direction, which then provides a special cone of search directions, all being feasible to the convex region under consideration.

math.OC

First-order methods for the convex hull membership problem

The convex hull membership problem (CHMP) consists in deciding whether a certain point belongs to the convex hull of a finite set of points, a decision problem with important applications in computational geometry and in foundations of linear programming. In this study, we review, compare and analyze first-order methods for CHMP, namely, Frank-Wolfe type methods, Projected Gradient methods and a recently introduced geometric algorithm, called Triangle Algorithm (TA). We discuss the connections between this algorithm and Frank-Wolfe, showing that TA can be interpreted as an inexact Frank-Wolfe. Despite this similarity, TA is strongly based on a theorem of alternatives known as distance duality. By using this theorem, we propose suitable stopping criteria for CHMP to be integrated into Frank-Wolfe type and Projected Gradient, specializing these methods to the membership decision problem. Interestingly, Frank-Wolfe integrated with such stopping criteria coincides with a greedy version of the Triangle Algorithm which is, in its turn, equivalent to an algorithm due to von Neumann. We report numerical experiments on random instances of CHMP, carefully designed to cover different scenarios, that indicate which algorithm is preferable according to the geometry of the convex hull and the relative position of the query point. Concerning potential applications, we present two illustrative examples, one related to linear programming feasibility problems and another related to image classification problems.

math.OC

On the centralization of the circumcentered-reflection method

This paper is devoted to deriving the first circumcenter iteration scheme that does not employ a product space reformulation for finding a point in the intersection of two closed convex sets. We introduce a so-called centralized version of the circumcentered-reflection method (CRM). Developed with the aim of accelerating classical projection algorithms, CRM is successful for tracking a common point of a finite number of affine sets. In the case of general convex sets, CRM was shown to possibly diverge if Pierra's product space reformulation is not used. In this work, we prove that there exists an easily reachable region consisting of what we refer to as centralized points, where pure circumcenter steps possess properties yielding convergence. The resulting algorithm is called centralized CRM (cCRM). In addition to having global convergence, cCRM converges linearly under an error bound condition, and superlinearly if the two target sets are so that their intersection have nonempty interior and their boundaries are locally differentiable manifolds. We also run numerical experiments with successful results.

math.OC

Circumcentering approximate reflections for solving the convex feasibility problem

The circumcentered-reflection method (CRM) has been applied for solving convex feasibility problems. CRM iterates by computing a circumcenter upon a composition of reflections with respect to convex sets. Since reflections are based on exact projections, their computation might be costly. In this regard, we introduce the circumcentered approximate-reflection method (CARM), whose reflections rely on outer-approximate projections. The appeal of CARM is that, in rather general situations, the approximate projections we employ are available under low computational cost. We derive convergence of CARM and linear convergence under an error bound condition. We also present successful theoretical and numerical comparisons of CARM to the original CRM, to the classical method of alternating projections (MAP) and to a correspondent outer-approximate version of MAP, referred to as MAAP. Along with our results and numerical experiments, we present a couple of illustrative examples.

math.OC

The Block-wise Circumcentered-Reflection Method

The elementary Euclidean concept of circumcenter has recently been employed to improve two aspects of the classical Douglas--Rachford method for projecting onto the intersection of affine subspaces. The so-called circumcentered-reflection method is able to both accelerate the average reflection scheme by the Douglas--Rachford method and cope with the intersection of more than two affine subspaces. We now introduce the technique of circumcentering in blocks, which, more than just an option over the basic algorithm of circumcenters, turns out to be an elegant manner of generalizing the method of alternating projections. Linear convergence for this novel block-wise circumcenter framework is derived and illustrated numerically. Furthermore, we prove that the original circumcentered-reflection method essentially finds the best approximation solution in one single step if the given affine subspaces are hyperplanes.

math.OC

Infeasibility and error bound imply finite convergence of alternating projections

This paper combines two ingredients in order to get a rather surprising result on one of the most studied, elegant and powerful tools for solving convex feasibility problems, the method of alternating projections (MAP). Going back to names such as Kaczmarz and von Neumann, MAP has the ability to track a pair of points realizing minimum distance between two given closed convex sets. Unfortunately, MAP may suffer from arbitrarily slow convergence, and sublinear rates are essentially only surpassed in the presence of some Lipschitzian error bound, which is our first ingredient. The second one is a seemingly unfavorable and unexpected condition, namely, infeasibility. For two non-intersecting closed convex sets satisfying an error bound, we establish finite convergence of MAP. In particular, MAP converges in finitely many steps when applied to a polyhedron and a hyperplane in the case in which they have empty intersection. Moreover, the farther the target sets lie from each other, the fewer are the iterations needed by MAP for finding a best approximation pair. Insightful examples and further theoretical and algorithmic discussions accompany our results, including the investigation of finite termination of other projection methods.

math.OC

The circumcentered-reflection method achieves better rates than alternating projections

We study the convergence rate of the Circumcentered-Reflection Method (CRM) for solving the convex feasibility problem and compare it with the Method of Alternating Projections (MAP). Under an error bound assumption, we prove that both methods converge linearly, with asymptotic constants depending on a parameter of the error bound, and that the one derived for CRM is strictly better than the one for MAP. Next, we analyze two classes of fairly generic examples. In the first one, the angle between the convex sets approaches zero near the intersection, so that the MAP sequence converges sublinearly, but CRM still enjoys linear convergence. In the second class of examples, the angle between the sets does not vanish and MAP exhibits its standard behavior, i.e., it converges linearly, yet, perhaps surprisingly, CRM attains superlinear convergence.

math.OC

On the Circumcentered-Reflection Method for the Convex Feasibility Problem

The ancient concept of circumcenter has recently given birth to the Circumcentered-Reflection method (CRM). CRM was first employed to solve best approximation problems involving affine subspaces. In this setting, it was shown to outperform the most prestigious projection based schemes, namely, the Douglas-Rachford method (DRM) and the method of alternating projections (MAP). We now prove convergence of CRM for finding a point in the intersection of a finite number of closed convex sets. This striking result is derived under a suitable product space reformulation in which a point in the intersection of a closed convex set with an affine subspace is sought. It turns out that CRM skillfully tackles the reformulated problem. We also show that for a point in the affine set the CRM iteration is always closer to the solution set than both the MAP and DRM iterations. Our theoretical results and numerical experiments, showing outstanding performance, establish CRM as a powerful tool for solving general convex feasibility problems.

math.OC