arXiv subjects

Radu Ioan Boţ

Publications and source records attributed to Radu Ioan Boţ.

8 recordsLinked to original sources

arXiv2025

Accelerating Diagonal Methods for Bilevel Optimization: Unified Convergence via Continuous-Time Dynamics

We analyze fast diagonal methods for simple bilevel programs. Guided by the analysis of the corresponding continuous-time dynamics, we provide a unified convergence analysis under general geometric conditions, including Hölderian growth and the Attouch-Czarnecki condition. Our results yield explicit convergence rates and guarantee weak convergence to a solution of the bilevel problem. In particular, we improve and extend recent results on accelerated schemes, offering novel insights into the trade-offs between geometry, regularization decay, and algorithmic design. Numerical experiments illustrate the advantages of more flexible methods and support our theoretical findings.

math.OC↗

arXiv2025

A full splitting algorithm for fractional programs with structured numerators and denominators

In this paper, we consider a class of nonconvex and nonsmooth fractional programming problems, that involve the sum of a convex, possibly nonsmooth function composed with a linear operator and a differentiable, possibly nonconvex function in the numerator and a convex, possibly nonsmooth function composed with a linear operator in the denominator. These problems have applications in various fields. We propose an adaptive full-splitting proximal subgradient algorithm that addresses the challenge of decoupling the composition of the nonsmooth component with the linear operator in the numerator. We specifically evaluate the nonsmooth function in the numerator using its proximal operator of its conjugate function. Furthermore, the smooth component in the numerator is evaluated through its gradient, and the nonsmooth in the denominator is managed using its subgradient. We demonstrate subsequential convergence toward an approximate lifted stationary point and ensure global convergence under the Kurdyka-Łojasiewicz property, all achieved without full-row rank assumptions on the linear operators. We provide further discussions on {\it the tightness of the convergence results of the proposed algorithm and its related variants, and the reasoning behind aiming for an approximate lifted stationary point}. We construct a series of counter-examples to show that the proposed algorithm and its variant might diverge when seeking exact solutions. A practical version incorporating a nonmonotone line search is also developed to enhance its performance significantly. Our theoretical findings are validated through simulations involving limited-angle CT reconstruction and the robust sharp-ratio-type minimization problem.

math.OC↗

arXiv2024

Extra-Gradient Method with Flexible Anchoring: Strong Convergence and Fast Residual Decay

In this paper, we introduce a novel Extra-Gradient method with anchor term governed by general parameters. Our method is derived from an explicit discretization of a Tikhonov-regularized monotone flow in Hilbert space, which provides a theoretical foundation for analyzing its convergence properties. We establish strong convergence to specific points within the solution set, as well as convergence rates expressed in terms of the regularization parameters. Notably, our approach recovers the fast residual decay rate $O(k^{-1})$ for standard parameter choices. Numerical experiments highlight the competitiveness of the method and demonstrate how its flexible design enhances practical performance.

math.OC↗

arXiv2023

Inertial Proximal Block Coordinate Method for a Class of Nonsmooth Sum-of-Ratios Optimization Problems

In this paper, we consider a class of nonsmooth sum-of-ratios fractional optimization problems with block structure. This model class is ubiquitous and encompasses several important nonsmooth optimization problems in the literature. We first propose an inertial proximal block coordinate method for solving this class of problems by exploiting the underlying structure. The global convergence of our method is guaranteed under the Kurdyka--Lojasiewicz (KL) property and some mild assumptions. We then identify the explicit exponents of the KL property for three important structured fractional optimization problems. In particular, for the sparse generalized eigenvalue problem with either cardinality regularization or sparsity constraint, we show that the KL exponents are 1/2, and so, the proposed method exhibits linear convergence rate. Finally, we illustrate our theoretical results with both analytic and simulated numerical examples.

math.OC↗

arXiv2023

Alternating proximal-gradient steps for (stochastic) nonconvex-concave minimax problems

Minimax problems of the form $\min_x \max_y Ψ(x,y)$ have attracted increased interest largely due to advances in machine learning, in particular generative adversarial networks. These are typically trained using variants of stochastic gradient descent for the two players. Although convex-concave problems are well understood with many efficient solution methods to choose from, theoretical guarantees outside of this setting are sometimes lacking even for the simplest algorithms. In particular, this is the case for alternating gradient descent ascent, where the two agents take turns updating their strategies. To partially close this gap in the literature we prove a novel global convergence rate for the stochastic version of this method for finding a critical point of $g(\cdot) := \max_y Ψ(\cdot,y)$ in a setting which is not convex-concave.

math.OC↗

arXiv2020

Extrapolated Proximal Subgradient Algorithms for Nonconvex and Nonsmooth Fractional Programs

In this paper, we consider a broad class of nonsmooth and nonconvex fractional programs, where the numerator can be written as the sum of a continuously differentiable convex function whose gradient is Lipschitz continuous and a proper lower semicontinuous (possibly nonconvex) function, and the denominator is weakly convex over the constraint set. This model problem includes the composite optimization problems studied extensively lately, and encompasses many important modern fractional optimization problems arising from diverse areas such as the recently proposed scale invariant sparse signal reconstruction problem in signal processing. We propose a proximal subgradient algorithm with extrapolations for solving this optimization model and show that the iterated sequence generated by the algorithm is bounded and any of its limit points is a stationary point of the model problem. The choice of our extrapolation parameter is flexible and includes the popular extrapolation parameter adopted in the restarted Fast Iterative Shrinking-Threshold Algorithm (FISTA). By providing a unified analysis framework of descent methods, we establish the convergence of the full sequence under the assumption that a suitable merit function satisfies the Kurdyka--Łojasiewicz (KL) property. In particular, our algorithm exhibits linear convergence for the scale invariant sparse signal reconstruction problem and the Rayleigh quotient problem over spherical constraint. In the case where the denominator is the maximum of finitely many continuously differentiable weakly convex functions, we also propose an enhanced extrapolated proximal subgradient algorithm with guaranteed convergence to a stronger notion of stationary points of the model problem. Finally, we illustrate the proposed methods by both analytical and simulated numerical examples.

math.OC↗

arXiv2020

Two steps at a time -- taking GAN training in stride with Tseng's method

Motivated by the training of Generative Adversarial Networks (GANs), we study methods for solving minimax problems with additional nonsmooth regularizers. We do so by employing \emph{monotone operator} theory, in particular the \emph{Forward-Backward-Forward (FBF)} method, which avoids the known issue of limit cycling by correcting each update by a second gradient evaluation. Furthermore, we propose a seemingly new scheme which recycles old gradients to mitigate the additional computational cost. In doing so we rediscover a known method, related to \emph{Optimistic Gradient Descent Ascent (OGDA)}. For both schemes we prove novel convergence rates for convex-concave minimax problems via a unifying approach. The derived error bounds are in terms of the gap function for the ergodic iterates. For the deterministic and the stochastic problem we show a convergence rate of $\mathcal{O}(1/k)$ and $\mathcal{O}(1/\sqrt{k})$, respectively. We complement our theoretical results with empirical improvements in the training of Wasserstein GANs on the CIFAR10 dataset.

math.OC↗

arXiv2019

Inducing strong convergence of trajectories in dynamical systems associated to monotone inclusions with composite structure

In this work we investigate dynamical systems designed to approach the solution sets of inclusion problems involving the sum of two maximally monotone operators. Our aim is to design methods which guarantee strong convergence of trajectories towards the minimum norm solution of the underlying monotone inclusion problem. To that end, we investigate in detail the asymptotic behavior of dynamical systems perturbed by a Tikhonov regularization where either the maximally monotone operators themselves, or the vector field of the dynamical system is regularized. In both cases we prove strong convergence of the trajectories towards minimum norm solutions to an underlying monotone inclusion problem, and we illustrate numerically qualitative differences between these two complementary regularization strategies. The so-constructed dynamical systems are either of Krasnoselskii-Mann, of forward-backward type or of forward-backward-forward type, and with the help of injected regularization we demonstrate seminal results on the strong convergence of Hilbert space valued evolutions designed to solve monotone inclusion and equilibrium problems.

math.OC↗