Searcharxiv⌕ Search

arXiv subjects

Lateef O. Jolaoso

Publications and source records attributed to Lateef O. Jolaoso.

7 recordsLinked to original sources

Primal Methods for Constrained Variational Inequalities: Optimal Rates, Feasibility Trade-offs, and Lower Bounds

Since Zhang et al. \cite{Zhang2025} introduced the first purely primal methods for monotone variational inequalities subject to convex functional constraints, the field has lacked a complete understanding of the complexity limits of this oracle class. We develop a theory that resolves the critical gaps left by their pioneering framework. Our Optimal Primal Constrained Gradient Method (OPCGM) establishes convergence rates and lower bounds for algorithms restricted to local linear constraint approximations. For strongly monotone operators, we prove that a refined primal gradient method achieves the optimal $\mathcal{O}(1/T)$ rate for both optimality gap and constraint violation, eliminating the suboptimal exponent present in prior work. For Lipschitz monotone operators, we propose a primal extragradient variant that achieves the optimal $\mathcal{O}(1/ε)$ gap complexity using only quadratic programming oracles, at the cost of a constant asymptotic feasibility violation for the averaged iterate; we prove that the first half-step is necessarily infeasible for the natural class of constant-stepsize primal extragradient methods on smooth convex constraints with positive curvature. We establish lower complexity bounds for primal methods restricted to local linear approximations, proving $Ω(1/ε^2)$ for Lipschitz monotone variational inequalities with Lipschitz constant scaling as $Θ(1/ε)$, and $Ω(1/ε)$ for standard Lipschitz monotone variational inequalities. We design a single-loop primal method that achieves an $\mathcal{O}(1/\sqrt{T})$ gap rate with only a domain-radius estimate, at the cost of a problem-dependent asymptotic feasibility constant that we prove is unavoidable. For strongly monotone problems, we prove that the last iterate converges at the optimal $\mathcal{O}(1/T)$ rate.

math.OC↗

A Riemannian Inertial Adaptive Gradient Method with Momentum Restart

We propose RIAG-R, a Riemannian Inertial Adaptive Gradient method with gradient-triggered Restart, for optimization on Riemannian manifolds. RIAG-R combines three ingredients: an AdaGrad-type adaptive step-size that requires no knowledge of the Lipschitz constant; inertial extrapolation in the tangent space via the exponential map to accelerate convergence; and a restart rule, costing only one extra inner product per iteration, that detects and corrects the momentum-saturation pathology induced by manifold curvature. We establish a per-iteration descent inequality, an optimal $O(\varepsilon^{-2})$-iteration complexity, a convergence result for cluster points of the generated sequence, and a linear convergence rate under the Riemannian Polyak-Łojasiewicz inequality. Numerical experiments on four representative matrix manifold optimization problems including sphere, Stiefel, symmetric positive-definite, and Grassmann manifolds demonstrate that RIAG-R consistently outperforms non-inertial and restart-free inertial baselines, confirming that adaptive restart is essential to realizing the benefits of momentum on curved spaces.

math.OC↗

Three-Operator Splitting Method with Two-Step Inertial Extrapolation

The aim of this paper is to study the weak convergence analysis of sequence of iterates generated by a three-operator splitting method of Davis and Yin incorporated with two-step inertial extrapolation for solving monotone inclusion problem involving the sum of two maximal monotone operators and a co-coercive operator in Hilbert spaces. Our results improve on the setbacks observed recently in the literature that one-step inertial Douglas-Rachford splitting method may fail to provide acceleration. Our convergence results also dispense with the summability conditions imposed on inertial parameters and the sequence of iterates assumed in recent results on multi-step inertial methods in the literature. Numerical illustrations from image restoration problem and Smoothly Clipped Absolute Deviation (SCAD) penalty problem are given to show the efficiency and advantage gained by incorporating two-step inertial extrapolation over one-step inertial extrapolation for three-operator splitting method.

math.OC↗

New Douglas-Rashford Splitting Algorithms for Generalized DC Programming with Applications in Machine Learning

In this work, we propose some new Douglas-Rashford splitting algorithms for solving a class of generalized DC (difference of convex functions) in real Hilbert spaces. The proposed methods leverage the proximal properties of the nonsmooth component and a fasten control parameter which improves the convergence rate of the algorithms. We prove the convergence of these methods to the critical points of nonconvex optimization under reasonable conditions. We evaluate the performance and effectiveness of our methods through experimentation with three practical examples in machine learning. Our findings demonstrated that our methods offer efficiency in problem-solving and outperform state-of-the-art techniques like the DCA (DC Algorithm) and ADMM.

math.OC↗

Linear Convergence Results for Inertial Type Projection Algorithm for Quasi-Variational Inequalities

Many recently proposed gradient projection algorithms with inertial extrapolation step for solving quasi-variational inequalities in Hilbert spaces are proven to be strongly convergent with no linear rate given when the cost operator is strongly monotone and Lipschitz continuous. In this paper, our aim is to design an inertial type gradient projection algorithm for quasi-variational inequalities and obtain its linear rate of convergence. Therefore, our results fill in the gap for linear convergence results for inertial type gradient projection algorithms for quasi variational inequalities in Hilbert spaces. We perform numerical implementations of our proposed algorithm and give numerical comparisons with other related inertial type gradient projection algorithms for quasi variational inequalities in the literature.

math.OC↗

A robust regularized extreme learning machine for regression problems based on self-adaptive accelerated extra-gradient algorithm

The Extreme Learning Machine (ELM) technique is a machine learning approach for constructing feed-forward neural networks with a single hidden layer and their models. The ELM model can be constructed while being trained by concurrently reducing both the modeling errors and the norm of the output weights. Usually, the squared loss is widely utilized in the objective function of ELM, which can be treated as a LASSO problem and hence applying the fast iterative shrinkage thresholding algorithm (FISTA). However, in this paper, the minimization problem is solved from a variational inequalities perspective giving rise to improved algorithms that are more efficient than the FISTA. A fast general extra-gradient algorithm which is a form of first-order algorithm is developed with inertial acceleration techniques and variable stepsize which is updated at every iteration. The strong convergence and linear rate of convergence of the proposed algorithm are established under mild conditions. In terms of experiments, two experiments were presented to illustrate the computational advantage of the proposed method to other algorithms.

math.OC↗

A fresh look at nonsmooth Levenberg--Marquardt methods with applications to bilevel optimization

In this paper, we revisit the classical problem of solving over-determined systems of nonsmooth equations numerically. We suggest a nonsmooth Levenberg--Marquardt method for its solution which, in contrast to the existing literature, does not require local Lipschitzness of the data functions. This is possible when using Newton-differentiability instead of semismoothness as the underlying tool of generalized differentiation. Conditions for fast local convergence of the method are given. Afterwards, in the context of over-determined mixed nonlinear complementarity systems, our findings are applied, and globalized solution methods, based on a residual induced by the maximum and the Fischer--Burmeister function, respectively, are constructed. The assumptions for fast local convergence are worked out and compared. Finally, these methods are applied for the numerical solution of bilevel optimization problems. We recall the derivation of a stationarity condition taking the shape of an over-determined mixed nonlinear complementarity system involving a penalty parameter, formulate assumptions for local fast convergence of our solution methods explicitly, and present results of numerical experiments. Particularly, we investigate whether the treatment of the appearing penalty parameter as an additional variable is beneficial or not.

math.OC↗