arXiv subjects

Juan Peypouquet

Publications and source records attributed to Juan Peypouquet.

At least 19 recordsLinked to original sources

arXiv2026

Asymptotic behaviour of coupled random dynamical systems with multiscale aspects

We examine a class of stochastic differential inclusions involving multiscale effects designed to solve a class of generalized variational inequalities. This class of problems contains constrained convex non-smooth optimization problems, constrained saddle-point problems and various equilibrium problems in economics and engineering. In order to respect constraints we adopt a penalty approach, introducing an explicit time-dependency into the evolution system. The resulting dynamics are described in terms of a non-autonomous stochastic evolution equation governed by maximally monotone operators in the drift and perturbed by a Brownian motion. We study the asymptotic behavior, as well as finite time convergence rates in terms of gap functions. The condition we use to prove convergence involves a Legendre transform of the function describing the set constrained domain, a condition first used by Attouch and Czarnecki (J. Differ. Equations, Vol. 248, Issue 6, 2010) in the context of deterministic evolution equations. We also establish a large deviations principle showing that individual trajectories exhibit exponential concentration around the solution set. Finally we show how our continuous-time approach relates to penalty-regulated algorithms of forward-backward type after performing a suitable Euler-Maruyama discretization.

math.OC↗

arXiv2026

Stochastic Krasnoselskii-Mann Iterations: Convergence without Uniformly Bounded Variance

We investigate the Stochastic Krasnoselskii-Mann iterations for expected nonexpansive fixed-point problems in a real separable Hilbert space. We establish convergence guarantees under significantly weaker assumptions on the variance than those typically used in the literature. In particular, instead of a uniform bound on the variance of the stochastic oracle, we only assume finite variance at a single fixed point. Under this assumption, we prove almost sure weak convergence of the iterates, derive convergence rates for the expected residual and the last-iterate residual, and obtain almost sure convergence rates for the running minimum residual. Notably, we recover the best-known stochastic oracle complexity without imposing uniformly bounded variance. We illustrate the applicability of our results to Stochastic Gradient Descent, where we recover known guarantees, to Stochastic Three-Operator Splitting and Stochastic Backward-Forward Splitting, for which we obtain the first results that avoid uniform variance bounds, and to a novel Stochastic Lifted Three-Operator Splitting.

math.OC↗

arXiv2026

A unified analysis on the speedup of accelerated gradient methods: An inertial dynamics approach

Nesterov's Accelerated Gradient Method is one of the most popular first order optimization algorithms. When applied to $L$-smooth, $μ$-strongly convex functions, it converges at a rate of $\mathcal{O}\left(\left(1-\sqrt{\fracμ{L}}\right)^{k}\right)$. The more recent {\it Triple Momentum Method} and the {\it Information Theoretic Exact Method} enjoy an improved rate of $\mathcal{O}\left(\left(1-2\sqrt{\fracμ{L}}\right)^{k}\right)$. Their analysis relies on {\it integral quadratic constraints} and {\it performance estimation techniques}, respectively. In this work, we provide a dynamic explanation for this {\it factor 2 speedup} based on the subtle relationships between the coefficients of an inertial system with Hessian-driven damping. The standard explicit discretization of this second order ordinary differential equation produces intuitive variants of Nesterov's method with sped-up convergence rates. The proof strategy allows us to extend the analysis beyond the strongly convex setting to account for convex functions with quadratic growth or the Polyak-Łojasiewicz inequality. Under uniqueness of the minimizer, we establish a factor $\sqrt{2}$ speedup with respect to the state of the art. With no assumption on the set of minimizers, the new convergence rate (asymptotically) matches that of Gradient Descent, a fact that was previously unknown.

math.OC↗

arXiv2026

Stochastic Nonconvex Bilevel Optimization: Improved Rates Without Rare-Visit Assumption

We investigate stochastic simple bilevel optimization with smooth and possibly nonconvex upper- and lower-level objectives. Existing stochastic extensions of dynamic barrier gradient descent (DBGD) either obtain fast convergence under an unverifiable trajectory-dependent ``rare-visit'' assumption, or remove this assumption at a substantially higher oracle cost. We show that a simple denominator-only regularization of the DBGD multiplier eliminates the need for such an assumption while preserving fast convergence rates. Specifically, our method achieves $(\varepsilon, \varepsilon)$-stationarity in $O(\varepsilon^{-2})$ iterations using $O(\varepsilon^{-4})$ upper-level and $O(\varepsilon^{-7})$ lower-level stochastic gradients, which improves upon the best assumption-free complexities. We additionally derive anytime parameter schedules.

math.OC↗

arXiv2026

Convergence Rate Analysis for Monotone Accelerated Proximal Gradient Method

We propose a monotone accelerated proximal gradient method for solving convex composite optimization problems, guaranteeing nonincreasing function values along the iterates -- a property that improves numerical stability and is not enjoyed by standard accelerated schemes. The method generalizes the Monotone FISTA algorithm of Beck and Teboulle. In the convex setting, we establish the optimal rate of $\mathcal{O}\left( \frac{1}{k^2} \right)$ and show that all weak subsequential limit points of the iterates are minimizers. In the strongly convex setting, we establish a linear rate $\mathcal{O}\left( \frac{1}{k^2}(1+ρ)^{-k} \right)$ with $ρ\sim \fracμ{3L}$, without requiring knowledge of the strong convexity parameter -- roughly a five-fold improvement in the contraction constant over the best previously known rate for monotone methods, and more than 30% larger than that of the best known rate for non-monotone accelerated methods. Following a similar idea, we propose a variant of Nesterov's accelerated proximal method and establish a linear rate under strong convexity, which is the same as the one above, and is faster than the known results for non-monotone methods.

math.OC↗

arXiv2026

Preconditioned primal-dual algorithms for saddle point problems: non-ergodic convergence rates

We study a family of preconditioned primal dual algorithms for convex-concave saddle point problems by the dynamics introduced in \cite{apidopoulos2026preconditioned}. The proposed framework exploits the possible smooth + nonsmooth structure of the saddle point formulation. It includes, but is not limited to, linearly constrained convex optimization problems. The proposed antisymmetric preconditioners allow us to establish non ergodic convergence rates, accounting for possible computational errors in the implementation of the method. Finally, we present numerical experiments to indicate our well performed preconditioned primal dual algorithms.

math.OC↗

arXiv2026

Inertial Primal Dual Dynamics with Hessian-driven Damping for Saddle Point Problems

Featuring Hessian-driven damping, two inertial primal dual dynamical systems are proposed for solving smooth saddle point problems with bilinear coupling. For convex-concave functions, we establish a convergence rate $\mathcal{O}\left( \frac{1}{t^2} \right)$ for the primal dual gap; for strongly convex-strongly concave functions, we obtain an asymptotic rate $\mathcal{O}\left( \frac{1}{t^{α-1}} \right)$ ($α\ge 3$ is the damping parameter) without knowledge of the strong convexity parameters, and an accelerated linear convergence rate when the strong convexity parameters are known. As an application of the proposed inertial systems, we also consider the affinely constrained convex optimization problem, and develop an inertial system with Hessian-driven damping, which complements existing results.

math.OC↗

arXiv2026

Inertial forward-backward algorithm with exterior penalization and Tikhonov regularization

In a real Hilbertian setting, we develop in this paper numerical splitting techniques guaranteeing strong convergence to the least norm solution of constrained variational inequalities. We develop a multiscale inertial forward-backward splitting algorithm for solving constrained monotone inclusion problems with multiscale penalization and vanishing Tikhonov regularization. The proposed framework accommodates smooth, nonsmooth, and mixed smooth--nonsmooth penalty operators, providing a unified treatment of a broad class of constrained monotone inclusion problems. In this general framework, we establish weak convergence of the generated iterates. By introducing a discrete Tikhonov central path, we further prove strong convergence to the minimum-norm solution of the problem under a mild constraint qualification condition on the problem data.

math.OC↗

arXiv2026

Accelerated Backward Forward Method for Convex Optimization

We analyze the convergence rate of an accelerated backward forward method for solving convex composite optimization problems. The method was developed by Taylor, Hendrickx and Glineur, and is different from the FISTA algorithm in its placement of the proximal operator. When the smooth part of the objective function is convex, we establish a fast convergence rate of $\mathcal{O}\left( \frac{1}{k^2} \right)$ for the function values, and prove the weak convergence of the iterates. When the smooth part is strongly convex, we propose a variant of the method, and establish an accelerated linear convergence rate for the function values.

math.OC↗

arXiv2026

Towards faster first order methods: A continuous-time model to interpolate between speed and function value restart

We introduce a new restarting scheme for a continuous inertial dynamics with Hessian driven-damping, and establish a linear convergence rate for the function values along the restarted trajectories. The proposed routine is implemented without knowing the strong convexity parameter, and is a generalization of existing speed restart schemes. It interpolates between speed and function value restarts, considerably delaying the restarting time, while preserving convergence and function value decrease. Numerical experiments show an improvement in the convergence rates for both continuous-time dynamical systems, and the associated accelerated first-order algorithms derived via time discretization.

math.OC↗

arXiv2026

Bias-Optimal Bounds for SGD: A Computer-Aided Lyapunov Analysis

The non-asymptotic analysis of Stochastic Gradient Descent (SGD) typically yields bounds that decompose into a bias term and a variance term. In this work, we focus on the bias component and study the extent to which SGD can match the optimal convergence behavior of deterministic gradient descent. Assuming only (strong) convexity and smoothness of the objective, we derive new bounds that are bias-optimal, in the sense that the bias term coincides with the worst-case rate of gradient descent. Our results hold for the full range of constant step-sizes $γL \in (0,2)$, including critical and large step-size regimes that were previously unexplored without additional variance assumptions. The bounds are obtained through the construction of a simple Lyapunov energy whose monotonicity yields sharp convergence guarantees. To design the parameters of this energy, we employ the Performance Estimation Problem framework, which we also use to provide numerical evidence for the optimality of the associated variance terms.

math.OC↗

arXiv2025

Adaptive Accelerated Gradient Method for Smooth Convex Optimization

We propose an adaptive accelerated gradient method for solving smooth convex optimization problems. The method incorporates a scheme to determine the step size adaptively, by means of a local estimation of the smoothness constant, which is assumed unknown, without resorting to line search procedures. The sequence generated by this method converges weakly to a minimizer of the objective function, and the function values converge at a fast rate of $\mathcal{O}\left( \frac{1}{k^2} \right)$. Moreover, if the objective function is strongly convex, the function values converge at a linear rate.

math.OC↗

arXiv2025

Global Optimization Algorithm through High-Resolution Sampling

We present an optimization algorithm that can identify a global minimum of a potentially nonconvex smooth function with high probability, assuming the Gibbs measure of the potential satisfies a logarithmic Sobolev inequality. Our contribution is twofold: on the one hand we propose a global optimization method, which is built on an oracle sampling algorithm producing arbitrarily accurate samples from a given Gibbs measure. On the other hand, we propose a new sampling algorithm, drawing inspiration from both overdamped and underdamped Langevin dynamics, as well as from the high-resolution differential equation known for its acceleration in deterministic settings. While the focus of the paper is primarily theoretical, we demonstrate the effectiveness of our algorithms on the Rastrigin function, where it outperforms recent approaches.

math.OC↗

arXiv2025

Asymptotic behavior of penalty dynamics for constrained variational inequalities

We propose a comprehensive framework for solving constrained variational inequalities via various classes of evolution equations displaying multi-scale aspects. In an infinite-dimensional Hilbertian framework, the class of dynamical systems we propose combine Tikhonov regularization and exterior penalization terms in order to induce strong convergence of trajectories to least norm solutions in the constrained domain. Our construction thus unifies the literature on regularization methods and penalty-based dynamical systems. An extension to a full splitting formulation of the constrained domain is also provided, with associated weak convergence results involving the Attouch-Czarnecki condition.

math.OC↗

arXiv2025

Last-Iterate Complexity of SGD for Convex and Smooth Stochastic Problems

Most results on Stochastic Gradient Descent (SGD) in the convex and smooth setting are presented under the form of bounds on the ergodic function value gap. It is an open question whether bounds can be derived directly on the last iterate of SGD in this context. Recent advances suggest that it should be possible. For instance, it can be achieved by making the additional, yet unverifiable, assumption that the variance of the stochastic gradients is uniformly bounded. In this paper, we show that there is no need of such an assumption, and that SGD enjoys a $\tilde O \left( T^{-1/2} \right)$ last-iterate complexity rate for convex smooth stochastic problems.

math.OC↗

arXiv2025

Fast convex optimization via inertial systems with asymptotically vanishing viscosity and Hessian-driven damping

We study the convergence rate of a family of inertial algorithms, which can be obtained by discretization of an inertial system combining asymptotic vanishing viscous and Hessian-driven damping. We establish a fast sublinear convergence rate in case the objective function is convex and satisfies Polyak-Łojasiewicz inequality. We also establish a linear convergence rate for strongly convex functions. The results can provide more insights into the convergence property of Nesterov's accelerated gradient method.

math.OC↗

arXiv2025

Krasnoselskii-Mann Iterations: Inertia, Perturbations and Approximation

This paper is concerned with the study of a family of fixed point iterations combining relaxation with different inertial (acceleration) principles. We provide a systematic, unified and insightful analysis of the hypotheses that ensure their weak, strong and linear convergence, either matching or improving previous results obtained by analysing particular cases separately. We also show that these methods are robust with respect to different kinds of perturbations--which may come from computational errors, intentional deviations, as well as regularisation or approximation schemes--under surprisingly weak assumptions. Although we mostly focus on theoretical aspects, numerical illustrations in image inpainting and electricity production markets reveal possible trends in the behaviour of these types of methods.

math.OC↗

arXiv2025

Preconditioned primal-dual dynamics in convex optimization: non-ergodic convergence rates

We introduce and analyze a continuous primal-dual dynamical system in the context of the minimization problem $f(x)+g(Ax)$, where $f$ and $g$ are convex functions and $A$ is a linear operator. In this setting, the trajectories of the Arrow-Hurwicz continuous flow may not converge, accumulating at points that are not solutions. Our proposal is inspired by the primal-dual algorithm of Chambolle and Pock (2011), where convergence and splitting on the primal-dual variable are ensured by adequately preconditioning the proximal-point algorithm. We consider a family of preconditioners, which are allowed to depend on time and on the operator $A$, but not on the functions $f$ and $g$, and analyze asymptotic properties of the corresponding preconditioned flow. Fast convergence rates for the primal-dual gap and optimality of its (weak) limit points are obtained, in the general case, for asymptotically antisymmetric preconditioners, and, in the case of linearly constrained optimization problems, under milder hypotheses. Numerical examples support our theoretical findings, especially in favor of the antisymmetric preconditioners.

math.OC↗