arXiv subjects

Cheik Traoré

Publications and source records attributed to Cheik Traoré.

7 recordsLinked to original sources

arXiv2026

Piecewise smooth functions and conservative fields: calculus for nonsmooth nonconvex optimization beyond stratification

In this paper, we show that the piecewise gradient associated with a representation of a continuous piecewise-$C^{p}$ function is a selection of a conservative field. Specifically, we prove that a set-valued map whose selections include the piecewise gradients, also called associated gradients, has the chain rule property along Lipschitz curves. As a consequence, continuous piecewise smooth functions are path differentiable and their Clarke subdifferentials satisfy the chain rule property. These results establish a connection between a representation-based calculus for nonsmooth automatic differentiation and the conservative-field framework. From an algorithmic perspective, we show that bounded iterates of the stochastic piecewise-gradient method converge to the set of conservative critical points. With an additional Lebesgue-null interface condition, a generic stepsize scaling, and a generic initialization, this result holds for the Clarke critical set. Finally, we demonstrate that, even with the interface condition, the graph of a Lipschitz continuous and piecewise-$C^{\infty}$ function need not admit a $C^1$ stratification satisfying Whitney's condition (a). Thus, the analysis developed here does not fall within the setting of Whitney stratifiable functions (e.g., semialgebraic functions).

math.OC↗

arXiv2026

The adjoint state method for parametric definable optimization without smoothness or uniqueness

We establish that nonconvex definable parametric optimization problems with possibly nonsmooth objectives, inequality constraints, conic constraint systems, and non-unique primal and dual solutions admit an adjoint state formula under a mere qualification condition. The adjoint construction yields a selection of a conservative field for the value function, providing a computable first-order object without requiring differentiation of the solution mapping. Through examples, we show that even in smooth problems, the formal adjoint construction fails without conservativity or definability, illustrating the relevance of these concepts to grasp theoretical aspects of the method. This work provides a tool which can be directly combined with existing primal-dual solvers for a wide range of parametric optimization problems.

math.OC↗

arXiv2026

Bregman Stochastic Proximal Point Algorithm with Variance Reduction

Stochastic algorithms, especially stochastic gradient descent (SGD), have proven to be the go-to methods in data science and machine learning. In recent years, the stochastic proximal point algorithm (SPPA) emerged, and it was shown to be more robust than SGD with respect to stepsize settings. However, SPPA still suffers from a decreased convergence rate due to the need for vanishing stepsizes, which is resolved by using variance reduction methods. In the deterministic setting, there are many problems that can be solved more efficiently when viewing them in a non-Euclidean geometry using Bregman distances. This paper combines these two worlds and proposes variance reduction techniques for the Bregman stochastic proximal point algorithm (BSPPA). As special cases, we obtain SAGA- and SVRG-like variance reduction techniques for BSPPA. Our theoretical and numerical results demonstrate improved stability and convergence rates compared to the vanilla BSPPA with constant and vanishing stepsizes, respectively. Our analysis, also, allow to recover the same variance reduction techniques for Bregman SGD in a unified way.

math.OC↗

arXiv2025

A Structured Proximal Stochastic Variance Reduced Zeroth-order Algorithm

Minimizing finite sums of functions is a central problem in optimization, arising in numerous practical applications. Such problems are commonly addressed using first-order optimization methods. However, these procedures cannot be used in settings where gradient information is unavailable. Finite-difference methods provide an alternative by approximating gradients through function evaluations along a set of directions. For finite-sum minimization problems, it was shown that incorporating variance-reduction techniques into finite-difference methods can improve convergence rates. Additionally, recent studies showed that imposing structure on the directions (e.g., orthogonality) enhances performance. However, the impact of structured directions on variance-reduced finite-difference methods remains unexplored. In this work, we close this gap by proposing a structured variance-reduced finite-difference algorithm for non-smooth finite-sum minimization. We analyze the proposed method, establishing convergence rates for non-convex functions and those satisfying the Polyak-Łojasiewicz condition. Our results show that our algorithm achieves state-of-the-art convergence rates while incurring lower per-iteration costs. Finally, numerical experiments highlight the strong practical performance of our method.

math.OC↗

arXiv2024

Variance reduction techniques for stochastic proximal point algorithms

In the context of finite sums minimization, variance reduction techniques are widely used to improve the performance of state-of-the-art stochastic gradient methods. Their practical impact is clear, as well as their theoretical properties. Stochastic proximal point algorithms have been studied as an alternative to stochastic gradient algorithms since they are more stable with respect to the choice of the step size. However, their variance-reduced versions are not as well studied as the gradient ones. In this work, we propose the first unified study of variance reduction techniques for stochastic proximal point algorithms. We introduce a generic stochastic proximal-based algorithm that can be specified to give the proximal version of SVRG, SAGA, and some of their variants. For this algorithm, in the smooth setting, we provide several convergence rates for the iterates and the objective function values, which are faster than those of the vanilla stochastic proximal point algorithm. More specifically, for convex functions, we prove a sublinear convergence rate of $O(1/k)$. In addition, under the Polyak-Łojasiewicz (PL) condition, we obtain linear convergence rates. Finally, our numerical experiments demonstrate the advantages of the proximal variance reduction methods over their gradient counterparts in terms of the stability with respect to the choice of the step size in most cases, especially for difficult problems.

math.OC↗

arXiv2023

Convergence of an Asynchronous Block-Coordinate Forward-Backward Algorithm for Convex Composite Optimization

In this paper, we study the convergence properties of a randomized block-coordinate descent algorithm for the minimization of a composite convex objective function, where the block-coordinates are updated asynchronously and randomly according to an arbitrary probability distribution. We prove that the iterates generated by the algorithm form a stochastic quasi-Fejér sequence and thus converge almost surely to a minimizer of the objective function. Moreover, we prove a general sublinear rate of convergence in expectation for the function values and a linear rate of convergence in expectation under an error bound condition of Tseng type.

math.OC↗

arXiv2021

Sequential convergence of AdaGrad algorithm for smooth convex optimization

We prove that the iterates produced by, either the scalar step size variant, or the coordinatewise variant of AdaGrad algorithm, are convergent sequences when applied to convex objective functions with Lipschitz gradient. The key insight is to remark that such AdaGrad sequences satisfy a variable metric quasi-Fejér monotonicity property, which allows to prove convergence.

math.OC↗