arXiv subjects

Konstantinos Oikonomidis

Publications and source records attributed to Konstantinos Oikonomidis.

8 recordsLinked to original sources

arXiv2026

Scaled relative graphs for pairs of operators beyond classical monotonicity

We introduce a generalization of the scaled relative graph (SRG) to pairs of operators, enabling the visualization of their relative incremental properties. This novel SRG framework provides the geometric counterpart for the study of nonlinear resolvents based on paired monotonicity conditions. We demonstrate that these conditions apply to linear operators composed with monotone mappings, a class that notably includes NPN transistors, allowing us to compute the response of multivalued, nonsmooth, and highly nonmonotone electrical circuits.

math.OC↗

arXiv2026

Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives

In this work, we develop proximal preconditioned gradient methods with a focus on spectral gradient methods providing a proximal extension to the Muon and Scion optimizers. We introduce a family of stochastic algorithms that can handle a wide variety of convex and nonconvex constraints and study its convergence under heavy-tailed noise, through a novel analysis tailored to the geometry of the proposed methods. We further propose a variance-reduced version, which achieves faster convergence under standard noise assumptions. Finally, we show that the polynomial iterations used in Muon are more accurately captured by a nonlinear preconditioner than by the ideal matrix sign, leading to a convergence analysis that more faithfully reflects practical implementations.

math.OC↗

arXiv2026

Nonlinearly preconditioned gradient flows

We study a continuous-time dynamical system which arises as the limit of a broad class of nonlinearly preconditioned gradient methods. Under mild assumptions, we establish existence of global solutions and derive Lyapunov-based convergence guarantees. For convex costs, we prove a sublinear decay in a geometry induced by some reference function, and under a generalized gradient-dominance condition we obtain exponential convergence. We further uncover a duality connection with mirror descent, and use it to establish that the flow of interest solves an infinite-horizon optimal-control problem of which the value function is the Bregman divergence generated by the cost. These results clarify the structure and optimization behavior of nonlinearly preconditioned gradient flows and connect them to known continuous-time models in non-Euclidean optimization.

math.OC↗

arXiv2026

On the Regularity of Generalized Conjugate Functions

We investigate regularity properties of generalized conjugate functions induced by a general coupling function and the associated generalized proximal mapping. Our main results provide verifiable conditions ensuring local single-valuedness, continuity, Lipschitz continuity, and differentiability of the generalized proximal mapping, and transfer these properties to generalized conjugates providing explicit derivative formulas. These results are based on a nonsmooth implicit function theorem for generalized equations, relying on graphical localizations and second-order variational tools. Beyond first-order regularity, we also derive conditions under which generalized conjugates are strictly twice differentiable.

math.OC↗

arXiv2025

The inexact power augmented Lagrangian method for constrained nonconvex optimization

This work introduces an unconventional inexact augmented Lagrangian method where the augmenting term is a Euclidean norm raised to a power between one and two. The proposed algorithm is applicable to a broad class of constrained nonconvex minimization problems that involve nonlinear equality constraints. In a first part of this work, we conduct a full complexity analysis of the method under a mild regularity condition, leveraging an accelerated first-order algorithm for solving the Hölder-smooth subproblems. Interestingly, this worst-case result indicates that using lower powers for the augmenting term leads to faster constraint satisfaction, albeit with a slower decrease of the dual residual. Notably, our analysis does not assume boundedness of the iterates. Thereafter, we present an inexact proximal point method for solving the weakly-convex and Hölder-smooth subproblems, and demonstrate that the combined scheme attains an improved rate that reduces to the best-known convergence rate whenever the augmenting term is a classical squared Euclidean norm. Different augmenting terms, involving a lower power, further improve the primal complexity at the cost of the dual complexity. Finally, numerical experiments validate the practical performance of unconventional augmenting terms.

math.OC↗

arXiv2025

Nonlinearly Preconditioned Gradient Methods: Momentum and Stochastic Analysis

We study nonlinearly preconditioned gradient methods for smooth nonconvex optimization problems, focusing on sigmoid preconditioners that inherently perform a form of gradient clipping akin to the widely used gradient clipping technique. Building upon this idea, we introduce a novel heavy ball-type algorithm and provide convergence guarantees under a generalized smoothness condition that is less restrictive than traditional Lipschitz smoothness, thus covering a broader class of functions. Additionally, we develop a stochastic variant of the base method and study its convergence properties under different noise assumptions. We compare the proposed algorithms with baseline methods on diverse tasks from machine learning including neural network training.

math.OC↗

arXiv2025

Nonlinearly Preconditioned Gradient Methods under Generalized Smoothness

We analyze nonlinearly preconditioned gradient methods for solving smooth minimization problems. We introduce a generalized smoothness property, based on the notion of abstract convexity, that is broader than Lipschitz smoothness and provide sufficient first- and second-order conditions. Notably, our framework encapsulates algorithms associated with the gradient clipping method and brings out novel insights for the class of $(L_0,L_1)$-smooth functions that has received widespread interest recently, thus allowing us to extend beyond already established methods. We investigate the convergence of the proposed method in both the convex and nonconvex setting.

math.OC↗

arXiv2025

Forward-backward splitting under the light of generalized convexity

In this paper we present a unifying framework for continuous optimization methods grounded in the concept of generalized convexity. Utilizing the powerful theory of $Φ$-convexity, we propose a conceptual algorithm that extends the classical difference-of-convex method, encompassing a broad spectrum of optimization algorithms. Relying exclusively on the tools of generalized convexity we develop a gap function analysis that strictly characterizes the decrease of the function values, leading to simplified and unified convergence results. As an outcome of this analysis, we naturally obtain a generalized PL inequality which ensures $q$-linear convergence rates of the proposed method, incorporating various well-established conditions from the existing literature. Moreover we propose a $Φ$-Bregman proximal point interpretation of the scheme that allows us to capture conditions that lead to sublinear rates under convexity.

math.OC↗