Searcharxiv⌕ Search

arXiv subjects

Evgeniy Kovalev

Publications and source records attributed to Evgeniy Kovalev.

2 recordsLinked to original sources

Nonsmooth Convex and Strongly Convex Optimization from Noisy Comparisons

We study nonsmooth convex and strongly convex optimization problems using only noisy pairwise comparisons: each query specifies two points and returns a single binary response indicating which point has the smaller function value, subject to noise. We distinguish two observation models: Stochastic Order Oracles, which compare two evaluations of the same random loss, and Preference Oracles, whose conditional response mean is a transfer function of the objective difference. For Stochastic Order Oracles, we prove that no comparison-based algorithm can guarantee vanishing expected suboptimality in general, even when sample losses are convex and Lipschitz and the mean objective is strongly convex. Under a normalized probit calibration, we recover convergence and establish matching upper and lower rates for convex and strongly convex objectives. For Preference Oracles with a linear response model, the binary response $O \in \{\pm1\}$ satisfies $\E[O\mid y,z]=λ(f(y)-f(z))$, with $λ>0$. In dimension $d$, we obtain an upper bound of $\widetilde (d^{3/2}/(λ\sqrt N))$ and establish, to our knowledge, the first minimax lower bound of $Ω(d/(λ\sqrt N))$, matching the optimal budget $N$ exponent, for both convex and strongly convex nonsmooth objectives. Beyond exact linearity, we analyze a simple projected comparison method for nonlinear transfer functions and establish convergence rates that depend on how accurately these functions can be approximated by a linear model near zero. Our analysis requires neither oddness nor twice differentiability of the transfer function.

math.OC↗

Normalized First-Order Methods for Convex (L0, L1)-Smooth Optimization with Inexact Gradients

Generalized smoothness, such as (L0, L1)-smoothness, have recently attracted considerable attention due to their ability to model optimization problems arising in modern machine and deep learning, where the classical Lipschitz assumptions of the gradient is often violated. At the same time, computing exact gradients may be impractical or computationally expensive in many applications. In this work, we study convex (L0, L1)-smooth optimization (for normalized gradient method we consider quasi-convex problems too) under access only to a normalized approximation recently proposed Comparison Oracle, which returns an inexact normalized gradient in linear time with a bounded absolute error. Within this framework, we develop comparison-oracle variants of Normalized Gradient Descent and Gradient Descent with Polyak stepsizes. We establish explicit upper bounds on the approximation error that guarantee convergence and derive convergence rates for all proposed methods. Unlike existing analyses, our results require neither classical smoothness assumptions nor access to exact gradients or their exact normalized counterparts. Finally, numerical experiments corroborate the theoretical findings.

math.OC↗