Searcharxiv⌕ Search

arXiv subjects

Alexander V. Gasnikov

Publications and source records attributed to Alexander V. Gasnikov.

7 recordsLinked to original sources

Descending Everest, Part III: Oracle Complexity of Monotone Variational Inequalities with $(L_0,L_1)$-Lipschitz Operators

We consider monotone variational inequalities in which the Jacobian norm of the operator $F:\mathbb{R}^d\to\mathbb{R}^d$ is bounded by an affine function of the norm of the operator itself: $\|DF(z)\|\le L_0+L_1\|F(z)\|$. The main case is the problem on the whole space, that is, finding a zero of $F$. We assume that a solution exists and that an upper bound $R$ on the distance from the starting point to the solution is known. Neither strong monotonicity nor cocoercivity is assumed. Global Lipschitz continuity is not assumed either: for $L_1>0$, the Jacobian norm may grow together with the norm of the operator. We obtain upper and lower bounds on the number of oracle calls sufficient to find a point $\hat z$ with $\|F(\hat z)\|\le\varepsilon$. For $L_0R/\varepsilon\ge1$, both bounds are, up to logarithmic factors in $L_1R$, of order $1+L_0R/\varepsilon+(L_1R)^{6/5}$. The upper bound is attained by a deterministic extragradient-type method, and the lower bound holds for arbitrary randomized methods. The same bounds are obtained for the Minty gap on the ball of radius $R$, where $L_0R/\varepsilon$ is replaced by $L_0R^2/η$. In addition, we show that for a variational inequality on a bounded set the numbers $L_0,L_1$ and the diameter of the set are not sufficient: a uniform complexity bound requires additional information, for example, an upper bound on the norm of the operator at the solution.

math.OC↗

Descending Everest, Part I: Oracle Complexity of Convex Optimization under $(L_0,L_1)$-Smoothness

For convex $(L_0,L_1)$-smooth functions, the curvature may grow together with the gradient norm. Therefore, even when $\|x^0-x^*\|\le R$, the gradient and the optimality gap at the starting point may be exponentially large in $L_1R$, and the bound $\sqrt{L_0R^2/\varepsilon}$, known for the accelerated method on functions with Lipschitz gradient, no longer describes the complexity of the problem. We determine the oracle complexity of this class for arbitrary deterministic first-order methods: for $L_0R^2/\varepsilon\ge1$, the optimal number of oracle calls $N$ satisfies $1+N=\widetildeΘ(\sqrt{L_0R^2/\varepsilon}+L_1R)$. The upper bound reads $O(\sqrt{L_0R^2/\varepsilon}+(1+L_1R)\log(2+L_1R))$: the method first passes through the large-gradient regime in $O(1+L_1R)$ iterations with an inexact line search, and then the standard accelerated method with a constant stepsize takes over. The lower bound is obtained by means of chains and the orthogonal invariance principle on functions conjugate to strongly convex functions on a polytope. If the initial optimality gap is at most $4\varepsilon$, the optimal bound becomes $1+N=\widetildeΘ(\sqrt{L_0R^2/\varepsilon}+(L_1R)^{2/3})$.

math.OC↗

Descending Everest, Part II: Oracle Complexity of Minimizing a Sum of Convex $(L_0,L_1)$-Smooth Functions

We study the minimization of the average of $n$ convex $(L_0,L_1)$-smooth functions: the Hessian norm of each function is bounded by an affine function of its gradient norm. We assume known upper bounds $R$ on the distance from the starting point to a minimizer and $δ$ on the difference between the minimum of the average and the average of the infima of the components. We derive upper and lower bounds on the oracle complexity of reaching accuracy $\varepsilon$ in function value in expectation. Up to logarithmic factors, both bounds are of order $n(1+(L_1R)^{2/3}) +\sqrt{\frac{n(L_0+L_1^2δ)R^2}{\varepsilon}+n(L_1R)^2}$. The upper bound is attained by the proposed method YETI. The method combines ideas of accelerated proximal methods and variance reduction methods with a heuristic for choosing additional points. The number of oracle calls made by the heuristic is bounded, which preserves the upper bound on the total number of queries needed to reach the target accuracy. In the experiments, YETI is compared with first-order methods on synthetic problems and Poisson regression problems. We also examine the stopping criterion and the effect of the heuristic separately. The lower bound is obtained by an explicit construction of families of instances that are hard for any first-order method.

math.OC↗

TIGER: Tunable-Geometry Accelerated Gradient Sliding with Adaptive Safeguards under Similarity

We study distributed composite optimization under function similarity when synchronized gradient queries dominate local computation. We propose TIGER, an accelerated gradient-sliding method in problem-adapted Bregman geometry. The method applies to constrained non-Euclidean domains, allows the communicated correction term to be nonconvex, and terminates each local subproblem using a computable stationarity residual. Each iteration reuses one synchronized correction gradient during the local solve and makes a second query to construct the composite subgradient for the mirror step. Under relative strong convexity and cocoercivity of the stabilized model error, the communication complexity is $O((1+\sqrt{δ_ω/μ})\log(1/\varepsilon))$, where $μ$ is the relative strong-convexity parameter and $δ_ω$ is a geometry-adjusted similarity parameter. The analysis recovers the Hessian-similarity rate in Hilbert geometry and provides an explicit finite bound on $δ_ω$ in entropy geometry on a truncated simplex. We also introduce Adaptive TIGER, which selects the stabilization coefficient and the similarity estimate during the run. Under uniform backtracking bounds, the adaptive method has the same asymptotic communication order as fixed-parameter TIGER, including the queries spent on backtracking. Numerical experiments compare Adaptive TIGER with several applicable first-order methods on entropic Wasserstein barycenter problems using the Fashion-MNIST dataset.

math.OC↗

Saddle-Point Problems with a Low-Dimensional Block Do Not Need Accurate Inner Solves

Many learning problems couple a high-dimensional block of parameters with a handful of adversarial or dual variables: worst-group risk over a few groups, or learning under a few constraints. We consider $\min_{x\in X}\max_{y\in Y} f(x,y)$ with $X\subseteq\mathbb{R}^n$, a compact convex set and strongly convex with condition number $κ$, and we count the oracle calls for $x$ and for $y$ separately. The textbook approach runs a cutting-plane method in $y$ and solves every inner problem to accuracy $\varepsilon$ with an accelerated method; it needs $O(m\log(1/\varepsilon))$ calls for $y$ but $O(m\sqrtκ\log^2(1/\varepsilon))$ calls for $x$. We show that the inner problems need not be solved accurately. Our method, certificate transport, keeps a strongly convex lower model of a single slice and uses it as a prior for a short accelerated run on the next slice. By concavity in $y$, every call for $y$ then either cuts the localizer or moves the lower model to a mixture of the two slices with a certified increase of the lower bound. For $m=1$ this gives an $\varepsilon$-saddle point after $O(\sqrtκ\log(1/\varepsilon))$ calls for $x$, up to an initialization term, and $O(\log(1/\varepsilon))$ calls for $y$; both counts are optimal, even though $f(x,\cdot)$ is only assumed to be concave and Lipschitz. For general $m$, with centers of gravity of the localizer treated as computable, the method needs $O((m+\sqrt{mκ})\log(1/\varepsilon))$ calls for $x$ and the optimal $O(m\log(1/\varepsilon))$ calls for $y$. Under strong concavity in $y$, a two-point accelerated method makes the number of calls for $x$ independent of $m$.

math.OC↗

Application of Optimal Inexact Second-Order Acceleration to Distributed Stochastic Optimization under Statistical Similarity

We consider distributed stochastic convex optimization with a fixed budget of $N$ independent samples split among $m$ workers. Sample average approximation reduces the problem to a regularized finite-sum problem whose local Hessians are statistically similar. This allows the Hessian of the local objective at the server to be used as an inexact Hessian of the global objective, while the workers communicate only gradients. We apply the optimal accelerated inexact Newton extragradient method of (Chen et al., 2026) and propose its distributed restarted variant for the strongly convex empirical problem. The method reaches the statistical accuracy of order $N^{-1/2}$ in $\widetilde O\left(\max\{N^{1/7},m^{1/4}\}\right)$ communication rounds. Hence, with $m=N^{4/7}$ workers, it requires $\widetilde O\left(N^{1/7}\right)$ rounds, improving the dependence on the total sample size from $\widetilde O\left(N^{1/6}\right)$ for the previous accelerated cubic Newton construction of (Agafonov et al., 2021). Each iteration uses two gradient aggregation rounds and does not require Hessian communication.

math.OC↗

Certified Residual Quasi-Newton Methods for Distributed Variational Inequalities

Second-order methods for smooth monotone variational inequalities reach the optimal rate $O(T^{-3/2})$, but a distributed exact Jacobian costs $d$ times more communication than an operator value. We show that similarity does part of the work for free: if the server's Jacobian differs from the global one by at most $β$, using it gives $O(L_1D^3T^{-3/2}+βD^2T^{-1})$ at first-order communication cost. A quasi-Newton approximation of the residual Jacobian $\nabla F-\nabla F_1$, built from secants already communicated, improves the model but cannot remove the $T^{-1}$ term, because any uniform bound on the Jacobian error leaves it in the rate. We therefore certify the surrogate only along the candidate step: one Jacobian-vector product tests it, and a failed test is reused as an exact correction. This attains the exact rate $O(L_1D^3T^{-3/2})$ while transmitting only vectors. Experiments on LIBSVM and synthetic instances measure accuracy against communication.

math.OC↗