arXiv subjects

Kirill Novoselov

Publications and source records attributed to Kirill Novoselov.

6 recordsLinked to original sources

arXiv2026

Federated Affine Variational Inequalities: Exact Averaging and Complexity Limits

We study the classical Local Stochastic Extragradient method (Local SEG) for monotone variational inequalities under intermittent communication, where devices perform several local oracle calls between synchronization rounds. For affine monotone mean operators, including nonsymmetric bilinear saddle systems, we prove that equal device averaging commutes pathwise with both stages of the stochastic extragradient update. Local trajectories may therefore drift apart while their mean follows a centralized stochastic-extragradient recursion driven by averaged noise. The identity remains valid when individual oracle realizations are nonlinear and query dependent. Under conditionally centered oracle errors that are conditionally independent across devices within each stage, the reduction yields a restricted weak-gap rate that uses the local sequential depth for deterministic progress and the aggregate oracle budget across devices for variance reduction. When every device makes at least two local oracle calls per communication round, the bound contains no additional mixed client-drift term and is also attained by a terminal-only aggregation schedule. An affine Gaussian lower bound matches the depth and sample dependences before statistical saturation for distributed zero-respecting algorithms; its statistical component applies to arbitrary randomized algorithms. A complementary lower bound for feasible distributed zero-respecting methods shows that bounded geometry and query-dependent noise can create an additional communication-locality obstruction. These results delineate the exact-averaging regime of unconstrained affine Local SEG and a complexity boundary that arises under active feasibility constraints.

math.OC↗

arXiv2026

The Minimax Complexity of Federated Optimization under Interpolation: The Role of Oracle Models

We study the minimax complexity of smooth convex stochastic optimization with intermittent communication: $m$ workers run for $N$ rounds, and between consecutive communication rounds each worker makes $K$ sequential stochastic gradient queries. We focus on the interpolation regime, in which all sample losses share a common minimizer, so that the gradient noise vanishes at the solution. Let $L$ be the smoothness of the objective, $H_s\ge L$ the smoothness of the sample losses, and $R$ the distance from the initial point to a minimizer. We consider three oracle models that provide increasing information: an unbiased stochastic gradient oracle $g(x,ξ)$ satisfying $\mathbb{E}_ξ\|g(x,ξ)\|^2 \leq 2H_s\bigl(F(x)-F(x^*)\bigr)$, gradients of the sample losses at fresh samples, and repeated gradient queries to stored samples. Up to logarithmic factors, all our results are described by two rates: \[ R^2\min\left\{L,\;\frac{H_s}{KN},\; \frac{L}{N^2}+\frac{H_s}{mKN}\right\} \qquad\text{and}\qquad R^2\min\left\{L,\; \frac{\min\{L,H_s/K\}}{N^2}+\frac{H_s}{mKN}\right\}, \] Under the weak growth condition, the first rate is the minimax rate, attained by the best of three simple baselines, as in the classical bounded-variance setting. With repeated access to samples, the second rate becomes the minimax rate for general convex objectives. For general convex objectives with fresh sample gradients, the minimax rate lies between the two rates. Our lower bounds apply to arbitrary randomized intermittent-communication algorithms.

math.OC↗

arXiv2026

Beyond Aggregate Drift: Optimal Client-Specific Windows in Federated Learning

We study federated learning from continuously arriving data: each client samples from its own distribution, these distributions drift over time, and the server must track the minimizer of the clients' weighted average loss. Clients are not alike as they collect data at different rates, with different noise, and change at different speeds. The main question is how much history should each one keep? A single shared window must serve every client at once, and so makes a fast-changing client carry the sampling noise of a slow, nearly stationary one. We give each client a memory of its own, sized to that client's noise, arrival rate, and drift, and refresh it by restarted accelerated method that starts from a common point. The tracking error is then bounded by \[ R\left(\sum_m w_m^{3/2}\,σ_m\sqrt{γ_m/λ_m}\right)^{2/3}, \] for clients with weight $w_m$, noise $σ_m$, arrival rate $λ_m$, and drift rate $γ_m$ on a domain of diameter $R$. This is never worse than what any common window can achieve, and strictly better whenever noise and drift are spread unevenly across clients. A lower bound that puts different groups of clients on different drift clocks shows this quantity is the right one. The two bounds agree up to a constant whenever the clients occupy boundedly many refresh scales, and in general differ only by the sixth root of the number of scales. Experiments show the gain comes from the memory rather than from the solver: it persists across accelerated, stochastic-gradient, and one-step methods, and against a separately tuned common-memory baseline.

math.OC↗

arXiv2026

Optimal Rates for Federated Variational Inequalities under different relaxations of bounded variance assumption

We study how interpolation affects local steps and parallel sampling in monotone variational inequalities (VIs), with $m$ workers, $K$ oracle calls per worker per round, and $N$ rounds. Under the strong growth condition (SGC), we determine the minimax rate for affine VIs, attained by local extragradient with worker averaging. For nonlinear monotone VIs and convex-concave saddle problems with Lipschitz operators satisfying SGC, we improve the stochastic terms of bounded-variance expected-gap guarantees from $(KN)^{-1/2}$ to $(KN)^{-1}$ with local steps and from $(mKN)^{-1/2}$ to $(mKN)^{-1}$ with minibatches. Under strong monotonicity, we improve the noise-dependent condition-number scaling of existing extragradient bounds from quadratic to linear. The resulting monotone-gap and strongly monotone accuracy bounds match lower bounds up to logarithmic factors in $m$. Under generalized smoothness and SGC, adaptive steps give polynomial dependence on the initial radius. For affine VIs with globally bounded oracle variance, we also attain the minimax rate: the deterministic rate over $KN$ sequential calls plus the statistical rate over all $mKN$ samples. For composite VIs with smooth convex sample interpolation and an exactly evaluated monotone Lipschitz component, accelerated mirror-prox sliding achieves an $N^{-2}$ gradient term, a $(KN)^{-1}$ monotone term, and a $(mKN)^{-1}$ sampling term, without assuming SGC.

math.OC↗

arXiv2026

Asking the Crowd the Right Question: Bias-Cancelling Weights for Federated Learning

A federated objective is a weighted sum of client risks, and the weights are almost always fixed in advance. We treat them instead as the only instrument of a wisdom-of-crowds mechanism: clients are noisy views of one truth, each seeing it through an independent distortion that is unbiased across the crowd. That the optimal weights are inversely proportional to the clients' error energies is classical; we begin at the question that answer presupposes, which energies belong there and whether a crowd can recover them from itself. Excess risk on the truth is of the exact order of the aggregate bias energy, so no optimizer can repair a bad weight vector; the truth itself is identifiable only up to a linear tilt, so subtracting estimated client biases provably reproduces uniform weighting. The expected per-client second moments, however, are exactly identified from the law of the crowd's disagreement by a well-conditioned linear inversion, a step random-effects meta-analysis cannot take because a source reports once; their realized counterparts are estimable up to an incoherence floor the algorithm can measure. This yields CROWD, which reads the disagreement off the optimization trajectory at no extra cost and matches a Bayesian minimax lower bound in the same constant: per instance as the horizon grows, and unconditionally as the prior becomes diffuse. For arbitrary distortions it stays competitive with the optimal weights, at a ratio governed by a geometric incoherence the algorithm can measure. On real scans split into sites with their own miscalibrated detectors it attains the oracle excess risk; on a companion federation that pulls bias and noise apart, weighting by noise variance is worse than not weighting at all, and CROWD is not.

math.OC↗

arXiv2026

Higher-Order vs. Quadratic Binary Optimization: Which Is Better for Probability Optimization with Tensor Sampling?

This article explores the comparative strengths of Higher-Order Unconstrained Binary Optimization (HUBO) and Quadratic Unconstrained Binary Optimization (QUBO) models in the context of probability optimization using tensor sampling techniques. HUBO can represent interactions beyond pairwise terms without introducing auxiliary variables, while QUBO benefits from a simpler algebraic structure and a wide range of established solvers. We combine a theoretical analysis of formulation expressiveness and reduction overhead with empirical experiments on synthetic polynomial objectives, RSA factorization, and Max-Cut instances. The results show that native HUBO formulations are often preferable when the original problem contains high-order interactions, because reducing such problems to QUBO increases the effective dimension seen by tensor samplers. At the same time, QUBO remains competitive for naturally quadratic problems. These findings provide practical guidance for selecting an optimization framework that balances model fidelity, dimension growth, and solver performance.

math.OC↗