arXiv subjects

Alexey Kroshnin

Publications and source records attributed to Alexey Kroshnin.

At least 19 recordsLinked to original sources

arXiv2026

SLDP: Semi-Local Differential Privacy for Density-Adaptive Analytics

Density-adaptive domain discretization is essential for high-utility privacy-preserving analytics but remains challenging under Local Differential Privacy (LDP) due to the privacy-budget costs associated with iterative refinement. We propose a novel framework, Semi-Local Differential Privacy (SLDP), that assigns a privacy region to each user based on local density and defines adjacency by the potential movement of a point within its privacy region. We present an interactive $(\varepsilon, \delta)$-SLDP protocol, orchestrated by an honest-but-curious server over a public channel, to estimate these regions privately. Crucially, our framework decouples the privacy cost from the number of refinement iterations, allowing for high-resolution grids without additional privacy budget cost. We experimentally demonstrate the framework's effectiveness on estimation tasks across synthetic and real-world datasets.

cs.LG↗

arXiv2025

friends.test: rank-based method for feature selection in interaction matrices

The analysis of the interaction matrix between two distinct sets is essential across diverse fields, from pharmacovigilance to transcriptomics. Not all interactions are equally informative: a marker gene associated with a few specific biological processes is more informative than a highly expressed non-specific gene associated with most observed processes. Identifying these interactions is challenging due to background connections. Furthermore, data heterogeneity across sources precludes universal identification criteria. To address this challenge, we introduce \textsf{friends.test}, a method for identifying specificity by detecting structural breaks in entity interactions. Rank-based representation of the interaction matrix ensures invariance to heterogeneous data and allows for integrating data from diverse sources. To automatically locate the boundary between specific interactions and background activity, we employ model fitting. We demonstrate the applicability of \textsf{friends.test} on the GSE112026 -- transnational data from head and neck cancer. A computationally efficient \textsf{R} implementation is available at https://github.com/favorov/friends.test.

q-bio.QM↗

arXiv2025

UCB-type Algorithm for Budget-Constrained Expert Learning

In many modern applications, a system must dynamically choose between several adaptive learning algorithms that are trained online. Examples include model selection in streaming environments, switching between trading strategies in finance, and orchestrating multiple contextual bandit or reinforcement learning agents. At each round, a learner must select one predictor among $K$ adaptive experts to make a prediction, while being able to update at most $M \le K$ of them under a fixed training budget. We address this problem in the \emph{stochastic setting} and introduce \algname{M-LCB}, a computationally efficient UCB-style meta-algorithm that provides \emph{anytime regret guarantees}. Its confidence intervals are built directly from realized losses, require no additional optimization, and seamlessly reflect the convergence properties of the underlying experts. If each expert achieves internal regret $\tilde O(T^\alpha)$, then \algname{M-LCB} ensures overall regret bounded by $\tilde O\!\Bigl(\sqrt{\tfrac{KT}{M}} \;+\; (K/M)^{1-\alpha}\,T^\alpha\Bigr)$. To our knowledge, this is the first result establishing regret guarantees when multiple adaptive experts are trained simultaneously under per-round budget constraints. We illustrate the framework with two representative cases: (i) parametric models trained online with stochastic losses, and (ii) experts that are themselves multi-armed bandit algorithms. These examples highlight how \algname{M-LCB} extends the classical bandit paradigm to the more realistic scenario of coordinating stateful, self-learning experts under limited resources.

cs.LG↗

arXiv2025

Improved Stochastic Optimization of LogSumExp

The LogSumExp function, dual to the Kullback-Leibler (KL) divergence, plays a central role in many important optimization problems, including entropy-regularized optimal transport (OT) and distributionally robust optimization (DRO). In practice, when the number of exponential terms inside the logarithm is large or infinite, optimization becomes challenging since computing the gradient requires differentiating every term. We propose a novel convexity- and smoothness-preserving approximation to LogSumExp that can be efficiently optimized using stochastic gradient methods. This approximation is rooted in a sound modification of the KL divergence in the dual, resulting in a new $f$-divergence called the Safe KL divergence. Our experiments and theoretical analysis of the LogSumExp-based stochastic optimization, arising in DRO and continuous OT, demonstrate the advantages of our approach over existing baselines.

math.OC↗

arXiv2025

On the signature of squared distance matrices of metric measure spaces

We consider the numbers of positive and negative eigenvalues of matrices of squared distances between randomly sampled i.i.d. points in a given metric measure space. These numbers and their limits, as the number of points grows, in fact contain some important information about the whole space. In particular, by knowing them, we can determine whether this space can be isometrically embedded in the Hilbert space. We show that the limits of these numbers exist almost surely, are nonrandom and the same for all Borel probability measures of full support, and, moreover, are naturally related to the operators defining the multidimensional scaling (MDS) method. We also relate them to the signature of the pseudo-Euclidean space in which the given metric space can be isometrically embedded. In addition, we provide several examples of explicit calculations or just estimates of those limits for sample metric spaces. In particular, for a large class of countable spaces (for instance, containing all graphs with bounded intrinsic metrics), we get that the number of negative eigenvalues increases to infinity as the size of samples grows. However, we are able to provide examples when the number of samples grows to infinity and the numbers of both negative and positive eigenvalues increases to infinity, or the number of positive eigenvalues is bounded (but as large as desired), and the number of positive ones is fixed. Finally, we consider the example of the universal countable Rado graph.

math.MG↗

arXiv2024

Bernstein-type and Bennett-type inequalities for unbounded matrix martingales

We derive explicit Bernstein-type and Bennett-type concentration inequalities for matrix-valued martingale processes with unbounded observations from the Hermitian space $\mathbb{H}(d)$. Specifically, we assume that the $\psi_{\alpha}$-Orlicz (quasi-)norms of their difference process are bounded for some $\alpha > 0$. Further, we generalize the obtained result by replacing the ambient dimension $d$ with the effective rank of the covariance of the observations. To illustrate the applicability of the results, we prove several corollaries, including an empirical version of Bernstein's inequality and an extension of the bounded difference inequality, also known as McDiarmid's inequality.

math.PR↗

arXiv2024

Diffusion & Adversarial Schr\"odinger Bridges via Iterative Proportional Markovian Fitting

The Iterative Markovian Fitting (IMF) procedure, which iteratively projects onto the space of Markov processes and the reciprocal class, successfully solves the Schr\"odinger Bridge (SB) problem. However, an efficient practical implementation requires a heuristic modification -- alternating between fitting forward and backward time diffusion at each iteration. This modification is crucial for stabilizing training and achieving reliable results in applications such as unpaired domain translation. Our work reveals a close connection between the modified version of IMF and the Iterative Proportional Fitting (IPF) procedure -- a foundational method for the SB problem, also known as Sinkhorn's algorithm. Specifically, we demonstrate that the heuristic modification of the IMF effectively integrates both IMF and IPF procedures. We refer to this combined approach as the Iterative Proportional Markovian Fitting (IPMF) procedure. Through theoretical and empirical analysis, we establish the convergence of the IPMF procedure under various settings, contributing to developing a unified framework for solving SB problems. Moreover, from a practical standpoint, the IPMF procedure enables a flexible trade-off between image similarity and generation quality, offering a new mechanism for tailoring models to specific tasks.

cs.LG↗

arXiv2023

Primal-Dual Gradient Methods for Searching Network Equilibria in Combined Models with Nested Choice Structure and Capacity Constraints

We consider a network equilibrium model (i.e. a combined model), which was proposed as an alternative to the classic four-step approach for travel forecasting in transportation networks. This model can be formulated as a convex minimization program. We extend the combined model to the case of the stable dynamics (SD) model in the traffic assignment stage, which imposes strict capacity constraints in the network. We propose a way to solve corresponding dual optimization problems with accelerated gradient methods and give theoretical guarantees of their convergence. We conducted numerical experiments with considered optimization methods on Moscow and Berlin networks.

math.OC↗

arXiv2022

Infinite multidimensional scaling for metric measure spaces

For a given metric measure space $(X,d,μ)$ we consider finite samples of points, calculate the matrix of distances between them and then reconstruct the points in some finite-dimensional space using the multidimensional scaling (MDS) algorithm with this distance matrix as an input. We show that this procedure gives a natural limit as the number of points in the samples grows to infinity and the density of points approaches the measure $μ$. This limit can be viewed as "infinite MDS" embedding of the original space, now not anymore into a finite-dimensional space but rather into an infinitedimensional Hilbert space. We further show that this embedding is stable with respect to the natural convergence of metric measure spaces. However, contrary to what is usually believed in applications, we show that in many cases it does not preserve distances, nor is even bi-Lipschitz, but may provide snowflake (Assouad-type) embeddings of the original space to a Hilbert space (this is, for instance, the case of a sphere and a flat torus equipped with their geodesic distances).

math.MG↗

arXiv2021

Generalized bootstrap in the Bures-Wasserstein space

This study proposes a bootstrap-based method for uncertainty quantification in two important statistical scenarios. First, we approximate the sampling distribution of empirical barycenters under the Bures--Wasserstein metric using a reweighted estimator. Our theoretical results guarantee the accuracy of this approximation and enable the construction of data-driven confidence sets. The methodology is validated through experiments on graph-structured data, including stochastic block models and brain connectomes. Additionally, we compare bootstrap-based confidence sets with the asymptotic confidence sets obtained in arXiv:1901.00226v2, evaluating both their statistical performance and computational complexity. Second, we investigate the generalized bootstrap framework for $M$-estimators without requiring a specific resampling scheme, thus covering both weighted and resampling methods under mild conditions. Both contributions rely on a novel Gaussian approximation result for $M$-estimators.

math.ST↗

arXiv2021

On the properties of some low-parameter models for color reproduction in terms of spectrum transformations and coverage of a color triangle

One of the classical approaches to solving color reproduction problems, such as color adaptation or color space transform, is the use of low-parameter spectral models. The strength of this approach is the ability to choose a set of properties that the model should have, be it a large coverage area of a color triangle, an accurate description of the addition or multiplication of spectra, knowing only the tristimulus corresponding to them. The disadvantage is that some of the properties of the mentioned spectral models are confirmed only experimentally. This work is devoted to the theoretical substantiation of various properties of spectral models. In particular, we prove that the banded model is the only model that simultaneously possesses the properties of closure under addition and multiplication. We also show that the Gaussian model is the limiting case of the von Mises model and prove that the set of protomers of the von Mises model unambiguously covers the color triangle in both the case of convex and non-convex spectral locus.

cs.CV↗

arXiv2020

Entropic-Wasserstein barycenters: PDE characterization, regularity and CLT

In this paper, we investigate properties of entropy-penalized Wasserstein barycenters introduced by Bigot, Cazelles and Papadakis (2019) as a regularization of Wasserstein barycenters first presented by Agueh and Carlier (2011). After characterizing these barycenters in terms of a system of Monge-Amp\`ere equations, we prove some global moment and Sobolev bounds as well as higher regularity properties. We finally establish a central limit theorem for entropic-Wasserstein barycenters.

math.AP↗

arXiv2020

Robust $k$-means Clustering for Distributions with Two Moments

We consider the robust algorithms for the $k$-means clustering problem where a quantizer is constructed based on $N$ independent observations. Our main results are median of means based non-asymptotic excess distortion bounds that hold under the two bounded moments assumption in a general separable Hilbert space. In particular, our results extend the renowned asymptotic result of Pollard, 1981 who showed that the existence of two moments is sufficient for strong consistency of an empirically optimal quantizer in $\mathbb{R}^d$. In a special case of clustering in $\mathbb{R}^d$, under two bounded moments, we prove matching (up to constant factors) non-asymptotic upper and lower bounds on the excess distortion, which depend on the probability mass of the lightest cluster of an optimal quantizer. Our bounds have the sub-Gaussian form, and the proofs are based on the versions of uniform bounds for robust mean estimators.

math.ST↗

arXiv2020

Fréchet barycenters in the Monge-Kantorovich spaces

We consider the space $\mathcal{P}(X)$ of probability measures on arbitrary Radon space $X$ endowed with a transportation cost $J(μ, ν)$ generated by a nonnegative continuous cost function. For a probability distribution on $\mathcal{P}(X)$ we formulate a notion of average with respect to this transportation cost, called here the Fréchet barycenter, prove a version of the law of large numbers for Fréchet barycenters, and discuss the structure of $\mathcal{P}(X)$ related to the transportation cost $J$.

math.PR↗

arXiv2020

On the Complexity of Approximating Wasserstein Barycenter

We study the complexity of approximating Wassertein barycenter of $m$ discrete measures, or histograms of size $n$ by contrasting two alternative approaches, both using entropic regularization. The first approach is based on the Iterative Bregman Projections (IBP) algorithm for which our novel analysis gives a complexity bound proportional to $\frac{mn^2}{\varepsilon^2}$ to approximate the original non-regularized barycenter. Using an alternative accelerated-gradient-descent-based approach, we obtain a complexity proportional to $\frac{mn^{2.5}}{\varepsilon} $. As a byproduct, we show that the regularization parameter in both approaches has to be proportional to $\varepsilon$, which causes instability of both algorithms when the desired accuracy is high. To overcome this issue, we propose a novel proximal-IBP algorithm, which can be seen as a proximal gradient method, which uses IBP on each iteration to make a proximal step. We also consider the question of scalability of these algorithms using approaches from distributed optimization and show that the first algorithm can be implemented in a centralized distributed setting (master/slave), while the second one is amenable to a more general decentralized distributed setting with an arbitrary network topology.

math.OC↗

arXiv2020

Inexact Model: A Framework for Optimization and Variational Inequalities

In this paper we propose a general algorithmic framework for first-order methods in optimization in a broad sense, including minimization problems, saddle-point problems and variational inequalities. This framework allows to obtain many known methods as a special case, the list including accelerated gradient method, composite optimization methods, level-set methods, proximal methods. The idea of the framework is based on constructing an inexact model of the main problem component, i.e. objective function in optimization or operator in variational inequalities. Besides reproducing known results, our framework allows to construct new methods, which we illustrate by constructing a universal method for variational inequalities with composite structure. This method works for smooth and non-smooth problems with optimal complexity without a priori knowledge of the problem smoothness. We also generalize our framework for strongly convex objectives and strongly monotone variational inequalities.

math.OC↗

arXiv2019

Gradient Methods for Problems with Inexact Model of the Objective

We consider optimization methods for convex minimization problems under inexact information on the objective function. We introduce inexact model of the objective, which as a particular cases includes $(δ,L)$ inexact oracle and relative smoothness condition. We analyze gradient method which uses this inexact model and obtain convergence rates for convex and strongly convex problems. To show potential applications of our general framework we consider three particular problems. The first one is clustering by electorial model introduced in [Nesterov, 2018]. The second one is approximating optimal transport distance, for which we propose a Proximal Sinkhorn algorithm. The third one is devoted to approximating optimal transport barycenter and we propose a Proximal Iterative Bregman Projections algorithm. We also illustrate the practical performance of our algorithms by numerical experiments.

math.OC↗

arXiv2019

Statistical inference for Bures-Wasserstein barycenters

In this work we introduce the concept of Bures-Wasserstein barycenter $Q_*$, that is essentially a Fréchet mean of some distribution $\mathbb{P}$ supported on a subspace of positive semi-definite Hermitian operators $\mathbb{H}_{+}(d)$. We allow a barycenter to be restricted to some affine subspace of $\mathbb{H}_{+}(d)$ and provide conditions ensuring its existence and uniqueness. We also investigate convergence and concentration properties of an empirical counterpart of $Q_*$ in both Frobenius norm and Bures-Wasserstein distance, and explain, how obtained results are connected to optimal transportation theory and can be applied to statistical inference in quantum mechanics.

math.ST↗