SearcharxivSearch

arXiv subjects

Christophe Vauthier

Publications and source records attributed to Christophe Vauthier.

4 recordsLinked to original sources

Highly Data Parallelizable Estimation of the Sliced-Wasserstein Distance Using Cumulative Distribution Functions

The Sliced Wasserstein (SW) distance has emerged as a computationally attractive alternative to the Wasserstein distance by leveraging one-dimensional optimal transport along random projections. Standard estimators of the SW distance rely on Monte Carlo averages of one-dimensional Wasserstein distances computed via quantile functions, which require sorting projected samples and access to full datasets. In this work, we introduce a new class of estimators for the Sliced Wasserstein distance based on cumulative distribution functions (CDFs) of projected measures, that avoid sorting and scale via massive dataset parallelism. This class includes several estimators, some of them being indexed by hyperparameters controlling their variance or smoothness. We show that they are especially well suited to scenarios in which CDFs are more tractable than quantile functions, such as mixtures of Gaussians, and moreover that they are also naturally compatible with federated learning, since CDFs of projected data can be computed and aggregated locally without requiring the exchange of raw samples.

stat.ML

Variational Analysis in the Wasserstein Hierarchy

Let $M$ be a complete connected Riemannian manifold. For $n \geq 0$, we endow the Wasserstein space $P^{(n)}_2(M) = P_2(\ldots P_2(M)\ldots)$, equipped with the Wasserstein distance $W_2$, with a variational structure that generalizes the standard variational structure on $P_2(M)$ provided by optimal transport theory. Our approach makes use of tools from category theory to lift the geometric structure of the manifold $M$ to the spaces $P^{(n)}_2(M)$, in order to establish in a principled way a rigorous theoretical framework for variational analysis on the space $P^{(n)}_2(M)$. In particular, we obtain a precise characterization of the constant speed geodesics of the space $P^{(n)}_2(M)$ in terms of optimal velocity plans. Moreover, we introduce a notion of gradient for functionals defined on $P^{(n)}_2(M)$, which allows us to study the differentiability and the convexity of various types of such functionals.

math.OC

Flowing Datasets with Wasserstein over Wasserstein Gradient Flows

Many applications in machine learning involve data represented as probability distributions. The emergence of such data requires radically novel techniques to design tractable gradient flows on probability distributions over this type of (infinite-dimensional) objects. For instance, being able to flow labeled datasets is a core task for applications ranging from domain adaptation to transfer learning or dataset distillation. In this setting, we propose to represent each class by the associated conditional distribution of features, and to model the dataset as a mixture distribution supported on these classes (which are themselves probability distributions), meaning that labeled datasets can be seen as probability distributions over probability distributions. We endow this space with a metric structure from optimal transport, namely the Wasserstein over Wasserstein (WoW) distance, derive a differential structure on this space, and define WoW gradient flows. The latter enables to design dynamics over this space that decrease a given objective functional. We apply our framework to transfer learning and dataset distillation tasks, leveraging our gradient flow construction as well as novel tractable functionals that take the form of Maximum Mean Discrepancies with Sliced-Wasserstein based kernels between probability distributions.

cs.LG

Towards Understanding Gradient Dynamics of the Sliced-Wasserstein Distance via Critical Point Analysis

In this paper, we investigate the properties of the Sliced Wasserstein Distance (SW) when employed as an objective functional. The SW metric has gained significant interest in the optimal transport and machine learning literature, due to its ability to capture intricate geometric properties of probability distributions while remaining computationally tractable, making it a valuable tool for various applications, including generative modeling and domain adaptation. Our study aims to provide a rigorous analysis of the critical points arising from the optimization of the SW objective. By computing explicit perturbations, we establish that stable critical points of SW cannot concentrate on segments. This stability analysis is crucial for understanding the behaviour of optimization algorithms for models trained using the SW objective. Furthermore, we investigate the properties of the SW objective, shedding light on the existence and convergence behavior of critical points. We illustrate our theoretical results through numerical experiments.

stat.ML