arXiv subjects

François-Xavier Vialard

Publications and source records attributed to François-Xavier Vialard.

At least 19 recordsLinked to original sources

arXiv2026

Gromov-Wasserstein at Scale, Beyond Squared Norms

A fundamental challenge in data science is to match disparate point sets with each other. While optimal transport efficiently minimizes point displacements under a bijectivity constraint, it is inherently sensitive to rotations. Conversely, minimizing distortions via the Gromov-Wasserstein (GW) framework addresses this limitation but introduces a non-convex, computationally demanding optimization problem. In this work, we identify a broad class of distortion penalties that reduce to a simple alignment problem within a lifted feature space. Leveraging this insight, we introduce an iterative GW solver with a linear memory footprint and quadratic (rather than cubic) time complexity. Our method is differentiable, comes with strong theoretical guarantees, and scales to hundreds of thousands of points in minutes. This efficiency unlocks a wide range of geometric applications and enables the exploration of the GW energy landscape, whose local minima encode the symmetries of the matching problem.

cs.CG↗

arXiv2026

A Unified Dual Method for Matching Problems

Matching problems are ubiquitous in data science as they enable the alignment of structured objects and distributions. While existing solvers are often tailored to specific matching formulations, we unify a broad class of such problems within a common mathematical and optimization framework based on duality theory. Theoretically, we demonstrate that matching objectives decomposable as a difference of convex (DC) functions can be recast as implicit registration problems. This connection links matching to another well-studied class of objectives and yields a dual formulation amenable to natural optimization strategies. We then apply these findings to quadratic matching (QM) problems, which admit DC decompositions and for which we provide extensive convergence guarantees. Our framework applies to Gromov-Wasserstein (GW), as well as its unbalanced formulation and several variants, which are increasingly popular QM problems. Numerically, we implement our algorithms at scale for various data modalities such as graphs, point clouds, meshes, and word embeddings. Finally, our modular approach allows us to explore new formulations such as fracture matching, broadening the scope of problems that can be addressed within this framework..

cs.CG↗

arXiv2026

Gauss-Newton drifting extends natural gradient descent

Drifting methods have recently been introduced as a promising new paradigm for training generative models, and have attracted a lot of attention so far. In this work, we focus on how the optimization procedure shapes their convergence properties, shedding light on the underlying machinery that may explain their empirical success. More specifically, when the drift field is a Wasserstein gradient, optimizing the drifting loss via plain Euclidean gradient descent seems to forfeit any hope of exploiting convexity and therefore of obtaining any convergence guarantees. Changing either the drift field or the optimization procedure, however, may lead to different dynamics: we show that keeping the Wasserstein gradient as drift field but replacing gradient descent with a Gauss-Newton scheme amounts to performing a natural gradient descent in parameter space. This connection allows us to provide global convergence guarantees for the optimization procedure under suitable convexity assumptions.

math.OC↗

arXiv2026

Token Sample Complexity of Attention

As context windows in large language models continue to expand, it is essential to characterize how attention behaves at extreme sequence lengths. We introduce token sample complexity: the rate at which attention computed on $n$ tokens converges to its infinite-token limit. We estimate finite-$n$ convergence bounds at two levels: pointwise uniform convergence of the attention map, and convergence of moments for the transformed token distribution. For compactly supported (and more generally sub-Gaussian) distributions, our first result shows that the attention map converges uniformly on a ball of radius $R$ at rate $C(R)/\sqrt{n}$, where $C(R)$ grows exponentially with $R$. For large $R$, this estimate loses practical value, and our second result addresses this issue by establishing convergence rates for the moments of the transformed distribution (the token output of the attention layer). In this case, the rate is $C'(R)/n^β$ with $β<\tfrac{1}{2}$, and $C'(R)$ depends polynomially on the size of the support of the distribution. The exponent $β$ depends on the attention geometry and the spectral properties of the token distribution. We also examine the regime in which the attention parameter tends to infinity and the softmax approaches a hardmax, and in this setting, we establish a logarithmic rate of convergence. Experiments on synthetic and real data support our predictions and show that the predicted slowdown is reflected in downstream accuracy.

cs.LG↗

arXiv2026

The Wasserstein--Ebin Metric: A Geometric Lift of Unbalanced Optimal Transport to the space of Riemannian metrics

We introduce dynamic and static formulations that formally extend unbalanced optimal transport from the space of positive densities to the space of Riemannian metrics. The first construction is based on a dynamic variational formulation in which the evolution of a Riemannian metric is driven by transport together with a source term. Choosing the $L^2$-metric to penalize the transport vector field and the Ebin metric to penalize the source component yields a new Riemannian metric on the manifold of Riemannian metrics, which we call the Wasserstein--Ebin metric. Our main result shows that the volume map defines a Riemannian submersion from the Wasserstein--Ebin metric to the Wasserstein--Fisher--Rao metric on the space of smooth densities. In addition, we construct a Riemannian submersion from the automorphism group of the tangent bundle onto the space of Riemannian metrics, providing a generalization of Otto's geometric description for the Wasserstein metric to the setting of the Wasserstein--Ebin metric. To propose a static formulation of unbalanced optimal Riemannian metric transport, we introduce two Kullback--Leibler-type divergences on the space of Riemannian metrics: one inspired by matrix information geometry, and another related, through the volume map, to the classical Kullback--Leibler divergence on densities. Establishing a link between the static and dynamic formulations remains an open direction for future work.

math.DG↗

arXiv2026

Learning Monge maps with constrained drifting models

We study the estimation of optimal transport (OT) maps between an arbitrary source probability measure and a log-concave target probability measure. Our contributions are twofold. First, we propose a new evolution equation in the set of transport maps. It can be seen as the gradient flow of a lift of some user-chosen divergence (e.g., the KL divergence, or relative entropy) to the space of transport maps, constrained to the convex set of optimal transport maps. We prove the existence of long-time solutions to this flow as well as its convergence toward the OT map as time goes to infinity, under standard convexity conditions on the divergence. Second, we study the practical implementation of this constrained gradient flow. We propose two time-discrete computational schemes-one explicit, one implicit-, and we prove the convergence of the latter to the OT map as time goes to infinity. We then parameterize the OT maps with convexity-constrained neural networks and train them with these discretizations of the constrained gradient flow. We show that this is equivalent to performing a natural gradient descent of the lift of the chosen divergence in the neural networks' parameter space, similarly to drifting generative models. Empirically, our scheme outperforms the standard Euclidean gradient descent methods used to train convexity-constrained neural networks in terms of approximation results for the OT map and convergence stability, and it still yields better results than the same approach combined with the widely used Adam optimizer.

math.OC↗

arXiv2026

Sliced ReLU attention: Quasi-linear contextual expressivity via sorting

We introduce sliced ReLU attention, a new attention mechanism that departs structurally from both softmax and its approximation alternatives. Instead of applying a nonlinearity to pairwise dot products, we operate on one-dimensional projections of key--query differences and leverage sorting to obtain quasi-linear complexity. This construction yields a differentiable, non-symmetric kernel that can be computed in O(n log(n)) through a sorting procedure, making it suitable for very long contexts. Beyond computational benefits, the model retains strong theoretical expressive power: we establish two in-context expressivity results, previously known for softmax attention, showing that sliced ReLU attention preserves the ability to perform nontrivial sequence-to-sequence disentangling tasks and satisfies a contextual universal approximation property. Finally, we illustrate the potential practical interest of this kernel in small to medium-scale experiments.

cs.LG↗

arXiv2025

Nonnegative cross-curvature in infinite dimensions: synthetic definition and spaces of measures

Nonnegative cross-curvature (NNCC) is a geometric property of a cost function defined on a product space that originates in optimal transportation and the Ma-Trudinger-Wang theory. Motivated by applications in optimization, gradient flows and mechanism design, we propose a variational formulation of nonnegative cross-curvature on c-convex domains applicable to infinite dimensions and nonsmooth settings. The resulting class of NNCC spaces is closed under Gromov-Hausdorff convergence and for this class, we extend many properties of classical nonnegative cross-curvature: stability under generalized Riemannian submersions, characterization in terms of the convexity of certain sets of c-concave functions, and in the metric case, it is a subclass of positively curved spaces in the sense of Alexandrov. One of our main results is that Wasserstein spaces of probability measures inherit the NNCC property from their base space. Additional examples of NNCC costs include the Bures-Wasserstein and Fisher-Rao squared distances, the Hellinger-Kantorovich squared distance (in some cases), the relative entropy on probability measures, and the 2-Gromov-Wasserstein squared distance on metric measure spaces.

math.MG↗

arXiv2025

Decreasing Entropic Regularization Averaged Gradient for Semi-Discrete Optimal Transport

Adding entropic regularization to Optimal Transport (OT) problems has become a standard approach for designing efficient and scalable solvers. However, regularization introduces a bias from the true solution. To mitigate this bias while still benefiting from the acceleration provided by regularization, a natural solver would adaptively decrease the regularization as it approaches the solution. Although some algorithms heuristically implement this idea, their theoretical guarantees and the extent of their acceleration compared to using a fixed regularization remain largely open. In the setting of semi-discrete OT, where the source measure is continuous and the target is discrete, we prove that decreasing the regularization can indeed accelerate convergence. To this end, we introduce DRAG: Decreasing (entropic) Regularization Averaged Gradient, a stochastic gradient descent algorithm where the regularization decreases with the number of optimization steps. We provide a theoretical analysis showing that DRAG benefits from decreasing regularization compared to a fixed scheme, achieving an unbiased $\mathcal{O}(1/t)$ sample and iteration complexity for both the OT cost and the potential estimation, and a $\mathcal{O}(1/\sqrt{t})$ rate for the OT map. Our theoretical findings are supported by numerical experiments that validate the effectiveness of DRAG and highlight its practical advantages.

stat.ML↗

arXiv2025

Stochastic Optimization in Semi-Discrete Optimal Transport: Convergence Analysis and Minimax Rate

We investigate the semi-discrete Optimal Transport (OT) problem, where a continuous source measure $μ$ is transported to a discrete target measure $ν$, with particular attention to the OT map approximation. In this setting, Stochastic Gradient Descent (SGD) based solvers have demonstrated strong empirical performance in recent machine learning applications, yet their theoretical guarantee to approximate the OT map is an open question. In this work, we answer it positively by providing both computational and statistical convergence guarantees of SGD. Specifically, we show that SGD methods can estimate the OT map with a minimax convergence rate of $\mathcal{O}(1/\sqrt{n})$, where $n$ is the number of samples drawn from $μ$. To establish this result, we study the averaged projected SGD algorithm, and identify a suitable projection set that contains a minimizer of the objective, even when the source measure is not compactly supported. Our analysis holds under mild assumptions on the source measure and applies to MTW cost functions,whic include $\|\cdot\|^p$ for $p \in (1, \infty)$. We finally provide numerical evidence for our theoretical results.

math.ST↗

arXiv2025

Understanding the training of infinitely deep and wide ResNets with Conditional Optimal Transport

We study the convergence of gradient flow for the training of deep neural networks. If Residual Neural Networks are a popular example of very deep architectures, their training constitutes a challenging optimization problem due notably to the non-convexity and the non-coercivity of the objective. Yet, in applications, those tasks are successfully solved by simple optimization algorithms such as gradient descent. To better understand this phenomenon, we focus here on a ``mean-field'' model of infinitely deep and arbitrarily wide ResNet, parameterized by probability measures over the product set of layers and parameters and with constant marginal on the set of layers. Indeed, in the case of shallow neural networks, mean field models have proven to benefit from simplified loss-landscapes and good theoretical guarantees when trained with gradient flow for the Wasserstein metric on the set of probability measures. Motivated by this approach, we propose to train our model with gradient flow w.r.t. the conditional Optimal Transport distance: a restriction of the classical Wasserstein distance which enforces our marginal condition. Relying on the theory of gradient flows in metric spaces we first show the well-posedness of the gradient flow equation and its consistency with the training of ResNets at finite width. Performing a local Polyak-Łojasiewicz analysis, we then show convergence of the gradient flow for well-chosen initializations: if the number of features is finite but sufficiently large and the risk is sufficiently small at initialization, the gradient flow converges towards a global minimizer. This is the first result of this type for infinitely deep and arbitrarily wide ResNets.

cs.LG↗

arXiv2025

Ultra-fast feature learning for the training of two-layer neural networks in the two-timescale regime

We study the convergence of gradient methods for the training of mean-field single-hidden-layer neural networks with square loss. For this high-dimensional and non-convex optimization problem, most known convergence results are either qualitative or rely on a neural tangent kernel analysis where nonlinear representations of the data are fixed. Using that this problem belongs to the class of separable nonlinear least squares problems, we consider here a Variable Projection (VarPro) or two-timescale learning algorithm, thereby eliminating the linear variables and reducing the learning problem to the training of nonlinear features. In a teacher-student scenario, we show such a strategy enables provable convergence rates for the sampling of a teacher feature distribution. Precisely, in the limit where the regularization strength vanishes, we show that the dynamic of the feature distribution corresponds to a weighted ultra-fast diffusion equation. Recent results on the asymptotic behavior of such PDEs then give quantitative guarantees for the convergence of the learned feature distribution.

cs.LG↗

arXiv2025

Fast Large Deformation Matching with the Energy Distance Kernel

We propose an efficient framework for point cloud and measure registration using bi-Lipschitz homeomorphisms, achieving O(n log n) complexity, where n is the number of points. By leveraging the Energy-Distance (ED) kernel, which can be approximated by its sliced one-dimensional projections, each computable in O(n log n), our method avoids hyperparameter tuning and enables efficient large-scale optimization. The main issue to be solved is the lack of regularity of the ED kernel. To this goal, we introduce two models that regularize the deformations and retain a low computational footprint. The first model relies on TV regularization, while the second model avoids the non-smooth TV regularization at the cost of restricting its use to the space of measures, or cloud of points. Last, we demonstrate the numerical robustness and scalability of our models on synthetic and real data.

math.OC↗

arXiv2024

Entropic Semi-Martingale Optimal Transport

Entropic Optimal Transport (EOT), also referred to as the Schrödinger problem, seeks to find a random processes with prescribed initial/final marginals and with minimal relative entropy with respect to a reference measure. The relative entropy forces the two measures to share the same support and only the drift of the controlled process can be adjusted, the diffusion being imposed by the reference measure. Therefore, at first sight, Semi-Martingale Optimal Transport (SMOT) problems (see [1]) seem out of the scope of applications of Entropic regularization techniques, which are otherwise very attractive from a computational point of view. However, when the process is observed only at discrete times, and become therefore a Markov chain, its relative entropy can remain finite even with variable diffusion coefficients, and discrete semi-martingales can be obtained as solutions of (multi-marginal) EOT problems.Given a (smooth) semi-martingale, the limit of the relative entropy of its time discretizations, scaled by the time step converges to the so-called ``specific relative entropy'', a convex functional of its variance process, similar to those used in SMOT.In this paper we use this observation to build an entropic time discretization of continuous SMOT problems. This allows to compute discrete approximations of solutions to continuous SMOT problems by a multi-marginal Sinkhorn algorithm, without the need of solving the non-linear Hamilton-Jacobi-Bellman pde's associated to the dual problem, as done for example in [1, 2]. We prove a convergence result of the time discrete entropic problem to the continuous time problem, we propose an implementation and provide numerical experiments supporting the theoretical convergence.

math.OC↗

arXiv2024

Semi-Discrete Optimal Transport: Nearly Minimax Estimation With Stochastic Gradient Descent and Adaptive Entropic Regularization

Optimal Transport (OT) based distances are powerful tools for machine learning to compare probability measures and manipulate them using OT maps. In this field, a setting of interest is semi-discrete OT, where the source measure $μ$ is continuous, while the target $ν$ is discrete. Recent works have shown that the minimax rate for the OT map is $\mathcal{O}(t^{-1/2})$ when using $t$ i.i.d. subsamples from each measure (two-sample setting). An open question is whether a better convergence rate can be achieved when the full information of the discrete measure $ν$ is known (one-sample setting). In this work, we answer positively to this question by (i) proving an $\mathcal{O}(t^{-1})$ lower bound rate for the OT map, using the similarity between Laguerre cells estimation and density support estimation, and (ii) proposing a Stochastic Gradient Descent (SGD) algorithm with adaptive entropic regularization and averaging acceleration. To nearly achieve the desired fast rate, characteristic of non-regular parametric problems, we design an entropic regularization scheme decreasing with the number of samples. Another key step in our algorithm consists of using a projection step that permits to leverage the local strong convexity of the regularized OT problem. Our convergence analysis integrates online convex optimization and stochastic gradient techniques, complemented by the specificities of the OT semi-dual. Moreover, while being as computationally and memory efficient as vanilla SGD, our algorithm achieves the unusual fast rates of our theory in numerical experiments.

stat.ML↗

arXiv2024

On the existence of Monge maps for the Gromov-Wasserstein problem

The Gromov--Wasserstein problem is a non-convex optimization problem over the polytope of transportation plans between two probability measures supported on two spaces, each equipped with a cost function evaluating similarities between points. Akin to the standard optimal transportation problem, it is natural to ask for conditions guaranteeing some structure on the optimizers, for instance if these are induced by a (Monge) map. We study this question in Euclidean spaces when the cost functions are either given by (i) inner products or (ii) squared distances, two standard choices in the literature. We establish the existence of an optimal map in case (i) and of an optimal 2-map (the union of the graphs of two maps) in case (ii), both under an absolute continuity condition on the source measure. Additionally, in case (ii) and in dimension one, we numerically design situations where optimizers of the Gromov--Wasserstein problem are 2-maps but are not maps. This suggests that our result cannot be improved in general for this cost. Still in dimension one, we additionally establish the optimality of monotone maps under some conditions on the measures, thereby giving insight on why such maps often appear to be optimal in numerical experiments.

math.OC↗

arXiv2024

Regularity theory and geometry of unbalanced optimal transport

Using the dual formulation only, we show that the regularity of unbalanced optimal transport also called entropy-transport inherits from the regularity of standard optimal transport. We provide detailed examples of Riemannian manifolds and costs for which unbalanced optimal transport is regular.Among all entropy-transport formulations, Wasserstein-Fisher-Rao (WFR) metric, also called Hellinger-Kantorovich, stands out since it admits a dynamic formulation, which extends the Benamou-Brenier formulation of optimal transport. After demonstrating the equivalence between dynamic and static formulations on a closed Riemannian manifold, we prove a polar factorization theorem, similar to the one due to Brenier and Mc-Cann. As a byproduct, we formulate the Monge-Amp{è}re equation associated with WFR metric, which also holds for more general costs. Last, we study the link between c-convex functions for the cost induced by the WFR metric and the cost on the cone. The main result is that the weak Ma-Trudinger-Wang condition on the cone implies the same condition on the manifold for the cost induced by WFR.

math.OC↗

arXiv2024

On the global convergence of Wasserstein gradient flow of the Coulomb discrepancy

In this work, we study the Wasserstein gradient flow of the Riesz energy defined on the space of probability measures. The Riesz kernels define a quadratic functional on the space of measure which is not in general geodesically convex in the Wasserstein geometry, therefore one cannot conclude to global convergence of the Wasserstein gradient flow using standard arguments. Our main result is the exponential convergence of the flow to the minimizer on a closed Riemannian manifold under the condition that the logarithm of the source and target measures are H{ö}lder continuous. To this goal, we first prove that the Polyak-Lojasiewicz inequality is satisfied for sufficiently regular solutions. The key regularity result is the global in-time existence of H{ö}lder solutions if the initial and target data are H{ö}lder continuous, proven either in Euclidean space or on a closed Riemannian manifold. For general measures, we prove using flow interchange techniques that there is no local minima other than the global one for the Coulomb kernel. In fact, we prove that a Lagrangian critical point of the functional for the Coulomb (or Energy distance) kernel is equal to the target everywhere except on singular sets with empty interior. In addition, singular enough measures cannot be critical points.

math.AP↗