arXiv subjects

Vladimir Bogachev

Publications and source records attributed to Vladimir Bogachev.

6 recordsLinked to original sources

arXiv2026

Tensorion: A Tensor-Aware Generalization of the Muon Optimizer

Common first-order optimizers, such as Adam, implicitly treat each parameter block as an unstructured vector, which disregards the multilinear weight structure present in many modern machine learning models. Recent work has shown that exploiting matrix structure can improve optimization dynamics. A notable example is Muon, which performs steepest descent under the spectral norm constraint. We take the next step and introduce Tensorion, a tensor-aware optimizer that extends Muon's constrained optimization perspective from matrices to higher-order tensors. Tensorion is built around a linear minimization oracle (LMO) over a tensor norm ball. The norm is carefully chosen to balance two objectives: tightly bounding the tensor spectral norm, while still keeping the LMO tractable. This LMO becomes computable because it reduces to operations on adaptively selected unfolding matrices. Notably, when restricted to order-2 tensors (i.e., matrices), Tensorion recovers Muon exactly. Experiments on tensor-based computer vision problems suggest that Tensorion can offer improved convergence behavior and more stable gradient updates compared with Adam-based and existing tensor-aware baselines in the evaluated settings.

cs.LG↗

arXiv2025

LoRA meets Riemannion: Muon Optimizer for Parametrization-independent Low-Rank Adapters

This work presents a novel, fully Riemannian framework for Low-Rank Adaptation (LoRA) that geometrically treats low-rank adapters by optimizing them directly on the fixed-rank manifold. This formulation eliminates the parametrization ambiguity present in standard Euclidean optimizers. Our framework integrates three key components to achieve this: (1) we derive Riemannion, a new Riemannian optimizer on the fixed-rank matrix manifold that generalizes the recently proposed Muon optimizer; (2) we develop a Riemannian gradient-informed LoRA initialization, and (3) we provide an efficient implementation without prominent overhead that uses automatic differentiation to compute arising geometric operations while adhering to best practices in numerical linear algebra. Comprehensive experimental results on both LLM and diffusion model architectures demonstrate that our approach yields consistent and noticeable improvements in convergence speed and final task performance over both standard LoRA and its state-of-the-art modifications.

cs.LG↗

arXiv2021

Optimal transportation of measures with a parameter

We consider optimal transportation of measures on metric and topological spaces in the case where the cost function and marginal distributions depend on a parameter with values in a metric space. The Hausdorff distance between the sets of probability measures with given marginals is estimated via the distances between the marginals themselves. The continuity of the cost of optimal transportation with respect to the parameter is proved in the case of continuous dependence of the cost function and marginal distributions on this parameter. Existence of approximate optimal plans continuous with respect to the parameter is established. It is shown that the optimal plan is continuous with respect to the parameter in case of uniqueness. Examples are constructed when there is no continuous selection of optimal plans. Finally, a general result on convergence of Monge optimal mappings is proved.

math.FA↗

arXiv2019

Kantorovich problems and conditional measures depending on a parameter

We study measurable dependence of measures on a parameter in the following two classical problems: constructing conditional measures and the Kantorovich optimal transportation. We obtain broad sufficient conditions for the existence of conditional probabilities measurably depending on a parameter in the case of parametric families of measures and mappings. A~particular emphasis is made on the Borel measurability (which cannot be always achieved). Our second main result gives sufficient conditions for the Borel measurability of optimal transports and transportation costs with respect to a parameter in the case where marginal measures and cost functions depend on a parameter. As a corollary we obtain the Borel measurability with respect to the parameter for conditional measures of optimal plans. Finally, we show that the Skorohod parametrization of measures by mappings can be also made measurable with respect to a parameter.

math.FA↗

arXiv2010

Uniqueness for solutions of Fokker-Planck equations on infinite dimensional spaces

We develop a general technique to prove uniqueness of solutions for Fokker--Planck equations on infinite dimensional spaces. We illustrate this method by implementing it for Fokker--Planck equations in Hilbert spaces with Kolmogorov operators with irregular coefficients and both non-degenerate or degenerate second order part.

math.PR↗

arXiv2009

Existence and uniqueness of solutions for Fokker-Planck equations on Hilbert spaces

We consider a stochastic differential equation in a Hilbert space with time-dependent coefficients for which no general existence and uniqueness results are known. We prove, under suitable assumptions, existence and uniqueness of a measure valued solution, for the corresponding Fokker--Planck equation. In particular, we verify the Chapman--Kolmogorov equations and get an evolution system of transition probabilities for the stochastic dynamics informally given by the stochastic differential equation.

math.PR↗