arXiv subjects

Oleksii Kachaiev

Publications and source records attributed to Oleksii Kachaiev.

6 recordsLinked to original sources

arXiv2026

Markov Chain Convergence in Kernel-Induced Geometry

We study the convergence of Markov chains in the geometry induced by Hilbertian embeddings associated with positive definite kernels, also called Maximum Mean Discrepancy (MMD). Under a contraction assumption in the kernel-induced geometry, and a standard spectral gap assumption, we establish non-asymptotic, high-probability bounds for the convergence of the empirical occupation measure to the invariant measure. The obtained bounds separate the roles of the two conditions: the spectral gap determines the effective sample size in the leading term, while the kernel contraction enters only through a linear correction. As an application, we study one-step prediction of dynamical systems from a single trajectory, deriving fast learning rates for a nonlinear least-squares estimator. The rates highlight the interplay between mixing properties in the kernel-induced geometry and the classical source and capacity conditions used in the analysis of least-squares.

math.PR↗

arXiv2026

A short operator proof of Hoeffding inequalities for Markov chains

We give a short operator-theoretic proof of the sharp Hoeffding inequality for additive functionals of a Markov chain on a general state space with an $L^2(π)$ spectral gap, recovering the bound of Fan et al. (2021). We further show that the same argument yields the Hoeffding inequality of Neeman et al. (2024) for Markov-dependent random matrices. The proof rests on two key observations. First, the transition operator factors as $P=Q_λ^{1/2} K Q_λ^{1/2}$ through the León-Perron operator $Q_λ$, with $\|K\|=1$. This bounds the moment generating function by a product of one-step operator norms. Second, each one-step operator is a rank-one perturbation of a multiplication operator, so bounding its norm reduces to verifying a scalar resolvent condition. Scalar convexity and the classical Hoeffding lemma then give the desired estimate. This bypasses the asymptotic cumulant generating function, essential-spectrum analysis, and extremal two-state comparison used in existing proofs. In the matrix setting, the multi-matrix Golden-Thompson inequality first reduces the moment generating function to an operator problem on a lifted Hilbert space. The same two steps then apply: the projection onto constants becomes finite-rank, and a Schur-complement argument yields the same scalar estimate. The matrix Hoeffding inequality of Neeman et al. (2024) follows. For complex Hermitian summands, working directly on the complex Hilbert space improves the bound by a factor of $2$.

math.PR↗

arXiv2026

Learning Ergodic Dynamical Systems from a Finite Trajectory

We consider the problem of learning from a single finite trajectory of an ergodic stochastic dynamical system. More precisely, we study discrete-time autonomous stochastic systems defining time-homogeneous Markov processes. We first focus on estimating the optimal one-step prediction function by nonlinear least squares, and derive high-probability guarantees measured with respect to the invariant measure of the process. These results make explicit how the non-independent and non-identically distributed nature of trajectory data modifies the classical statistical learning analysis. We then extend the framework to higher-order systems and finite-state spaces. Finally, we show that the same least squares and concentration arguments naturally extend to learning Koopman operators. Our approach combines tools from statistical learning theory and quantitative ergodic theory for Markov chains. It relies, in particular, on a concentration inequality for Hilbert-space-valued additive functionals of uniformly geometrically ergodic Markov chains.

stat.ML↗

arXiv2026

Learning to control switching nonlinear systems with Koopman operator regression

In this work, we consider the identification and control of nonlinear systems with finite action spaces. The unknown dynamics are estimated from finite samples with Koopman operator regression in a reproducing kernel Hilbert space, yielding a linear switching predictive model, the switches governed by the value of the control variable. In order to perform control in closed-loop, the learned dynamics are employed in an infinite-horizon optimal control problem with time-varying stage cost, which is solved by means of model predictive control. In a theoretical analysis, we derive learning rates for the Koopman dynamics approximation. We further quantify, under suitable assumptions, the sub-optimality of the model predictive control strategy, both in the case of exact Koopman dynamics, and in the case of learned ones. Numerical simulations on the Duffing oscillator complement our theoretical findings.

math.OC↗

arXiv2025

SKADA-Bench: Benchmarking Unsupervised Domain Adaptation Methods with Realistic Validation On Diverse Modalities

Unsupervised Domain Adaptation (DA) consists of adapting a model trained on a labeled source domain to perform well on an unlabeled target domain with some data distribution shift. While many methods have been proposed in the literature, fair and realistic evaluation remains an open question, particularly due to methodological difficulties in selecting hyperparameters in the unsupervised setting. With SKADA-bench, we propose a framework to evaluate DA methods on diverse modalities, beyond computer vision task that have been largely explored in the literature. We present a complete and fair evaluation of existing shallow algorithms, including reweighting, mapping, and subspace alignment. Realistic hyperparameter selection is performed with nested cross-validation and various unsupervised model selection scores, on both simulated datasets with controlled shifts and real-world datasets across diverse modalities, such as images, text, biomedical, and tabular data. Our benchmark highlights the importance of realistic validation and provides practical guidance for real-life applications, with key insights into the choice and impact of model selection approaches. SKADA-bench is open-source, reproducible, and can be easily extended with novel DA methods, datasets, and model selection criteria without requiring re-evaluating competitors. SKADA-bench is available on Github at https://github.com/scikit-adaptation/skada-bench.

cs.LG↗

arXiv2024

Learning to Embed Distributions via Maximum Kernel Entropy

Empirical data can often be considered as samples from a set of probability distributions. Kernel methods have emerged as a natural approach for learning to classify these distributions. Although numerous kernels between distributions have been proposed, applying kernel methods to distribution regression tasks remains challenging, primarily because selecting a suitable kernel is not straightforward. Surprisingly, the question of learning a data-dependent distribution kernel has received little attention. In this paper, we propose a novel objective for the unsupervised learning of data-dependent distribution kernel, based on the principle of entropy maximization in the space of probability measure embeddings. We examine the theoretical properties of the latent embedding space induced by our objective, demonstrating that its geometric structure is well-suited for solving downstream discriminative tasks. Finally, we demonstrate the performance of the learned kernel across different modalities.

cs.LG↗