arXiv subjects

Viacheslav Borovitskiy

Publications and source records attributed to Viacheslav Borovitskiy.

25 records · Page 2Linked to original sources

arXiv2022

On power sum kernels on symmetric groups

In this note, we introduce a family of "power sum" kernels and the corresponding Gaussian processes on symmetric groups $\mathrm{S}_n$. Such processes are bi-invariant: the action of $\mathrm{S}_n$ on itself from both sides does not change their finite-dimensional distributions. We show that the values of power sum kernels can be efficiently calculated, and we also propose a method enabling approximate sampling of the corresponding Gaussian processes with polynomial computational complexity. By doing this we provide the tools that are required to use the introduced family of kernels and the respective processes for statistical modeling and machine learning.

stat.ME↗

arXiv2022

Bellman function method for general operators on martingales

It is shown that the Bellman function method can be applied to study the $L^p$-norms of general operators on martingales, i.e., of operators that are not necessarily martingale transforms. Informally, we provide a single Bellman-type function that "encodes" the $L^p$-boundedness of "almost all" operators from Gundy's extrapolation theorem. As examples of such operators, we consider the Haar transforms and the operator whose $L^p$-boundedness underlies Rubio de Francia's inequality for the Walsh system.

math.FA↗

arXiv2022

Quadric Hypersurface Intersection for Manifold Learning in Feature Space

The knowledge that data lies close to a particular submanifold of the ambient Euclidean space may be useful in a number of ways. For instance, one may want to automatically mark any point far away from the submanifold as an outlier or to use the geometry to come up with a better distance metric. Manifold learning problems are often posed in a very high dimension, e.g. for spaces of images or spaces of words. Today, with deep representation learning on the rise in areas such as computer vision and natural language processing, many problems of this kind may be transformed into problems of moderately high dimension, typically of the order of hundreds. Motivated by this, we propose a manifold learning technique suitable for moderately high dimension and large datasets. The manifold is learned from the training data in the form of an intersection of quadric hypersurfaces -- simple but expressive objects. At test time, this manifold can be used to introduce a computationally efficient outlier score for arbitrary new data points and to improve a given similarity metric by incorporating the learned geometric structure into it.

cs.LG↗

arXiv2021

Vector-valued Gaussian Processes on Riemannian Manifolds via Gauge Independent Projected Kernels

Gaussian processes are machine learning models capable of learning unknown functions in a way that represents uncertainty, thereby facilitating construction of optimal decision-making systems. Motivated by a desire to deploy Gaussian processes in novel areas of science, a rapidly-growing line of research has focused on constructively extending these models to handle non-Euclidean domains, including Riemannian manifolds, such as spheres and tori. We propose techniques that generalize this class to model vector fields on Riemannian manifolds, which are important in a number of application areas in the physical sciences. To do so, we present a general recipe for constructing gauge independent kernels, which induce Gaussian vector fields, i.e. vector-valued Gaussian processes coherent with geometry, from scalar-valued Riemannian kernels. We extend standard Gaussian process training methods, such as variational inference, to this setting. This enables vector-valued Gaussian processes on Riemannian manifolds to be trained using standard methods and makes them accessible to machine learning practitioners.

stat.ML↗

arXiv2021

Littlewood-Paley-Rubio de Francia Inequality for the Two-parameter Walsh System

A version of Littlewood-Paley-Rubio de Francia inequality for the two-parameter Walsh system is proved: for any family of disjoint rectangles $I_k = I_k^1 \times I_k^2$ in ${\mathbb{Z}_+ \times \mathbb{Z}_+}$ and a family of functions $f_k$ with Walsh spectrum inside $I_k$ the following is true \[\left\lVert\sum\limits_k f_k\right\rVert_{L^p} \leq C_p \left\lVert\left(\sum\limits_k \left|f_k\right|^2\right)^{1/2}\right\rVert_{L^p}, \qquad 1 < p \leq 2,\] where $C_p$ does not depend on the choice of rectangles $\left\{I_k\right\}$ or functions $\left\{f_k\right\}$. The arguments are based on the atomic theory of two-parameter martingale Hardy spaces. In the course of the proof, a two-parameter version of Gundy's theorem on the boundedness of operators taking martingales to measurable functions is formulated, which might be of independent interest.

math.FA↗

arXiv2021

Pathwise Conditioning of Gaussian Processes

As Gaussian processes are used to answer increasingly complex questions, analytic solutions become scarcer and scarcer. Monte Carlo methods act as a convenient bridge for connecting intractable mathematical expressions with actionable estimates via sampling. Conventional approaches for simulating Gaussian process posteriors view samples as draws from marginal distributions of process values at finite sets of input locations. This distribution-centric characterization leads to generative strategies that scale cubically in the size of the desired random vector. These methods are prohibitively expensive in cases where we would, ideally, like to draw high-dimensional vectors or even continuous sample paths. In this work, we investigate a different line of reasoning: rather than focusing on distributions, we articulate Gaussian conditionals at the level of random variables. We show how this pathwise interpretation of conditioning gives rise to a general family of approximations that lend themselves to efficiently sampling Gaussian process posteriors. Starting from first principles, we derive these methods and analyze the approximation errors they introduce. We, then, ground these results by exploring the practical implications of pathwise conditioning in various applied settings, such as global optimization and reinforcement learning.

stat.ML↗

arXiv2020

Efficiently Sampling Functions from Gaussian Process Posteriors

Gaussian processes are the gold standard for many real-world modeling problems, especially in cases where a model's success hinges upon its ability to faithfully represent predictive uncertainty. These problems typically exist as parts of larger frameworks, wherein quantities of interest are ultimately defined by integrating over posterior distributions. These quantities are frequently intractable, motivating the use of Monte Carlo methods. Despite substantial progress in scaling up Gaussian processes to large training sets, methods for accurately generating draws from their posterior distributions still scale cubically in the number of test locations. We identify a decomposition of Gaussian processes that naturally lends itself to scalable sampling by separating out the prior from the data. Building off of this factorization, we propose an easy-to-use and general-purpose approach for fast posterior sampling, which seamlessly pairs with sparse approximations to afford scalability both during training and at test time. In a series of experiments designed to test competing sampling schemes' statistical properties and practical ramifications, we demonstrate how decoupled sample paths accurately represent Gaussian process posteriors at a fraction of the usual cost.

stat.ML↗