Searcharxiv⌕ Search

arXiv subjects

Marco Cuturi

Publications and source records attributed to Marco Cuturi.

117 records · Page 7Linked to original sources
arXiv2012

Positivity and Transportation

We prove in this paper that the weighted volume of the set of integral transportation matrices between two integral histograms r and c of equal sum is a positive definite kernel of r and c when the set of considered weights forms a positive definite matrix. The computation of this quantity, despite being the subject of a significant research effort in algebraic statistics, remains an intractable challenge for histograms of even modest dimensions. We propose an alternative kernel which, rather than considering all matrices of the transportation polytope, only focuses on a sub-sample of its vertices known as its Northwestern corner solutions. The resulting kernel is positive definite and can be computed with a number of operations O(R^2d) that grows linearly in the complexity of the dimension d, where R^2, the total amount of sampled vertices, is a parameter that controls the complexity of the kernel.

stat.ML↗
arXiv2011

Ground Metric Learning

Transportation distances have been used for more than a decade now in machine learning to compare histograms of features. They have one parameter: the ground metric, which can be any metric between the features themselves. As is the case for all parameterized distances, transportation distances can only prove useful in practice when this parameter is carefully chosen. To date, the only option available to practitioners to set the ground metric parameter was to rely on a priori knowledge of the features, which limited considerably the scope of application of transportation distances. We propose to lift this limitation and consider instead algorithms that can learn the ground metric using only a training set of labeled histograms. We call this approach ground metric learning. We formulate the problem of learning the ground metric as the minimization of the difference of two polyhedral convex functions over a convex set of distance matrices. We follow the presentation of our algorithms with promising experimental results on binary classification tasks using GIST descriptors of images taken in the Caltech-256 set.

stat.ML↗
arXiv2011

Autoregressive Kernels For Time Series

We propose in this work a new family of kernels for variable-length time series. Our work builds upon the vector autoregressive (VAR) model for multivariate stochastic processes: given a multivariate time series x, we consider the likelihood function p_θ(x) of different parameters θin the VAR model as features to describe x. To compare two time series x and x', we form the product of their features p_θ(x) p_θ(x') which is integrated out w.r.t θusing a matrix normal-inverse Wishart prior. Among other properties, this kernel can be easily computed when the dimension d of the time series is much larger than the lengths of the considered time series x and x'. It can also be generalized to time series taking values in arbitrary state spaces, as long as the state space itself is endowed with a kernel κ. In that case, the kernel between x and x' is a a function of the Gram matrices produced by κon observations and subsequences of observations enumerated in x and x'. We describe a computationally efficient implementation of this generalization that uses low-rank matrix factorization techniques. These kernels are compared to other known kernels using a set of benchmark classification tasks carried out with support vector machines.

stat.ML↗
arXiv2009

Positive Definite Kernels in Machine Learning

This survey is an introduction to positive definite kernels and the set of methods they have inspired in the machine learning literature, namely kernel methods. We first discuss some properties of positive definite kernels as well as reproducing kernel Hibert spaces, the natural extension of the set of functions $\{k(x,\cdot),x\in\mathcal{X}\}$ associated with a kernel $k$ defined on a space $\mathcal{X}$. We discuss at length the construction of kernel functions that take advantage of well-known statistical models. We provide an overview of numerous data-analysis methods which take advantage of reproducing kernel Hilbert spaces and discuss the idea of combining several kernels to improve the performance on certain tasks. We also provide a short cookbook of different kernels which are particularly useful for certain data-types such as images, graphs or speech segments.

stat.ML↗
arXiv2009

Kernels for Measures Defined on the Gram Matrix of their Support

We present in this work a new family of kernels to compare positive measures on arbitrary spaces $\Xcal$ endowed with a positive kernel $κ$, which translates naturally into kernels between histograms or clouds of points. We first cover the case where $\Xcal$ is Euclidian, and focus on kernels which take into account the variance matrix of the mixture of two measures to compute their similarity. The kernels we define are semigroup kernels in the sense that they only use the sum of two measures to compare them, and spectral in the sense that they only use the eigenspectrum of the variance matrix of this mixture. We show that such a family of kernels has close bonds with the laplace transforms of nonnegative-valued functions defined on the cone of positive semidefinite matrices, and we present some closed formulas that can be derived as special cases of such integral expressions. By focusing further on functions which are invariant to the addition of a null eigenvalue to the spectrum of the variance matrix, we can define kernels between atomic measures on arbitrary spaces $\Xcal$ endowed with a kernel $κ$ by using directly the eigenvalues of the centered Gram matrix of the joined support of the compared measures. We provide explicit formulas suited for applications and present preliminary experiments to illustrate the interest of the approach.

stat.ML↗
arXiv2006

A kernel for time series based on global alignments

We propose in this paper a new family of kernels to handle times series, notably speech data, within the framework of kernel methods which includes popular algorithms such as the Support Vector Machine. These kernels elaborate on the well known Dynamic Time Warping (DTW) family of distances by considering the same set of elementary operations, namely substitutions and repetitions of tokens, to map a sequence onto another. Associating to each of these operations a given score, DTW algorithms use dynamic programming techniques to compute an optimal sequence of operations with high overall score. In this paper we consider instead the score spanned by all possible alignments, take a smoothed version of their maximum and derive a kernel out of this formulation. We prove that this kernel is positive definite under favorable conditions and show how it can be tuned effectively for practical applications as we report encouraging results on a speech recognition task.

cs.CV↗
arXiv2005

Multiresolution Kernels

We present in this work a new methodology to design kernels on data which is structured with smaller components, such as text, images or sequences. This methodology is a template procedure which can be applied on most kernels on measures and takes advantage of a more detailed "bag of components" representation of the objects. To obtain such a detailed description, we consider possible decompositions of the original bag into a collection of nested bags, following a prior knowledge on the objects' structure. We then consider these smaller bags to compare two objects both in a detailed perspective, stressing local matches between the smaller bags, and in a global or coarse perspective, by considering the entire bag. This multiresolution approach is likely to be best suited for tasks where the coarse approach is not precise enough, and where a more subtle mixture of both local and global similarities is necessary to compare objects. The approach presented here would not be computationally tractable without a factorization trick that we introduce before presenting promising results on an image retrieval task.

cs.LG↗
arXiv2003

A covariance kernel for proteins

We propose a new kernel for biological sequences which borrows ideas and techniques from information theory and data compression. This kernel can be used in combination with any kernel method, in particular Support Vector Machines for protein classification. By incorporating prior biological assumptions on the properties of amino-acid sequences and using a Bayesian averaging framework, we compute the value of this kernel in linear time and space, benefiting from previous achievements proposed in the field of universal coding. Encouraging classification results are reported on a standard protein homology detection experiment.

q-bio.GN↗