arXiv subjects

Pierre-Olivier Amblard

Publications and source records attributed to Pierre-Olivier Amblard.

At least 37 records · Page 2Linked to original sources

arXiv2020

Projections of determinantal point processes

Let $\mathbf x=\{x^{(1)},\dots,x^{(n)}\}$ be a space filling-design of $n$ points defined in $[0{,}1]^d$. In computer experiments, an important property seeked for $\mathbf x$ is a nice coverage of $[0{,}1]^d$. This property could be desirable as well as for any projection of $\mathbf x$ onto $[0{,}1]^ι$ for $ι<d$ . Thus we expect that $\mathbf x_I=\{x_I^{(1)},\dots,x_I^{(n)}\}$, which represents the design $\mathbf x$ with coordinates associated to any index set $I\subseteq\{1,\dots,d\}$, remains regular in $[0{,}1]^ι$ where $ι$ is the cardinality of $I$. This paper examines the conservation of nice coverage by projection using spatial point processes, and more specifically using the class of determinantal point processes. We provide necessary conditions on the kernel defining these processes, ensuring that the projected point process $\mathbf{X}_I$ is repulsive, in the sense that its pair correlation function is uniformly bounded by 1, for all $I\subseteq\{1,\dots,d\}$. We present a few examples, compare them using a new normalized version of Ripley's function. Finally, we illustrate the interest of this research for Monte-Carlo integration.

math.PR↗

arXiv2020

Smoothing graph signals via random spanning forests

Another facet of the elegant link between random processes on graphs and Laplacian-based numerical linear algebra is uncovered: based on random spanning forests, novel Monte-Carlo estimators for graph signal smoothing are proposed. These random forests are sampled efficiently via a variant of Wilson's algorithm --in time linear in the number of edges. The theoretical variance of the proposed estimators are analyzed, and their application to several problems are considered, such as Tikhonov denoising of graph signals or semi-supervised learning for node classification on graphs.

cs.DM↗

arXiv2020

Determinantal Point Processes for Coresets

When faced with a data set too large to be processed all at once, an obvious solution is to retain only part of it. In practice this takes a wide variety of different forms, and among them "coresets" are especially appealing. A coreset is a (small) weighted sample of the original data that comes with the following guarantee: a cost function can be evaluated on the smaller set instead of the larger one, with low relative error. For some classes of problems, and via a careful choice of sampling distribution (based on the so-called "sensitivity" metric), iid random sampling has turned to be one of the most successful methods for building coresets efficiently. However, independent samples are sometimes overly redundant, and one could hope that enforcing diversity would lead to better performance. The difficulty lies in proving coreset properties in non-iid samples. We show that the coreset property holds for samples formed with determinantal point processes (DPP). DPPs are interesting because they are a rare example of repulsive point processes with tractable theoretical properties, enabling us to prove general coreset theorems. We apply our results to both the k-means and the linear regression problems, and give extensive empirical evidence that the small additional computational cost of DPP sampling comes with superior performance over its iid counterpart. Of independent interest, we also provide analytical formulas for the sensitivity in the linear regression and 1-means cases.

stat.ML↗

arXiv2019

Estimating the inverse trace using random forests on graphs

Some data analysis problems require the computation of (regularised) inverse traces, i.e. quantities of the form $\Tr (q \bI + \bL)^{-1}$. For large matrices, direct methods are unfeasible and one must resort to approximations, for example using a conjugate gradient solver combined with Girard's trace estimator (also known as Hutchinson's trace estimator). Here we describe an unbiased estimator of the regularized inverse trace, based on Wilson's algorithm, an algorithm that was initially designed to draw uniform spanning trees in graphs. Our method is fast, easy to implement, and scales to very large matrices. Its main drawback is that it is limited to diagonally dominant matrices $\bL$.

stat.CO↗

arXiv2019

Optimal Measurement Times for a Small Number of Measures of a Brownian Motion over a Finite Period

The measure timetable plays a critical role for the accuracy of the estimator. This article deals with the optimization of the schedule of measures for observing a random process in time using a Kalman filter, when the length of the process is finite and fixed, and a fixed number of measures are available. The measuring devices are allowed to differ. The mean variance of the estimator is chosen as criterion for optimality. The cases of $1$ or $2$ measures are studied in detail, and analytical formulas are provided.

eess.SP↗

arXiv2018

Asymptotic Equivalence of Fixed-size and Varying-size Determinantal Point Processes

Determinantal Point Processes (DPPs) are popular models for point processes with repulsion. They appear in numerous contexts, from physics to graph theory, and display appealing theoretical properties. On the more practical side of things, since DPPs tend to select sets of points that are some distance apart (repulsion), they have been advocated as a way of producing random subsets with high diversity. DPPs come in two variants: fixed-size and varying-size. A sample from a varying-size DPP is a subset of random cardinality, while in fixed-size "$k$-DPPs" the cardinality is fixed. The latter makes more sense in many applications, but unfortunately their computational properties are less attractive, since, among other things, inclusion probabilities are harder to compute. In this work we show that as the size of the ground set grows, $k$-DPPs and DPPs become equivalent, meaning that their inclusion probabilities converge. As a by-product, we obtain saddlepoint formulas for inclusion probabilities in $k$-DPPs. These turn out to be extremely accurate, and suffer less from numerical difficulties than exact methods do. Our results also suggest that $k$-DPPs and DPPs also have equivalent maximum likelihood estimators. Finally, we obtain results on asymptotic approximations of elementary symmetric polynomials which may be of independent interest.

math.ST↗

arXiv2018

Optimized Algorithms to Sample Determinantal Point Processes

In this technical report, we discuss several sampling algorithms for Determinantal Point Processes (DPP). DPPs have recently gained a broad interest in the machine learning and statistics literature as random point processes with negative correlation, i.e., ones that can generate a "diverse" sample from a set of items. They are parametrized by a matrix $\mathbf{L}$, called $L$-ensemble, that encodes the correlations between items. The standard sampling algorithm is separated in three phases: 1/~eigendecomposition of $\mathbf{L}$, 2/~an eigenvector sampling phase where $\mathbf{L}$'s eigenvectors are sampled independently via a Bernoulli variable parametrized by their associated eigenvalue, 3/~a Gram-Schmidt-type orthogonalisation procedure of the sampled eigenvectors. In a naive implementation, the computational cost of the third step is on average $\mathcal{O}(Nμ^3)$ where $μ$ is the average number of samples of the DPP. We give an algorithm which runs in $\mathcal{O}(Nμ^2)$ and is extremely simple to implement. If memory is a constraint, we also describe a dual variant with reduced memory costs. In addition, we discuss implementation details often missing in the literature.

stat.CO↗

arXiv2017

Échantillonnage de signaux sur graphes via des processus déterminantaux

We consider the problem of sampling k-bandlimited graph signals, ie, linear combinations of the first k graph Fourier modes. We know that a set of k nodes embedding all k-bandlimited signals always exists, thereby enabling their perfect reconstruction after sampling. Unfortunately, to exhibit such a set, one needs to partially diagonalize the graph Laplacian, which becomes prohibitive at large scale. We propose a novel strategy based on determinantal point processes that side-steps partial diagonalisation and enables reconstruction with only O(k) samples. While doing so, we exhibit a new general algorithm to sample determinantal process, faster than the state-of-the-art algorithm by an order k.

cs.DS↗

arXiv2017

Graph sampling with determinantal processes

We present a new random sampling strategy for k-bandlimited signals defined on graphs, based on determinantal point processes (DPP). For small graphs, ie, in cases where the spectrum of the graph is accessible, we exhibit a DPP sampling scheme that enables perfect recovery of bandlimited signals. For large graphs, ie, in cases where the graph's spectrum is not accessible, we investigate, both theoretically and empirically, a sub-optimal but much faster DPP based on loop-erased random walks on the graph. Preliminary experiments show promising results especially in cases where the number of measurements should stay as small as possible and for graphs that have a strong community structure. Our sampling scheme is efficient and can be applied to graphs with up to $10^6$ nodes.

cs.LG↗

arXiv2015

Compressed and quantized correlation estimators

In passive monitoring using sensor networks, low energy supplies drastically constrain sensors in terms of calculation and communication abilities. Designing processing algorithms at the sensor level that take into account these constraints is an important problem in this context. We study here the estimation of correlation functions between sensors using compressed acquisition and one-bit-quantization. The estimation is achieved directly using compressed samples, without considering any reconstruction of the signals. We show that if the signals of interest are far from white noise, estimation of the correlation using $M$ compressed samples out of $N\geq M$ can be more advantageous than estimation of the correlation using $M$ consecutive samples. The analysis consists of studying the asymptotic performance of the estimators at a fixed compression rate. We provide the analysis when the compression is realized by a random projection matrix composed of independent and identically distributed entries. The framework includes widely used random projection matrices, such as Gaussian and Bernoulli matrices, and it also includes very sparse matrices. However, it does not include subsampling without replacement, for which a separate analysis is provided. When considering one-bit-quantization as well, the theoretical analysis is not tractable. However, empirical evidence allows the conclusion that in practical situations, compressed and quantized estimators behave sufficiently correctly to be useful in, for example, time-delay estimation and model estimation.

stat.AP↗

arXiv2015

A Primer on Reproducing Kernel Hilbert Spaces

Reproducing kernel Hilbert spaces are elucidated without assuming prior familiarity with Hilbert spaces. Compared with extant pedagogic material, greater care is placed on motivating the definition of reproducing kernel Hilbert spaces and explaining when and why these spaces are efficacious. The novel viewpoint is that reproducing kernel Hilbert space theory studies extrinsic geometry, associating with each geometric configuration a canonical overdetermined coordinate system. This coordinate system varies continuously with changing geometric configurations, making it well-suited for studying problems whose solutions also vary continuously with changing geometry. This primer can also serve as an introduction to infinite-dimensional linear algebra because reproducing kernel Hilbert spaces have more properties in common with Euclidean spaces than do more general Hilbert spaces.

math.HO↗

arXiv2013

A non-parametric efficient evaluation of Partial Directed Coherence

Studying the flow of information between different areas of the brain can be performed by using the so-called Partial Directed Coherence. This measure is usually evaluated by first identifying a multivariate autoregressive model, and then by using Fourier transforms of the impulse responses identified and applying appropriate normalizations. Here, we present another route to evaluate the partial directed coherences in multivariate time series. The method proposed is non parametric, and utilises the strong spectral factorization of the inverse of the spectral density matrix of the multivariate process. To perform the factorization, we have recourse to an algorithm developed by Davis and his collaborators. We present simulations as well as an application on a real data set (Local Field Potentials in the sleeping mouse) to illustrate the methodology. A comparison to the usual approach in term of complexity is detailed. For long AR models, the proposed approach is of interest.

q-bio.NC↗

arXiv2012

The relation between Granger causality and directed information theory: a review

This report reviews the conceptual and theoretical links between Granger causality and directed information theory. We begin with a short historical tour of Granger causality, concentrating on its closeness to information theory. The definitions of Granger causality based on prediction are recalled, and the importance of the observation set is discussed. We present the definitions based on conditional independence. The notion of instantaneous coupling is included in the definitions. The concept of Granger causality graphs is discussed. We present directed information theory from the perspective of studies of causal influences between stochastic processes. Causal conditioning appears to be the cornerstone for the relation between information theory and Granger causality. In the bivariate case, the fundamental measure is the directed information, which decomposes as the sum of the transfer entropies and a term quantifying instantaneous coupling. We show the decomposition of the mutual information into the sums of the transfer entropies and the instantaneous coupling measure, a relation known for the linear Gaussian case. We study the multivariate case, showing that the useful decomposition is blurred by instantaneous coupling. The links are further developed by studying how measures based on directed information theory naturally emerge from Granger causality inference frameworks as hypothesis testing.

cs.IT↗

arXiv2012

Basic properties of the Multivariate Fractional Brownian Motion

This paper reviews and extends some recent results on the multivariate fractional Brownian motion (mfBm) and its increment process. A characterization of the mfBm through its covariance function is obtained. Similarly, the correlation and spectral analyses of the increments are investigated. On the other hand we show that (almost) all mfBm's may be reached as the limit of partial sums of (super)linear processes. Finally, an algorithm to perfectly simulate the mfBm is presented and illustrated by some simulations.

math.PR↗

arXiv2012

Causal conditioning and instantaneous coupling in causality graphs

The paper investigates the link between Granger causality graphs recently formalized by Eichler and directed information theory developed by Massey and Kramer. We particularly insist on the implication of two notions of causality that may occur in physical systems. It is well accepted that dynamical causality is assessed by the conditional transfer entropy, a measure appearing naturally as a part of directed information. Surprisingly the notion of instantaneous causality is often overlooked, even if it was clearly understood in early works. In the bivariate case, instantaneous coupling is measured adequately by the instantaneous information exchange, a measure that supplements the transfer entropy in the decomposition of directed information. In this paper, the focus is put on the multivariate case and conditional graph modeling issues. In this framework, we show that the decomposition of directed information into the sum of transfer entropy and information exchange does not hold anymore. Nevertheless, the discussion allows to put forward the two measures as pillars for the inference of causality graphs. We illustrate this on two synthetic examples which allow us to discuss not only the theoretical concepts, but also the practical estimation issues.

cs.IT↗

arXiv2011

Relating Granger causality to directed information theory for networks of stochastic processes

This paper addresses the problem of inferring circulation of information between multiple stochastic processes. We discuss two possible frameworks in which the problem can be studied: directed information theory and Granger causality. The main goal of the paper is to study the connection between these two frameworks. In the case of directed information theory, we stress the importance of Kramer's causal conditioning. This type of conditioning is necessary not only in the definition of the directed information but also for handling causal side information. We also show how directed information decomposes into the sum of two measures, the first one related to Schreiber's transfer entropy quantifies the dynamical aspects of causality, whereas the second one, termed instantaneous information exchange, quantifies the instantaneous aspect of causality. After having recalled the definition of Granger causality, we establish its connection with directed information theory. The connection is particularly studied in the Gaussian case, showing that Geweke's measures of Granger causality correspond to the transfer entropy and the instantaneous information exchange. This allows to propose an information theoretic formulation of Granger causality.

cs.IT↗

arXiv2011

Wavelet analysis of the multivariate fractional Brownian motion

The work developed in the paper concerns the multivariate fractional Brownian motion (mfBm) viewed through the lens of the wavelet transform. After recalling some basic properties on the mfBm, we calculate the correlation structure of its wavelet transform. We particularly study the asymptotic behavior of the correlation, showing that if the analyzing wavelet has a sufficient number of null first order moments, the decomposition eliminates any possible long-range (inter)dependence. The cross-spectral density is also considered in a second part. Its existence is proved and its evaluation is performed using a von Bahr-Essen like representation of the function $\sign(t) |t|^α$. The behavior of the cross-spectral density of the wavelet field at the zero frequency is also developed and confirms the results provided by the asymptotic analysis of the correlation.

math.PR↗

arXiv2011

Identification of the Multivariate Fractional Brownian Motion

This paper deals with the identification of the multivariate fractional Brownian motion, a recently developed extension of the fractional Brownian motion to the multivariate case. This process is a $p$-multivariate self-similar Gaussian process parameterized by $p$ different Hurst exponents $H_i$, $p$ scaling coefficients $σ_i$ (of each component) and also by $p(p-1)$ coefficients $ρ_{ij},η_{ij}$ (for $i,j=1,...,p$ with $j>i$) allowing two components to be more or less strongly correlated and allowing the process to be time reversible or not. We investigate the use of discrete filtering techniques to estimate jointly or separately the different parameters and prove the efficiency of the methodology with a simulation study and the derivation of asymptotic results.

math.ST↗