arXiv subjects

Mathieu Gerber

Publications and source records attributed to Mathieu Gerber.

26 records · Page 2Linked to original sources

arXiv2015

Convergence Results for a Class of Time-Varying Simulated Annealing Algorithms

We provide a set of conditions which ensure the almost sure convergence of a class of simulated annealing algorithms on a bounded set $\mathcal{X}\subset\mathbb{R}^d$ based on a time-varying Markov kernel. The class of algorithms considered in this work encompasses the one studied in Belisle (1992) and Yang (2000) as well as its derandomized version recently proposed by Gerber and Bornn (2016). To the best of our knowledge, the results we derive are the first examples of almost sure convergence results for simulated annealing based on a time-varying kernel. In addition, the assumptions on the Markov kernel and on the cooling schedule have the advantage of being trivial to verify in practice.

math.PR↗

arXiv2015

Convergence of Sequential Quasi-Monte Carlo Smoothing Algorithms

Gerber and Chopin (2015) recently introduced Sequential quasi-Monte Carlo (SQMC) algorithms as an efficient way to perform filtering in state-space models. The basic idea is to replace random variables with low-discrepancy point sets, so as to obtain faster convergence than with standard particle filtering. Gerber and Chopin (2015) describe briefly several ways to extend SQMC to smoothing, but do not provide supporting theory for this extension. We discuss more thoroughly how smoothing may be performed within SQMC, and derive convergence results for the so-obtained smoothing algorithms. We consider in particular SQMC equivalents of forward smoothing and forward filtering backward sampling, which are the most well-known smoothing techniques. As a preliminary step, we provide a generalization of the classical result of Hlawka and Mück (1972) on the transformation of QMC point sets into low discrepancy point sets with respect to non uniform distributions. As a corollary of the latter, we note that we can slightly weaken the assumptions to prove the consistency of SQMC.

stat.CO↗

arXiv2015

Bayesian Inference for the Multivariate Extended-Skew Normal Distribution

The multivariate extended skew-normal distribution allows for accommodating raw data which are skewed and heavy tailed, and has at least three appealing statistical properties, namely closure under conditioning, affine transformations, and marginalization. In this paper we propose a Bayesian computational approach based on a sequential Monte Carlo (SMC) sampler to estimate such distributions. The practical implementation of each step of the algorithm is discussed and the elicitation of prior distributions takes into consideration some unusual behaviour of the likelihood function and the corresponding Fisher information matrix. Using Monte Carlo simulations, we provide strong evidence regarding the performances of the SMC sampler as well as some new insights regarding the parametrizations of the extended skew-normal distribution. A generalization to the extended skew-normal sample selection model is also presented. Finally we proceed with the analysis of two real datasets.

stat.ME↗

arXiv2015

Towards automatic calibration of the number of state particles within the SMC$^2$ algorithm

SMC$^2$ is an efficient algorithm for sequential estimation and state inference of state-space models. It generates $N_θ$ parameter particles $θ^{m}$, and, for each $θ^{m}$, it runs a particle filter of size $N_{x}$ (i.e. at each time step, $N_{x}$ particles are generated in the state space $\mathcal{X}$). We discuss how to automatically calibrate $N_{x}$ in the course of the algorithm. Our approach relies on conditional Sequential Monte Carlo updates, monitoring the state of the pseudo random number generator and on an estimator of the variance of the unbiased estimate of the likelihood that is produced by the particle filters, which is obtained using nonparametric regression techniques. We observe that our approach is both less CPU intensive and with smaller Monte Carlo errors than the initial version of SMC$^2$.

stat.CO↗

arXiv2015

Improving Simulated Annealing through Derandomization

We propose and study a version of simulated annealing (SA) on continuous state spaces based on $(t,s)_R$-sequences. The parameter $R\in\bar{\mathbb{N}}$ regulates the degree of randomness of the input sequence, with the case $R=0$ corresponding to IID uniform random numbers and the limiting case $R=\infty$ to $(t,s)$-sequences. Our main result, obtained for rectangular domains, shows that the resulting optimization method, which we refer to as QMC-SA, converges almost surely to the global optimum of the objective function $\varphi$ for any $R\in\mathbb{N}$. When $\varphi$ is univariate, we are in addition able to show that the completely deterministic version of QMC-SA is convergent. A key property of these results is that they do not require objective-dependent conditions on the cooling schedule. As a corollary of our theoretical analysis, we provide a new almost sure convergence result for SA which shares this property under minimal assumptions on $\varphi$. We further explain how our results in fact apply to a broader class of optimization methods including for example threshold accepting, for which to our knowledge no convergence results currently exist. We finally illustrate the superiority of QMC-SA over SA algorithms in a numerical study.

stat.CO↗

arXiv2015

Application of Sequential Quasi-Monte Carlo to Autonomous Positioning

Sequential Monte Carlo algorithms (also known as particle filters) are popular methods to approximate filtering (and related) distributions of state-space models. However, they converge at the slow $1/\sqrt{N}$ rate, which may be an issue in real-time data-intensive scenarios. We give a brief outline of SQMC (Sequential Quasi-Monte Carlo), a variant of SMC based on low-discrepancy point sets proposed by Gerber and Chopin (2015), which converges at a faster rate, and we illustrate the greater performance of SQMC on autonomous positioning problems.

stat.CO↗

arXiv2014

Sequential Quasi-Monte Carlo

We derive and study SQMC (Sequential Quasi-Monte Carlo), a class of algorithms obtained by introducing QMC point sets in particle filtering. SQMC is related to, and may be seen as an extension of, the array-RQMC algorithm of L'Ecuyer et al. (2006). The complexity of SQMC is $O(N \log N)$, where $N$ is the number of simulations at each iteration, and its error rate is smaller than the Monte Carlo rate $O_P(N^{-1/2})$. The only requirement to implement SQMC is the ability to write the simulation of particle $x_t^n$ given $x_{t-1}^n$ as a deterministic function of $x_{t-1}^n$ and a fixed number of uniform variates. We show that SQMC is amenable to the same extensions as standard SMC, such as forward smoothing, backward smoothing, unbiased likelihood evaluation, and so on. In particular, SQMC may replace SMC within a PMCMC (particle Markov chain Monte Carlo) algorithm. We establish several convergence results. We provide numerical evidence that SQMC may significantly outperform SMC in practical scenarios.

stat.CO↗

arXiv2014

On Integration Methods Based on Scrambled Nets of Arbitrary Size

We consider the problem of evaluating $I(\varphi):=\int_{[0,1)^s}\varphi(x) dx$ for a function $\varphi \in L^2[0,1)^{s}$. In situations where $I(\varphi)$ can be approximated by an estimate of the form $N^{-1}\sum_{n=0}^{N-1}\varphi(x^n)$, with $\{x^n\}_{n=0}^{N-1}$ a point set in $[0,1)^s$, it is now well known that the $O_P(N^{-1/2})$ Monte Carlo convergence rate can be improved by taking for $\{x^n\}_{n=0}^{N-1}$ the first $N=\lambda b^m$ points, $\lambda\in\{1,\dots,b-1\}$, of a scrambled $(t,s)$-sequence in base $b\geq 2$. In this paper we derive a bound for the variance of scrambled net quadrature rules which is of order $o(N^{-1})$ without any restriction on $N$. As a corollary, this bound allows us to provide simple conditions to get, for any pattern of $N$, an integration error of size $o_P(N^{-1/2})$ for functions that depend on the quadrature size $N$. Notably, we establish that sequential quasi-Monte Carlo (M. Gerber and N. Chopin, 2015, \emph{J. R. Statist. Soc. B, to appear.}) reaches the $o_P(N^{-1/2})$ convergence rate for any values of $N$. In a numerical study, we show that for scrambled net quadrature rules we can relax the constraint on $N$ without any loss of efficiency when the integrand $\varphi$ is a discontinuous function while, for sequential quasi-Monte Carlo, taking $N=\lambda b^m$ may only provide moderate gains.

stat.CO↗