Searcharxiv⌕ Search

arXiv subjects

Petros Dellaportas

Publications and source records attributed to Petros Dellaportas.

At least 37 records · Page 2Linked to original sources
arXiv2022

Doubly-online changepoint detection for monitoring health status during sports activities

We provide an online framework for analyzing data recorded by smart watches during running activities. In particular, we focus on identifying variations in the behavior of one or more measurements caused by changes in physical condition, such as physical discomfort, periods of prolonged de-training, or even the malfunction of measuring devices. Our framework considers data as a sequence of running activities represented by multivariate time series of physical and biometric data. We combine classical changepoint detection models with an unknown number of components with Gaussian state space models to detect distributional changes between a sequence of activities. The model considers multiple sources of dependence due to the sequential nature of subsequent activities, the autocorrelation structure within each activity, and the contemporaneous dependence between different variables. We provide an online Expectation-Maximization (EM) algorithm involving a sequential Monte Carlo (SMC) approximation of changepoint predicted probabilities. As a byproduct of our model assumptions, our proposed approach processes sequences of multivariate time series in a doubly-online framework. While classical changepoint models detect changes between subsequent activities, the state space framework coupled with the online EM algorithm provides the additional benefit of estimating the real-time probability that a current activity is a changepoint.

stat.AP↗
arXiv2022

Variance Reduction for Metropolis-Hastings Samplers

We introduce a general framework that constructs estimators with reduced variance for random walk Metropolis and Metropolis-adjusted Langevin algorithms. The resulting estimators require negligible computational cost and are derived in a post-process manner utilising all proposal values of the Metropolis algorithms. Variance reduction is achieved by producing control variates through the approximate solution of the Poisson equation associated with the target density of the Markov chain. The proposed method is based on approximating the target density with a Gaussian and then utilising accurate solutions of the Poisson equation for the Gaussian case. This leads to an estimator that uses two key elements: (i) a control variate from the Poisson equation that contains an intractable expectation under the proposal distribution, (ii) a second control variate to reduce the variance of a Monte Carlo estimate of this latter intractable expectation. Simulated data examples are used to illustrate the impressive variance reduction achieved in the Gaussian target case and the corresponding effect when target Gaussianity assumption is violated. Real data examples on Bayesian logistic regression and stochastic volatility models verify that considerable variance reduction is achieved with negligible extra computational cost.

stat.ME↗
arXiv2022

How Good are Low-Rank Approximations in Gaussian Process Regression?

We provide guarantees for approximate Gaussian Process (GP) regression resulting from two common low-rank kernel approximations: based on random Fourier features, and based on truncating the kernel's Mercer expansion. In particular, we bound the Kullback-Leibler divergence between an exact GP and one resulting from one of the afore-described low-rank approximations to its kernel, as well as between their corresponding predictive densities, and we also bound the error between predictive mean vectors and between predictive covariance matrices computed using the exact versus using the approximate GP. We provide experiments on both simulated data and standard benchmarks to evaluate the effectiveness of our theoretical bounds.

stat.ML↗
arXiv2021

How Good are Low-Rank Approximations in Gaussian Process Regression?

We provide guarantees for approximate Gaussian Process (GP) regression resulting from two common low-rank kernel approximations: based on random Fourier features, and based on truncating the kernel's Mercer expansion. In particular, we bound the Kullback-Leibler divergence between an exact GP and one resulting from one of the afore-described low-rank approximations to its kernel, as well as between their corresponding predictive densities, and we also bound the error between predictive mean vectors and between predictive covariance matrices computed using the exact versus using the approximate GP. We provide experiments on both simulated data and standard benchmarks to evaluate the effectiveness of our theoretical bounds.

stat.ML↗
arXiv2021

Entropy-based adaptive Hamiltonian Monte Carlo

Hamiltonian Monte Carlo (HMC) is a popular Markov Chain Monte Carlo (MCMC) algorithm to sample from an unnormalized probability distribution. A leapfrog integrator is commonly used to implement HMC in practice, but its performance can be sensitive to the choice of mass matrix used therein. We develop a gradient-based algorithm that allows for the adaptation of the mass matrix by encouraging the leapfrog integrator to have high acceptance rates while also exploring all dimensions jointly. In contrast to previous work that adapt the hyperparameters of HMC using some form of expected squared jumping distance, the adaptation strategy suggested here aims to increase sampling efficiency by maximizing an approximation of the proposal entropy. We illustrate that using multiple gradients in the HMC proposal can be beneficial compared to a single gradient-step in Metropolis-adjusted Langevin proposals. Empirical evidence suggests that the adaptation method can outperform different versions of HMC schemes by adjusting the mass matrix to the geometry of the target distribution and by providing some control on the integration time.

stat.CO↗
arXiv2021

Bayesian prediction of jumps in large panels of time series data

We take a new look at the problem of disentangling the volatility and jumps processes of daily stock returns. We first provide a computational framework for the univariate stochastic volatility model with Poisson-driven jumps that offers a competitive inference alternative to the existing tools. This methodology is then extended to a large set of stocks for which we assume that their unobserved jump intensities co-evolve in time through a dynamic factor model. To evaluate the proposed modelling approach we conduct out-of-sample forecasts and we compare the posterior predictive distributions obtained from the different models. We provide evidence that joint modelling of jumps improves the predictive ability of the stochastic volatility models.

q-fin.ST↗
arXiv2020

Gradient-based Adaptive Markov Chain Monte Carlo

We introduce a gradient-based learning method to automatically adapt Markov chain Monte Carlo (MCMC) proposal distributions to intractable targets. We define a maximum entropy regularised objective function, referred to as generalised speed measure, which can be robustly optimised over the parameters of the proposal distribution by applying stochastic gradient optimisation. An advantage of our method compared to traditional adaptive MCMC methods is that the adaptation occurs even when candidate state values are rejected. This is a highly desirable property of any adaptation strategy because the adaptation starts in early iterations even if the initial proposal distribution is far from optimum. We apply the framework for learning multivariate random walk Metropolis and Metropolis-adjusted Langevin proposals with full covariance matrices, and provide empirical evidence that our method can outperform other MCMC algorithms, including Hamiltonian Monte Carlo schemes.

stat.ML↗
arXiv2019

Copula-like Variational Inference

This paper considers a new family of variational distributions motivated by Sklar's theorem. This family is based on new copula-like densities on the hypercube with non-uniform marginals which can be sampled efficiently, i.e. with a complexity linear in the dimension of state space. Then, the proposed variational densities that we suggest can be seen as arising from these copula-like densities used as base distributions on the hypercube with Gaussian quantile functions and sparse rotation matrices as normalizing flows. The latter correspond to a rotation of the marginals with complexity $\mathcal{O}(d \log d)$. We provide some empirical evidence that such a variational family can also approximate non-Gaussian posteriors and can be beneficial compared to Gaussian approximations. Our method performs largely comparably to state-of-the-art variational approximations on standard regression and classification benchmarks for Bayesian Neural Networks.

stat.ML↗
arXiv2019

Scalable Bayesian Learning for State Space Models using Variational Inference with SMC Samplers

We present a scalable approach to performing approximate fully Bayesian inference in generic state space models. The proposed method is an alternative to particle MCMC that provides fully Bayesian inference of both the dynamic latent states and the static parameters of the model. We build up on recent advances in computational statistics that combine variational methods with sequential Monte Carlo sampling and we demonstrate the advantages of performing full Bayesian inference over the static parameters rather than just performing variational EM approximations. We illustrate how our approach enables scalable inference in multivariate stochastic volatility models and self-exciting point process models that allow for flexible dynamics in the latent intensity function.

stat.ML↗
arXiv2018

Fully Scalable Gaussian Processes using Subspace Inducing Inputs

We introduce fully scalable Gaussian processes, an implementation scheme that tackles the problem of treating a high number of training instances together with high dimensional input data. Our key idea is a representation trick over the inducing variables called subspace inducing inputs. This is combined with certain matrix-preconditioning based parametrizations of the variational distributions that lead to simplified and numerically stable variational lower bounds. Our illustrative applications are based on challenging extreme multi-label classification problems with the extra burden of the very large number of class labels. We demonstrate the usefulness of our approach by presenting predictive performances together with low computational times in datasets with extremely large number of instances and input dimensions.

stat.ML↗
arXiv2018

Bayesian forecasting of mortality rates using latent Gaussian models

We provide forecasts for mortality rates by using two different approaches. First we employ dynamic non-linear logistic models based on Heligman-Pollard formula. Second, we assume that the dynamics of the mortality rates can be modelled through a Gaussian Markov random field. We use efficient Bayesian methods to estimate the parameters and the latent states of the proposed models. Both methodologies are tested with past data and are used to forecast mortality rates both for large (UK and Wales) and small (New Zealand) populations up to 21 years ahead. We demonstrate that predictions for individual survivor functions and other posterior summaries of demographic and actuarial interest are readily obtained. Our results are compared with other competing forecasting methods.

stat.AP↗
arXiv2017

Efficient sequential Monte Carlo algorithms for integrated population models

State-space models are commonly used to describe different forms of ecological data. We consider the case of count data with observation errors. For such data the system process is typically multi-dimensional consisting of coupled Markov processes, where each component corresponds to a different characterisation of the population, such as age group, gender or breeding status. The associated system process equations describe the biological mechanisms under which the system evolves over time. However, there is often limited information in the count data alone to sensibly estimate demographic parameters of interest, so these are often combined with additional ecological observations leading to an integrated data analysis. Unfortunately, fitting these models to the data can be challenging, especially if the state-space model for the count data is non-linear or non-Gaussian. We propose an efficient particle Markov chain Monte Carlo algorithm to estimate the demographic parameters without the need for resorting to linear or Gaussian approximations. In particular, we exploit the integrated model structure to enhance the efficiency of the algorithm. We then incorporate the algorithm into a sequential Monte Carlo sampler in order to perform model comparison with regards to the dependence structure of the demographic parameters. Finally, we demonstrate the applicability and computational efficiency of our algorithms on two real datasets.

stat.ME↗
arXiv2014

Communication impacting financial markets

Behavioral finance has become an increasingly important subfield of finance. However the main parts of behavioral finance, prospect theory included, understand financial markets through individual investment behavior. Behavioral finance thereby ignores any interaction between participants. We introduce a socio-financial model that studies the impact of communication on the pricing in financial markets. Considering the simplest possible case where each market participant has either a positive (bullish) or negative (bearish) sentiment with respect to the market, we model the evolution of the sentiment in the population due to communication in subgroups of different sizes. Nonlinear feedback effects between the market performance and changes in sentiments are taking into account by assuming that the market performance is dependent on changes in sentiments (e.g. a large sudden positive change in bullishness would lead to more buying). The market performance in turn has an impact on the sentiment through the transition probabilities to change an opinion in a group of a given size. The idea is that if for example the market has observed a recent downturn, it will be easier for even a bearish minority to convince a bullish majority to change opinion compared to the case where the meeting takes place in a bullish upturn of the market. Within the framework of our proposed model, financial markets stylized facts such as volatility clustering and extreme events may be perceived as arising due to abrupt sentiment changes via ongoing communication of the market participants. The model introduces a new volatility measure which is apt of capturing volatility clustering and from maximum likelihood analysis we are able to apply the model to real data and give additional long term insight into where a market is heading.

q-fin.GN↗
arXiv2014

Arbitrage-free prediction of the implied volatility smile

This paper gives an arbitrage-free prediction for future prices of an arbitrary co-terminal set of options with a given maturity, based on the observed time series of these option prices. The statistical analysis of such a multi-dimensional time series of option prices corresponding to $n$ strikes (with $n$ large, e.g. $n\geq 40$) and the same maturity, is a difficult task due to the fact that option prices at any moment in time satisfy non-linear and non-explicit no-arbitrage restrictions. Hence any $n$-dimensional time series model also has to satisfy these implicit restrictions at each time step, a condition that is impossible to meet since the model innovations can take arbitrary values. We solve this problem for any $n\in\NN$ in the context of Foreign Exchange (FX) by first encoding the option prices at each time step in terms of the parameters of the corresponding risk-neutral measure and then performing the time series analysis in the parameter space. The option price predictions are obtained from the predicted risk-neutral measure by effectively integrating it against the corresponding option payoffs. The non-linear transformation between option prices and the risk-neutral parameters applied here is \textit{not} arbitrary: it is the standard mapping used by market makers in the FX option markets (the SABR parameterisation) and is given explicitly in closed form. Our method is not restricted to the FX asset class nor does it depend on the type of parameterisation used. Statistical analysis of FX market data illustrates that our arbitrage-free predictions outperform the naive random walk forecasts, suggesting a potential for building management strategies for portfolios of derivative products, akin to the ones widely used in the underlying equity and futures markets.

q-fin.PR↗
arXiv2012

Joint Specification of Model Space and Parameter Space Prior Distributions

We consider the specification of prior distributions for Bayesian model comparison, focusing on regression-type models. We propose a particular joint specification of the prior distribution across models so that sensitivity of posterior model probabilities to the dispersion of prior distributions for the parameters of individual models (Lindley's paradox) is diminished. We illustrate the behavior of inferential and predictive posterior quantities in linear and log-linear regressions under our proposed prior densities with a series of simulated and real data examples.

stat.ME↗
arXiv2010

Control Variates for Reversible MCMC Samplers

A general methodology is introduced for the construction and effective application of control variates to estimation problems involving data from reversible MCMC samplers. We propose the use of a specific class of functions as control variates, and we introduce a new, consistent estimator for the values of the coefficients of the optimal linear combination of these functions. The form and proposed construction of the control variates is derived from our solution of the Poisson equation associated with a specific MCMC scenario. The new estimator, which can be applied to the same MCMC sample, is derived from a novel, finite-dimensional, explicit representation for the optimal coefficients. The resulting variance-reduction methodology is primarily applicable when the simulated data are generated by a conjugate random-scan Gibbs sampler. MCMC examples of Bayesian inference problems demonstrate that the corresponding reduction in the estimation variance is significant, and that in some cases it can be quite dramatic. Extensions of this methodology in several directions are given, including certain families of Metropolis-Hastings samplers and hybrid Metropolis-within-Gibbs algorithms. Corresponding simulation examples are presented illustrating the utility of the proposed methods. All methodological and asymptotic arguments are rigorously justified under easily verifiable and essentially minimal conditions.

stat.CO↗
arXiv2010

Notes on Using Control Variates for Estimation with Reversible MCMC Samplers

A general methodology is presented for the construction and effective use of control variates for reversible MCMC samplers. The values of the coefficients of the optimal linear combination of the control variates are computed, and adaptive, consistent MCMC estimators are derived for these optimal coefficients. All methodological and asymptotic arguments are rigorously justified. Numerous MCMC simulation examples from Bayesian inference applications demonstrate that the resulting variance reduction can be quite dramatic.

stat.CO↗
arXiv2007

Inference for stochastic volatility models using time change transformations

We address the problem of parameter estimation for diffusion driven stochastic volatility models through Markov chain Monte Carlo (MCMC). To avoid degeneracy issues we introduce an innovative reparametrisation defined through transformations that operate on the time scale of the diffusion. A novel MCMC scheme which overcomes the inherent difficulties of time change transformations is also presented. The algorithm is fast to implement and applies to models with stochastic volatility. The methodology is tested through simulation based experiments and illustrated on data consisting of US treasury bill rates.

stat.ME↗