Searcharxiv⌕ Search

arXiv subjects

Daniel Simpson

Publications and source records attributed to Daniel Simpson.

At least 37 records · Page 2Linked to original sources
arXiv2018

Visualization in Bayesian workflow

Bayesian data analysis is about more than just computing a posterior distribution, and Bayesian visualization is about more than trace plots of Markov chains. Practical Bayesian data analysis, like all data analysis, is an iterative process of model building, inference, model checking and evaluation, and model expansion. Visualization is helpful in each of these stages of the Bayesian workflow and it is indispensable when drawing inferences from the types of modern, high-dimensional models that are used by applied researchers.

stat.ME↗
arXiv2018

Spatial modelling with R-INLA: A review

Coming up with Bayesian models for spatial data is easy, but performing inference with them can be challenging. Writing fast inference code for a complex spatial model with realistically-sized datasets from scratch is time-consuming, and if changes are made to the model, there is little guarantee that the code performs well. The key advantages of R-INLA are the ease with which complex models can be created and modified, without the need to write complex code, and the speed at which inference can be done even for spatial problems with hundreds of thousands of observations. R-INLA handles latent Gaussian models, where fixed effects, structured and unstructured Gaussian random effects are combined linearly in a linear predictor, and the elements of the linear predictor are observed through one or more likelihoods. The structured random effects can be both standard areal model such as the Besag and the BYM models, and geostatistical models from a subset of the Matérn Gaussian random fields. In this review, we discuss the large success of spatial modelling with R-INLA and the types of spatial models that can be fitted, we give an overview of recent developments for areal models, and we give an overview of the stochastic partial differential equation (SPDE) approach and some of the ways it can be extended beyond the assumptions of isotropy and separability. In particular, we describe how slight changes to the SPDE approach leads to straight-forward approaches for non-stationary spatial models and non-separable space-time models.

stat.ME↗
arXiv2017

Constructing Priors that Penalize the Complexity of Gaussian Random Fields

Priors are important for achieving proper posteriors with physically meaningful covariance structures for Gaussian random fields (GRFs) since the likelihood typically only provides limited information about the covariance structure under in-fill asymptotics. We extend the recent Penalised Complexity prior framework and develop a principled joint prior for the range and the marginal variance of one-dimensional, two-dimensional and three-dimensional Matérn GRFs with fixed smoothness. The prior is weakly informative and penalises complexity by shrinking the range towards infinity and the marginal variance towards zero. We propose guidelines for selecting the hyperparameters, and a simulation study shows that the new prior provides a principled alternative to reference priors that can leverage prior knowledge to achieve shorter credible intervals while maintaining good coverage. We extend the prior to a non-stationary GRF parametrized through local ranges and marginal standard deviations, and introduce a scheme for selecting the hyperparameters based on the coverage of the parameters when fitting simulated stationary data. The approach is applied to a dataset of annual precipitation in southern Norway and the scheme for selecting the hyperparameters leads to concervative estimates of non-stationarity and improved predictive performance over the stationary model.

stat.ME↗
arXiv2017

Using stacking to average Bayesian predictive distributions

The widely recommended procedure of Bayesian model averaging is flawed in the M-open setting in which the true data-generating process is not one of the candidate models being fit. We take the idea of stacking from the point estimation literature and generalize to the combination of predictive distributions, extending the utility function to any proper scoring rule, using Pareto smoothed importance sampling to efficiently compute the required leave-one-out posterior distributions and regularization to get more stability. We compare stacking of predictive distributions to several alternatives: stacking of means, Bayesian model averaging (BMA), pseudo-BMA using AIC-type weighting, and a variant of pseudo-BMA that is stabilized using the Bayesian bootstrap. Based on simulations and real-data applications, we recommend stacking of predictive distributions, with BB-pseudo-BMA as an approximate alternative when computation cost is an issue.

stat.ME↗
arXiv2017

The prior can generally only be understood in the context of the likelihood

A key sticking point of Bayesian analysis is the choice of prior distribution, and there is a vast literature on potential defaults including uniform priors, Jeffreys' priors, reference priors, maximum entropy priors, and weakly informative priors. These methods, however, often manifest a key conceptual tension in prior modeling: a model encoding true prior information should be chosen without reference to the model of the measurement process, but almost all common prior modeling techniques are implicitly motivated by a reference likelihood. In this paper we resolve this apparent paradox by placing the choice of prior into the context of the entire Bayesian analysis, from inference to prediction to model evaluation.

stat.ME↗
arXiv2016

Spatial Modeling, with Application to Complex Survey Data: Discussion of "Model-based Geostatistics for Prevalence Mapping in Low-Resource Settings", by Diggle and Giorgi

Prevalence mapping in low resource settings is an increasingly important endeavor to guide policy making and to spatially and temporally characterize the burden of disease. We will focus our discussion on consideration of the complex design when analyzing survey data, and on spatial modeling. With respect to the former, we consider two approaches: direct use of the weights, and a model-based approach using a spatial model to acknowledge clustering. For the latter we consider continuously indexed Markovian Gaussian random field models.

stat.ME↗
arXiv2016

An intuitive Bayesian spatial model for disease mapping that accounts for scaling

In recent years, disease mapping studies have become a routine application within geographical epidemiology and are typically analysed within a Bayesian hierarchical model formulation. A variety of model formulations for the latent level have been proposed but all come with inherent issues. In the classical BYM model, the spatially structured component cannot be seen independently from the unstructured component. This makes prior definitions for the hyperparameters of the two random effects challenging. There are alternative model formulations that address this confounding, however, the issue on how to choose interpretable hyperpriors is still unsolved. Here, we discuss a recently proposed parameterisation of the BYM model that leads to improved parameter control as the hyperparameters can be seen independently from each other. Furthermore, the need for a scaled spatial component is addressed, which facilitates assignment of interpretable hyperpriors and make these transferable between spatial applications with different graph structures. We provide implementation details for the new model formulation which preserve sparsity properties, and we investigate systematically the model performance and compare it to existing parameterisations. Through a simulation study, we show that the new model performs well, both showing good learning abilities and good shrinkage behaviour. In terms of model choice criteria, the proposed model performs at least equally well as existing parameterisations, but only the new formulation offers parameters that are interpretable and hyperpriors that have a clear meaning.

stat.ME↗
arXiv2015

On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoods

A large number of statistical models are "doubly-intractable": the likelihood normalising term, which is a function of the model parameters, is intractable, as well as the marginal likelihood (model evidence). This means that standard inference techniques to sample from the posterior, such as Markov chain Monte Carlo (MCMC), cannot be used. Examples include, but are not confined to, massive Gaussian Markov random fields, autologistic models and Exponential random graph models. A number of approximate schemes based on MCMC techniques, Approximate Bayesian computation (ABC) or analytic approximations to the posterior have been suggested, and these are reviewed here. Exact MCMC schemes, which can be applied to a subset of doubly-intractable distributions, have also been developed and are described in this paper. As yet, no general method exists which can be applied to all classes of models with doubly-intractable posteriors. In addition, taking inspiration from the Physics literature, we study an alternative method based on representing the intractable likelihood as an infinite series. Unbiased estimates of the likelihood can then be obtained by finite time stochastic truncation of the series via Russian Roulette sampling, although the estimates are not necessarily positive. Results from the Quantum Chromodynamics literature are exploited to allow the use of possibly negative estimates in a pseudo-marginal MCMC scheme such that expectations with respect to the posterior distribution are preserved. The methodology is reviewed on well-known examples such as the parameters in Ising models, the posterior for Fisher-Bingham distributions on the $d$-Sphere and a large-scale Gaussian Markov Random Field model describing the Ozone Column data. This leads to a critical assessment of the strengths and weaknesses of the methodology with pointers to ongoing research.

stat.ME↗
arXiv2015

Going off grid: Computationally efficient inference for log-Gaussian Cox processes

This paper introduces a new method for performing computational inference on log-Gaussian Cox processes. The likelihood is approximated directly by making novel use of a continuously specified Gaussian random field. We show that for sufficiently smooth Gaussian random field prior distributions, the approximation can converge with arbitrarily high order, while an approximation based on a counting process on a partition of the domain only achieves first-order convergence. The given results improve on the general theory of convergence of the stochastic partial differential equation models, introduced by Lindgren et al. (2011). The new method is demonstrated on a standard point pattern data set and two interesting extensions to the classical log-Gaussian Cox process framework are discussed. The first extension considers variable sampling effort throughout the observation window and implements the method of Chakraborty et al. (2011). The second extension constructs a log-Gaussian Cox process on the world's oceans. The analysis is performed using integrated nested Laplace approximation for fast approximate inference.

stat.CO↗
arXiv2015

Does non-stationary spatial data always require non-stationary random fields?

A stationary spatial model is an idealization and we expect that the true dependence structures of physical phenomena are spatially varying, but how should we handle this non-stationarity in practice? We study the challenges involved in applying a flexible non-stationary model to a dataset of annual precipitation in the conterminous US, where exploratory data analysis shows strong evidence of a non-stationary covariance structure. The aim of this paper is to investigate the modelling pipeline once non-stationarity has been detected in spatial data. We show that there is a real danger of over-fitting the model and that careful modelling is necessary in order to properly account for varying second-order structure. In fact, the example shows that sometimes non-stationary Gaussian random fields are not necessary to model non-stationary spatial data.

stat.ME↗
arXiv2015

The MCMC split sampler: A block Gibbs sampling scheme for latent Gaussian models

A novel computationally efficient Markov chain Monte Carlo (MCMC) scheme for latent Gaussian models (LGMs) is proposed in this paper. The sampling scheme is a two block Gibbs sampling scheme designed to exploit the model structure of LGMs. We refer to the proposed sampling scheme as the MCMC split sampler. The principle idea behind the MCMC split sampler is to split the latent Gaussian parameters into two vectors. The former vector consists of latent parameters which appear in the data density function, while the latter vector consists of latent parameters which do not appear in it. The former vector is placed in the first block of the proposed sampling scheme and the latter vector is placed in the second block along with any potential hyperparameters. The resulting conditional posterior density functions within the blocks allow the MCMC split sampler to handle, by design, LGMs with latent models imposed on more than just the mean structure of the data density function. The MCMC split sampler is also designed to be applicable for any choice of a parametric data density function. Moreover, it scales well in terms of computational efficiency when the dimension of the latent model increase.

stat.CO↗
arXiv2015

Spatial Modelling of Temperature and Humidity using Systems of Stochastic Partial Differential Equations

This work is motivated by constructing a weather simulator for precipitation. Temperature and humidity are two of the most important driving forces of precipitation, and the strategy is to have a stochastic model for temperature and humidity, and use a deterministic model to go from these variables to precipitation. Temperature and humidity are empirically positively correlated. Generally speaking, if variables are empirically dependent, then multivariate models should be considered. In this work we model humidity and temperature in southern Norway. We want to construct bivariate Gaussian random fields (GRFs) based on this dataset. The aim of our work is to use the bivariate GRFs to capture both the dependence structure between humidity and temperature as well as their spatial dependencies. One important feature for the dataset is that the humidity and temperature are not necessarily observed at the same locations. Both univariate and bivariate spatial models are fitted and compared. For modeling and inference the SPDE approach for univariate models and the systems of SPDEs approach for multivariate models have been used. To evaluate the performance of the difference between the univariate and bivariate models, we compare predictive performance using some commonly used scoring rules: mean absolute error, mean-square error and continuous ranked probability score. The results illustrate that we can capture strong positive correlation between the temperature and the humidity. Furthermore, the results also agree with the physical or empirical knowledge. At the end, we conclude that using the bivariate GRFs to model this dataset is superior to the approach with independent univariate GRFs both when evaluating point predictions and for quantifying prediction uncertainty.

stat.AP↗
arXiv2014

Computationally efficient spatial modeling of annual maximum 24 hour precipitation. An application to data from Iceland

We propose a computationally efficient statistical method to obtain distributional properties of annual maximum 24 hour precipitation on a 1 km by 1 km regular grid over Iceland. A latent Gaussian model is built which takes into account observations, spatial variations and outputs from a local meteorological model. A covariate based on the meteorological model is constructed at each observational site and each grid point in order to assimilate available scientific knowledge about precipitation into the statistical model. The model is applied to two data sets on extreme precipitation, one uncorrected data set and one data set that is corrected for phase and wind. The observations are assumed to follow the generalized extreme value distribution. At the latent level, we implement SPDE spatial models for both the location and scale parameters of the likelihood. An efficient MCMC sampler which exploits the model structure is constructed, which yields fast continuous spatial predictions for spatially varying model parameters and quantiles.

stat.ME↗
arXiv2014

Exploring a New Class of Non-stationary Spatial Gaussian Random Fields with Varying Local Anisotropy

Gaussian random fields (GRFs) constitute an important part of spatial modelling, but can be computationally infeasible for general covariance structures. An efficient approach is to specify GRFs via stochastic partial differential equations (SPDEs) and derive Gaussian Markov random field (GMRF) approximations of the solutions. We consider the construction of a class of non-stationary GRFs with varying local anisotropy, where the local anisotropy is introduced by allowing the coefficients in the SPDE to vary with position. This is done by using a form of diffusion equation driven by Gaussian white noise with a spatially varying diffusion matrix. This allows for the introduction of parameters that control the GRF by parametrizing the diffusion matrix. These parameters and the GRF may be considered to be part of a hierarchical model and the parameters estimated in a Bayesian framework. The results show that the use of an SPDE with non-constant coefficients is a promising way of creating non-stationary spatial GMRFs that allow for physical interpretability of the parameters, although there are several remaining challenges that would need to be solved before these models can be put to general practical use.

stat.ME↗
arXiv2013

Multivariate Gaussian Random Fields Using Systems of Stochastic Partial Differential Equations

In this paper a new approach for constructing \emph{multivariate} Gaussian random fields (GRFs) using systems of stochastic partial differential equations (SPDEs) has been introduced and applied to simulated data and real data. By solving a system of SPDEs, we can construct multivariate GRFs. On the theoretical side, the notorious requirement of non-negative definiteness for the covariance matrix of the GRF is satisfied since the constructed covariance matrices with this approach are automatically symmetric positive definite. Using the approximate stochastic weak solutions to the systems of SPDEs, multivariate GRFs are represented by multivariate Gaussian \emph{Markov} random fields (GMRFs) with sparse precision matrices. Therefore, on the computational side, the sparse structures make it possible to use numerical algorithms for sparse matrices to do fast sampling from the random fields and statistical inference. Therefore, the \emph{big-n} problem can also be partially resolved for these models. These models out-preform existing multivariate GRF models on a commonly used real dataset.

stat.ME↗