Searcharxiv⌕ Search

arXiv subjects

James M. Flegal

Publications and source records attributed to James M. Flegal.

At least 19 recordsLinked to original sources
arXiv2026

Inference Optimal Long Run Variance Estimation with Lugsail Kernels

For datasets with unknown but stationary serial dependence, a robust long run variance estimator is essential to handle diverse scenarios. Spectral variance estimators are commonly used but tend to exhibit significant negative bias in the presence of positive correlation. To overcome this, zero lugsail estimators have been introduced, offering zero asymptotic bias regardless of the correlation structure. However, there are currently no guidelines for selecting the optimal bandwidth for lugsail estimators, a critical component in the estimation process. We propose an inference optimal bandwidth rule for lugsail estimators, based on nonstandard fixed-smoothing limiting distributions developed in our study. This approach significantly improves bias correction, accounts for variability, and provides an estimator optimized for robust inference. Our theoretical findings are supported by a simulation study.

math.ST↗
arXiv2024

Implementing MCMC: Multivariate estimation with confidence

This paper addresses the key challenge of estimating the asymptotic covariance associated with the Markov chain central limit theorem, which is essential for visualizing and terminating Markov Chain Monte Carlo (MCMC) simulations. We focus on summarizing batching, spectral, and initial sequence covariance estimation techniques. We emphasize practical recommendations for modern MCMC simulations, where positive correlation is common and leads to negatively biased covariance estimates. Our discussion is centered on computationally efficient methods that remain viable even when the number of iterations is large, offering insights into improving the reliability and accuracy of MCMC output in such scenarios.

stat.CO↗
arXiv2021

Lugsail lag windows for estimating time-average covariance matrices

Lag windows are commonly used in time series, econometrics, steady-state simulation, and Markov chain Monte Carlo to estimate time-average covariance matrices. In the presence of positive correlation of the underlying process, estimators of this matrix almost always exhibit significant negative bias, leading to undesirable finite-sample properties. We propose a new family of lag windows specifically designed to improve finite-sample performance by offsetting this negative bias. Any existing lag window can be adapted into a lugsail equivalent with no additional assumptions. We use these lag windows within spectral variance estimators and demonstrate its advantages in a linear regression model with autocorrelated and heteroskedastic residuals. We further employ the lugsail lag windows in weighted batch means estimators due to their computational efficiency on large simulation output. We obtain bias and variance results for these multivariate estimators and significantly weaken the mixing condition on the process. Superior finite-sample properties are illustrated in a vector autoregressive process and a Bayesian logistic regression model.

stat.CO↗
arXiv2020

Assessing and Visualizing Simultaneous Simulation Error

Monte Carlo experiments produce samples in order to estimate features of a given distribution. However, simultaneous estimation of means and quantiles has received little attention, despite being common practice. In this setting we establish a multivariate central limit theorem for any finite combination of sample means and quantiles under the assumption of a strongly mixing process, which includes the standard Monte Carlo and Markov chain Monte Carlo settings. We build on this to provide a fast algorithm for constructing hyperrectangular confidence regions having the desired simultaneous coverage probability and a convenient marginal interpretation. The methods are incorporated into standard ways of visualizing the results of Monte Carlo experiments enabling the practitioner to more easily assess the reliability of the results. We demonstrate the utility of this approach in various Monte Carlo settings including simulation studies based on independent and identically distributed samples and Bayesian analyses using Markov chain Monte Carlo sampling.

stat.CO↗
arXiv2019

Batch size selection for variance estimators in MCMC

We consider batch size selection for a general class of multivariate batch means variance estimators, which are computationally viable for high-dimensional Markov chain Monte Carlo simulations. We derive the asymptotic mean squared error for this class of estimators. Further, we propose a parametric technique for estimating optimal batch sizes and discuss practical issues regarding the estimating process. Vector auto-regressive, Bayesian logistic regression, and Bayesian dynamic space-time examples illustrate the quality of the estimation procedure where the proposed optimal batch sizes outperform current batch size selection methods.

math.ST↗
arXiv2018

Weighted batch means estimators in Markov chain Monte Carlo

This paper proposes a family of weighted batch means variance estimators, which are computationally efficient and can be conveniently applied in practice. The focus is on Markov chain Monte Carlo simulations and estimation of the asymptotic covariance matrix in the Markov chain central limit theorem, where conditions ensuring strong consistency are provided. Finite sample performance is evaluated through auto-regressive, Bayesian spatial-temporal, and Bayesian logistic regression examples, where the new estimators show significant computational gains with a minor sacrifice in variance compared with existing methods.

math.ST↗
arXiv2018

A Machine Learning Approach to Galaxy-LSS Classification I: Imprints on Halo Merger Trees

The cosmic web plays a major role in the formation and evolution of galaxies and defines, to a large extent, their properties. However, the relation between galaxies and environment is still not well understood. Here we present a machine learning approach to study imprints of environmental effects on the mass assembly of haloes. We present a galaxy-LSS machine learning classifier based on galaxy properties sensitive to the environment. We then use the classifier to assess the relevance of each property. Correlations between galaxy properties and their cosmic environment can be used to predict galaxy membership to void/wall or filament/cluster with an accuracy of $93\%$. Our study unveils environmental information encoded in properties of haloes not normally considered directly dependent on the cosmic environment such as merger history and complexity. Understanding the physical mechanism by which the cosmic web is imprinted in a halo can lead to significant improvements in galaxy formation models. This is accomplished by extracting features from galaxy properties and merger trees, computing feature scores for each feature and then applying support vector machine to different feature sets. To this end, we have discovered that the shape and depth of the merger tree, formation time and density of the galaxy are strongly associated with the cosmic environment. We describe a significant improvement in the original classification algorithm by performing LU decomposition of the distance matrix computed by the feature vectors and then using the output of the decomposition as input vectors for support vector machine.

astro-ph.GA↗
arXiv2017

Multivariate Output Analysis for Markov chain Monte Carlo

Markov chain Monte Carlo (MCMC) produces a correlated sample for estimating expectations with respect to a target distribution. A fundamental question is when should sampling stop so that we have good estimates of the desired quantities? The key to answering this question lies in assessing the Monte Carlo error through a multivariate Markov chain central limit theorem (CLT). The multivariate nature of this Monte Carlo error largely has been ignored in the MCMC literature. We present a multivariate framework for terminating simulation in MCMC. We define a multivariate effective sample size, estimating which requires strongly consistent estimators of the covariance matrix in the Markov chain CLT; a property we show for the multivariate batch means estimator. We then provide a lower bound on the number of minimum effective samples required for a desired level of precision. This lower bound depends on the problem only in the dimension of the expectation being estimated, and not on the underlying stochastic process. This result is obtained by drawing a connection between terminating simulation via effective sample size and terminating simulation using a relative standard deviation fixed-volume sequential stopping rule; which we demonstrate is an asymptotically valid procedure. The finite sample properties of the proposed method are demonstrated in a variety of examples.

math.ST↗
arXiv2016

Estimating standard errors for importance sampling estimators with multiple Markov chains

The naive importance sampling estimator, based on samples from a single importance density, can be numerically unstable. Instead, we consider generalized importance sampling estimators where samples from more than one probability distribution are combined. We study this problem in the Markov chain Monte Carlo context, where independent samples are replaced with Markov chain samples. If the chains converge to their respective target distributions at a polynomial rate, then under two finite moment conditions, we show a central limit theorem holds for the generalized estimators. Further, we develop an easy to implement method to calculate valid asymptotic standard errors based on batch means. We also provide a batch means estimator for calculating asymptotically valid standard errors of Geyer(1994) reverse logistic estimator. We illustrate the method using a Bayesian variable selection procedure in linear regression. In particular, the generalized importance sampling estimator is used to perform empirical Bayes variable selection and the batch means estimator is used to obtain standard errors in a high-dimensional setting where current methods are not applicable.

math.ST↗
arXiv2016

Strong Consistency of Multivariate Spectral Variance Estimators

Markov chain Monte Carlo (MCMC) algorithms are used to estimate features of interest of a distribution. The Monte Carlo error in estimation has an asymptotic normal distribution whose multivariate nature has so far been ignored in the MCMC community. We present a class of multivariate spectral variance estimators for the asymptotic covariance matrix in the Markov chain central limit theorem and provide conditions for strong consistency. We examine the finite sample properties of the multivariate spectral variance estimators and its eigenvalues in the context of a vector autoregressive process of order 1.

math.ST↗
arXiv2015

Bayesian model selection on linear mixed-effects models for comparisons between multiple treatments and a control

We propose a novel Bayesian model selection technique on linear mixed-effects models to compare multiple treatments with a control. A fully Bayesian approach is implemented to estimate the marginal inclusion probabilities that provide a direct measure of the difference between treatments and the control, along with the model-averaged posterior distributions. Default priors are proposed for model selection incorporating domain knowledge and a component-wise Gibbs sampler is developed for efficient posterior computation. We demonstrate the proposed method based on simulated data and an experimental dataset from a longitudinal study of mouse lifespan and weight trajectories.

stat.AP↗
arXiv2015

Bayesian inference for a flexible class of bivariate beta distributions

Several bivariate beta distributions have been proposed in the literature. In particular, Olkin and Liu (2003) proposed a 3 parameter bivariate beta model, which Arnold and Ng (2011) extend to 5 and 8 parameter models. The 3 parameter model allows for only positive correlation, while the latter models can accommodate both positive and negative correlation. However, these come at the expense of a density that is mathematically intractable. The focus of this research is on Bayesian estimation for the 5 and 8 parameter models. Since the likelihood does not exist in closed form, we apply approximate Bayesian computation, a likelihood free approach. Simulation studies have been carried out for the 5 and 8 parameter cases under various priors and tolerance levels. We apply the 5 parameter model to a real data set by allowing the model to serve as a prior to correlated proportions of a bivariate beta binomial model. Results and comparisons are then discussed.

stat.CO↗
arXiv2014

Markov Chain Monte Carlo Estimation of Quantiles

We consider quantile estimation using Markov chain Monte Carlo and establish conditions under which the sampling distribution of the Monte Carlo error is approximately Normal. Further, we investigate techniques to estimate the associated asymptotic variance, which enables construction of an asymptotically valid interval estimator. Finally, we explore the finite sample properties of these methods through examples and provide some recommendations to practitioners.

math.ST↗
arXiv2014

A practical sequential stopping rule for high-dimensional MCMC and its application to spatial-temporal Bayesian models

A current challenge for many Bayesian analyses is determining when to terminate high-dimensional Markov chain Monte Carlo simulations. To this end, we propose using an automated sequential stopping procedure that terminates the simulation when the computational uncertainty is small relative to the posterior uncertainty. Such a stopping rule has previously been shown to work well in settings with posteriors of moderate dimension. In this paper, we illustrate its utility in high-dimensional simulations while overcoming some current computational issues. Further, we investigate the relationship between the stopping rule and effective sample size. As examples, we consider two complex Bayesian analyses on spatially and temporally correlated datasets. The first involves a dynamic space-time model on weather station data and the second a spatial variable selection model on fMRI brain imaging data. Our results show the sequential stopping rule is easy to implement, provides uncertainty estimates, and performs well in high-dimensional settings.

stat.CO↗
arXiv2013

A Modified Gibbs Sampler on General State Spaces

We present a modified Gibbs sampler for general state spaces. We establish that this modification can lead to substantial gains in statistical efficiency while maintaining the overall quality of convergence. We illustrate our results in two examples including a toy Normal-Normal model and a Bayesian version of the random effects model.

stat.CO↗
arXiv2013

Relative fixed-width stopping rules for Markov chain Monte Carlo simulations

Markov chain Monte Carlo (MCMC) simulations are commonly employed for estimating features of a target distribution, particularly for Bayesian inference. A fundamental challenge is determining when these simulations should stop. We consider a sequential stopping rule that terminates the simulation when the width of a confidence interval is sufficiently small relative to the size of the target parameter. Specifically, we propose relative magnitude and relative standard deviation stopping rules in the context of MCMC. In each setting, we develop sufficient conditions for asymptotic validity, that is conditions to ensure the simulation will terminate with probability one and the resulting confidence intervals will have the proper coverage probability. Our results are applicable in a wide variety of MCMC estimation settings, such as expectation, quantile, or simultaneous multivariate estimation. Finally, we investigate the finite sample properties through a variety of examples and provide some recommendations to practitioners.

math.ST↗
arXiv2012

Exact sampling for intractable probability distributions via a Bernoulli factory

Many applications in the field of statistics require Markov chain Monte Carlo methods. Determining appropriate starting values and run lengths can be both analytically and empirically challenging. A desire to overcome these problems has led to the development of exact, or perfect, sampling algorithms which convert a Markov chain into an algorithm that produces i.i.d. samples from the stationary distribution. Unfortunately, very few of these algorithms have been developed for the distributions that arise in statistical applications, which typically have uncountable support. Here we study an exact sampling algorithm using a geometrically ergodic Markov chain on a general state space. Our work provides a significant reduction to the number of input draws necessary for the Bernoulli factory, which enables exact sampling via a rejection sampling approach. We illustrate the algorithm on a univariate Metropolis-Hastings sampler and a bivariate Gibbs sampler, which provide a proof of concept and insight into hyper-parameter selection. Finally, we illustrate the algorithm on a Bayesian version of the one-way random effects model with data from a styrene exposure study.

stat.CO↗