arXiv subjects

Dootika Vats

Publications and source records attributed to Dootika Vats.

At least 19 recordsLinked to original sources

arXiv2026

Scalable Bayesian inference for high-dimensional mixed-type multivariate spatial data

Spatial generalized linear mixed-effects models are popularly used to analyze spatially indexed univariate responses. However, with modern technology, it is common to observe vector-valued mixed-type responses, e.g., a combination of binary, count, or continuous types, at each location. Methods for jointly modeling such mixed-type multivariate spatial responses are rare. Using multivariate Gaussian processes (GPs) in the latent layer, we present a class of Bayesian spatial methods applicable to any combination of exponential family responses. Since multivariate GP-based methods can suffer from computational bottlenecks when the number of spatial locations is high, we further employ a computationally efficient Vecchia approximation for fast posterior inference and prediction. Key theoretical properties of the proposed model, such as identifiability and the structure of the induced covariance, are established. Our approach employs a Markov chain Monte Carlo-based inference method that uses elliptical slice sampling within a blocked Metropolis-within-Gibbs sampling framework. We illustrate the efficacy of the proposed method through simulation studies and a real-data application on joint modeling of wildfire counts and burnt areas across the United States.

stat.ME↗

arXiv2026

Exact MCMC for Intractable Proposals

Accept-reject based Markov chain Monte Carlo (MCMC) methods are the workhorse algorithm for Bayesian inference. These algorithms, like Metropolis-Hastings, require choosing a proposal distribution which is typically informed by the desired target distribution. Surprisingly, proposal distributions with unknown normalizing constants are not uncommon, even though for such a choice of a proposal, the Metropolis-Hastings acceptance ratio cannot be evaluated exactly. Across the literature, authors resort to approximation methods that yield inexact MCMC or develop specialized algorithms to combat this problem. We show how Bernoulli factory MCMC algorithms, originally proposed for doubly intractable target distributions, can quite naturally be adapted to yield an exact MCMC sampling method. We present three diverse and relevant examples demonstrating the usefulness of the Bernoulli factory approach to this problem.

stat.CO↗

arXiv2026

Hamiltonian Monte Carlo for (Physics) Dummies

Sampling-based inference has seen a surge of interest in recent years. Hamiltonian Monte Carlo (HMC) has emerged as a powerful algorithm that leverages concepts from Hamiltonian dynamics to efficiently explore complex target distributions. Variants of HMC are available in popular software packages, enabling off-the-shelf implementations that have greatly benefited the statistics and machine learning communities. At the same time, the availability of such black-box implementations has made it challenging for users to understand the inner workings of HMC, especially when they are unfamiliar with the underlying physical principles. We provide a pedagogical overview of HMC that aims to bridge the gap between its theoretical foundations and practical applicability. This review article seeks to make HMC more accessible to applied researchers by highlighting its advantages, limitations, and role in enabling scalable and exact Bayesian inference for complex models.

stat.CO↗

arXiv2025

Solving the Poisson equation using coupled Markov chains

This article shows how coupled Markov chains that meet exactly after a random number of iterations can be used to generate unbiased estimators of the solutions of the Poisson equation. Through this connection, we re-derive known unbiased estimators of expectations with respect to the stationary distribution of a Markov chain and provide conditions for the finiteness of their moments. We further construct unbiased estimators of the asymptotic variance of Markov chain ergodic averages, and provide conditions for the finiteness of the estimators' moments of any order. If their second moment is finite, the average of independent copies of such estimators converges to the asymptotic variance at the Monte Carlo rate, comparing favorably to known rates for batch means and spectral variance estimators. The results are illustrated with numerical experiments.

stat.CO↗

arXiv2025

Proximal Hamiltonian Monte Carlo

Bayesian formulation of modern day signal processing problems has called for improved Markov chain Monte Carlo (MCMC) sampling algorithms for inference. The need for efficient sampling techniques has become indispensable for high dimensional distributions that often characterize many core signal processing problems, e.g., image denoising, sparse signal recovery, etc. A major issue in building effective sampling strategies, however, is the non-differentiability of the underlying posterior density. Such posteriors are popular in models designed to recover sparse signals. As a result, the use of efficient gradient-based MCMC sampling techniques becomes difficult. We circumvent this problem by proposing a Proximal Hamiltonian Monte Carlo (p-HMC) algorithm, which leverages elements from convex optimization like proximal mappings and Moreau-Yosida (MY) envelopes within Hamiltonian dynamics. Our method improves upon the current state of the art non-smooth Hamiltonian Monte Carlo as it achieves a relatively sharper approximation of the gradient of log posterior density and a computational burden of at most the current state-of-the-art. A chief contribution of this work is the theoretical analysis of p-HMC. We provide conditions for geometric ergodicity of the underlying HMC chain. On the practical front, we propose guidance on choosing the key p-HMC hyperparameter -- the regularization parameter in the MY-envelope. We demonstrate p-HMC's efficiency over other MCMC algorithms on benchmark problems of logistic regression and low-rank matrix estimation.

stat.CO↗

arXiv2025

MCMC Importance Sampling via Moreau-Yosida Envelopes

Non-differentiable priors are standard in modern parsimonious Bayesian models. Lack of differentiability, however, precludes gradient-based Markov chain Monte Carlo (MCMC) for posterior sampling. Recently proposed proximal MCMC approaches can partially remedy this limitation by using a differentiable approximation, constructed via Moreau-Yosida (MY) envelopes, to make proposals. In this work, we build an importance sampling paradigm by using the MY envelope as an importance distribution. Leveraging properties of the envelope, we establish asymptotic normality of the importance sampling estimator with an explicit expression for the asymptotic covariance matrix. Since the MY envelope density is smooth, it is amenable to gradient-based samplers. We provide sufficient conditions for geometric ergodicity of Metropolis-adjusted Langevin and Hamiltonian Monte Carlo algorithms, sampling from this importance distribution. Our numerical studies show that the proposed scheme can yield lower variance estimators compared to existing proximal MCMC alternatives, and is effective in low and high dimensions.

stat.CO↗

arXiv2025

Efficient Multivariate Initial Sequence Estimators for MCMC

Estimating Monte Carlo error is critical to valid simulation results in Markov chain Monte Carlo (MCMC) and initial sequence estimators were one of the first methods introduced for this. Over the last few years, focus has been on multivariate assessment of simulation error, and many multivariate generalizations of univariate methods have been developed. The multivariate initial sequence estimator is known to exhibit superior finite-sample performance compared to its competitors. However, the multivariate initial sequence estimator can be prohibitively slow, limiting its widespread use. We provide an efficient alternative to the multivariate initial sequence estimator that inherits both its asymptotic properties as well as the finite-sample superior performance. The effectiveness of the proposed estimator is shown via some MCMC example implementations. Further, we also present univariate and multivariate initial sequence estimators for when parallel MCMC chains are run and demonstrate their effectiveness over a popular alternative.

stat.CO↗

arXiv2025

On the Utility of Equal Batch Sizes for Inference in Stochastic Gradient Descent

Stochastic gradient descent (SGD) is an estimation tool for large data employed in machine learning and statistics. Due to the Markovian nature of the SGD process, inference is a challenging problem. An underlying asymptotic normality of the averaged SGD (ASGD) estimator allows for the construction of a batch-means estimator of the asymptotic covariance matrix. Instead of the usual increasing batch-size strategy, we propose a memory efficient equal batch-size strategy and show that under mild conditions, the batch-means estimator is consistent. A key feature of the proposed batching technique is that it allows for bias-correction of the variance, at no additional cost to memory. Further, since joint inference for large dimensional problems may be undesirable, we present marginal-friendly simultaneous confidence intervals, and show through an example on how covariance estimators of ASGD can be employed for improved predictions.

stat.CO↗

arXiv2024

Estimating Monte Carlo variance from multiple Markov chains

Modern computational advances have enabled easy parallel implementations of Markov chain Monte Carlo (MCMC). However, almost all work in estimating the variance of Monte Carlo averages, including the efficient batch means (BM) estimator, focuses on a single-chain MCMC run. We demonstrate that simply averaging covariance matrix estimators from multiple chains can yield critical underestimates in small Monte Carlo sample sizes, especially for slow-mixing Markov chains. We extend the work of \cite{arg:and:2006} and propose a multivariate replicated batch means (RBM) estimator that utilizes information from parallel chains, thereby correcting for the underestimation. Under weak conditions on the mixing rate of the process, RBM is strongly consistent and exhibits similar large-sample bias and variance to the BM estimator. We also exhibit superior theoretical properties of RBM by showing that the (negative) bias in the RBM estimator is less than the average BM estimator in the presence of positive correlation in MCMC. Consequently, in small runs, the RBM estimator can be dramatically superior and this is demonstrated through a variety of examples.

stat.ME↗

arXiv2023

Multivariate strong invariance principles in Markov chain Monte Carlo

Strong invariance principles in Markov chain Monte Carlo are crucial to theoretically grounded output analysis. Using the wide-sense regenerative nature of the process, we obtain explicit bounds in the strong invariance converging rates for partial sums of multivariate ergodic Markov chains. Consequently, we present results on the existence of strong invariance principles for both polynomially and geometrically ergodic Markov chains without requiring a 1-step minorization condition. Our tight and explicit rates have a direct impact on output analysis, as it allows the verification of important conditions in the strong consistency of certain variance estimators.

stat.CO↗

arXiv2022

Understanding Linchpin Variables in Markov Chain Monte Carlo

An introduction to the use of linchpin variables in Markov chain Monte Carlo (MCMC) is provided. Before the widespread adoption of MCMC methods, conditional sampling using linchpin variables was essentially the only practical approach for simulating from multivariate distributions. With the advent of MCMC, linchpin variables were largely ignored. However, there has been a resurgence of interest in using them in conjunction with MCMC methods and there are good reasons for doing so. A simple derivation of the method is provided, its validity, benefits, and limitations are discussed, and some examples in the research literature are presented.

stat.CO↗

arXiv2022

A principled stopping rule for importance sampling

Importance sampling (IS) is a Monte Carlo technique that relies on weighted samples, simulated from a proposal distribution, to estimate intractable integrals. The quality of the estimators improves with the number of samples. However, for achieving a desired quality of estimation, the required number of samples is unknown and depends on the quantity of interest, the estimator, and the chosen proposal. We present a sequential stopping rule that terminates simulation when the overall variability in estimation is relatively small. The proposed methodology closely connects to the idea of an effective sample size in IS and overcomes crucial shortcomings of existing metrics, e.g., it acknowledges multivariate estimation problems. Our stopping rule retains asymptotic guarantees and provides users a clear guideline on when to stop the simulation in IS.

stat.CO↗

arXiv2022

Dimension-free Mixing for High-dimensional Bayesian Variable Selection

Yang et al. (2016) proved that the symmetric random walk Metropolis--Hastings algorithm for Bayesian variable selection is rapidly mixing under mild high-dimensional assumptions. We propose a novel MCMC sampler using an informed proposal scheme, which we prove achieves a much faster mixing time that is independent of the number of covariates, under the same assumptions. To the best of our knowledge, this is the first high-dimensional result which rigorously shows that the mixing rate of informed MCMC methods can be fast enough to offset the computational cost of local posterior evaluation. Motivated by the theoretical analysis of our sampler, we further propose a new approach called "two-stage drift condition" to studying convergence rates of Markov chains on general state spaces, which can be useful for obtaining tight complexity bounds in high-dimensional settings. The practical advantages of our algorithm are illustrated by both simulation studies and real data analysis.

stat.ME↗

arXiv2022

Optimal Scaling of MCMC Beyond Metropolis

The problem of optimally scaling the proposal distribution in a Markov chain Monte Carlo algorithm is critical to the quality of the generated samples. Much work has gone into obtaining such results for various Metropolis-Hastings (MH) algorithms. Recently, acceptance probabilities other than MH are being employed in problems with intractable target distributions. There is little resource available on tuning the Gaussian proposal distributions for this situation. We obtain optimal scaling results for a general class of acceptance functions, which includes Barker's and Lazy-MH. In particular, optimal values for the Barker's algorithm are derived and found to be significantly different from that obtained for the MH algorithm. Our theoretical conclusions are supported by numerical simulations indicating that when the optimal proposal variance is unknown, tuning to the optimal acceptance probability remains an effective strategy.

stat.CO↗

arXiv2021

Globally-centered autocovariances in MCMC

Autocovariances are a fundamental quantity of interest in Markov chain Monte Carlo (MCMC) simulations with autocorrelation function (ACF) plots being an integral visualization tool for performance assessment. Unfortunately, for slow-mixing Markov chains, the empirical autocovariance can highly underestimate the truth. For multiple-chain MCMC sampling, we propose a globally-centered estimator of the autocovariance function (G-ACvF) that exhibits significant theoretical and empirical improvements. We show that the bias of the G-ACvF estimator is smaller than the bias of the current state-of-the-art. The impact of this improved estimator is evident in three critical output analysis applications: (1) ACF plots, (2) estimates of the Monte Carlo asymptotic covariance matrix, and (3) estimates of the effective sample size. Under weak conditions, we establish strong consistency of our improved asymptotic covariance estimator, and obtain its large-sample bias and variance. The performance of the new estimators is demonstrated through various examples.

stat.CO↗

arXiv2021

Lugsail lag windows for estimating time-average covariance matrices

Lag windows are commonly used in time series, econometrics, steady-state simulation, and Markov chain Monte Carlo to estimate time-average covariance matrices. In the presence of positive correlation of the underlying process, estimators of this matrix almost always exhibit significant negative bias, leading to undesirable finite-sample properties. We propose a new family of lag windows specifically designed to improve finite-sample performance by offsetting this negative bias. Any existing lag window can be adapted into a lugsail equivalent with no additional assumptions. We use these lag windows within spectral variance estimators and demonstrate its advantages in a linear regression model with autocorrelated and heteroskedastic residuals. We further employ the lugsail lag windows in weighted batch means estimators due to their computational efficiency on large simulation output. We obtain bias and variance results for these multivariate estimators and significantly weaken the mixing condition on the process. Superior finite-sample properties are illustrated in a vector autoregressive process and a Bayesian logistic regression model.

stat.CO↗

arXiv2021

Efficient Bernoulli factory MCMC for intractable posteriors

Accept-reject based Markov chain Monte Carlo (MCMC) algorithms have traditionally utilised acceptance probabilities that can be explicitly written as a function of the ratio of the target density at the two contested points. This feature is rendered almost useless in Bayesian posteriors with unknown functional forms. We introduce a new family of MCMC acceptance probabilities that has the distinguishing feature of not being a function of the ratio of the target density at the two points. We present two stable Bernoulli factories that generate events within this class of acceptance probabilities. The efficiency of our methods rely on obtaining reasonable local upper or lower bounds on the target density and we present two classes of problems where such bounds are viable: Bayesian inference for diffusions and MCMC on constrained spaces. The resulting portkey Barker's algorithms are exact and computationally more efficient that the current state-of-the-art.

stat.CO↗

arXiv2021

Variational Rejection Particle Filtering

We present a variational inference (VI) framework that unifies and leverages sequential Monte-Carlo (particle filtering) with \emph{approximate} rejection sampling to construct a flexible family of variational distributions. Furthermore, we augment this approach with a resampling step via Bernoulli race, a generalization of a Bernoulli factory, to obtain a low-variance estimator of the marginal likelihood. Our framework, Variational Rejection Particle Filtering (VRPF), leads to novel variational bounds on the marginal likelihood, which can be optimized efficiently with respect to the variational parameters and generalizes several existing approaches in the VI literature. We also present theoretical properties of the variational bound and demonstrate experiments on various models of sequential data, such as the Gaussian state-space model and variational recurrent neural net (VRNN), on which VRPF outperforms various existing state-of-the-art VI methods.

cs.LG↗