arXiv subjects

Shamriddha De

Publications and source records attributed to Shamriddha De.

3 recordsLinked to original sources

arXiv2026

Posterior Mode-Guided Dimension Reduction for Bayesian Model Averaging in Heavy-Tailed Linear Regression

For large model spaces in linear regression with spike-and-slab priors, the potential entrapment of Markov chain Monte Carlo (MCMC)-based methods poses significant challenges in posterior computation. Existing maximum a posteriori (MAP)-based methods provide more computationally viable alternatives, but fail to perform tail heaviness estimation and uncertainty quantification. To address these problems, we propose a method that blends MAP estimation with MCMC-based stochastic search algorithms within an error framework comprising a combination of the hyperbolic and Student-t distributions. The hyperbolic distribution has the light-tailed normal and heavy-tailed Laplace distributions as limiting cases, but is thinner-tailed than the Student-t family. Including the Student-t distribution in the error density enables better adaptation to heavier tails. Amalgamating the two error densities thus ensures a model with more flexible tail behavior when faced with unknown tail thickness in the data, compared to MAP estimators with fixed levels of tail heaviness that assume the errors have a normal or Laplace distribution. Under this proposed error model, the current work develops a two-step expectation conditional maximization (ECM)-guided MCMC algorithm. First, we conduct an ECM-based posterior maximization to guide variable selection. We then execute a Gibbs sampler on the resulting ECM-guided model space for tail heaviness estimation and uncertainty quantification. Through simulation studies and benchmark real datasets, our proposed method is shown to exhibit several advantages in variable selection and uncertainty quantification over state-of-the-art MAP-based methods. To implement our proposed method, we developed the R package FlexBayesReg, available at https://github.com/shamriddha1998/FlexBayesReg.

stat.ME↗

arXiv2025

An Online Algorithm for Bayesian Variable Selection in Logistic Regression Models With Streaming Data

In several modern applications, data are generated continuously over time, such as data generated from smartwatches. We assume data are collected and analyzed sequentially, in batches. Since traditional or offline methods can be extremely slow, Ghosh et al. (2025) proposed an online method for Bayesian model averaging (BMA). Inspired by the literature on renewable estimation, they developed an online Bayesian method for generalized linear models (GLMs) that reduces storage and computational demands dramatically compared to traditional methods for BMA. The method of Ghosh et al. (2025) works very well when the number of models is small. It can also work reasonably well in moderately large model spaces. For the latter case, the method relies on a screening stage to identify important models in the first several batches via offline methods. Thereafter, the model space remains fixed in all subsequent batches. In the post-screening stage, online updates are made to the model specific parameters, for models selected in the screening stage. For high-dimensional model spaces, the chance of missing important models in the screening stage is more likely. This necessitates the development of a method, which permits the model space to be updated as new batches of data arrive. In this article, we develop an online Bayesian model selection method for logistic regression, where the selected model can potentially change throughout the data collection process. We use simulation studies to show that our new method can outperform the method of Ghosh et al. (2025). Furthermore, we describe scenarios under which the gain from our new method is expected to be small. We revisit the traffic crash data analyzed by Ghosh et al. (2025) and illustrate that our new model selection method can have better performance for variable selection.

stat.ME↗

arXiv2024

Robust Bayesian Model Averaging for Linear Regression Models With Heavy-Tailed Errors

Our goal is to develop a Bayesian model averaging technique in linear regression models that accommodates heavier tailed error densities than the normal distribution. Motivated by the use of the Huber loss function in the presence of outliers, the Bayesian Huberized lasso with hyperbolic errors has been proposed and recently implemented in the literature (Park and Casella (2008); Kawakami and Hashimoto (2023)). Since the Huberized lasso cannot enforce regression coefficients to be exactly zero, we propose a fully Bayesian variable selection approach with spike and slab priors to address sparsity more effectively. The shapes of the hyperbolic and the Student-t density functions are different. Furthermore, the tails of a hyperbolic distribution are less heavy compared to those of a Cauchy distribution. Thus, we propose a flexible regression model with an error distribution encompassing both the hyperbolic and the Student-t family of distributions, along with an unknown tail heaviness parameter, that is estimated based on the data. It is known that the limiting form of both the hyperbolic and the Student-t distributions is a normal distribution. We develop an efficient Gibbs sampler with Metropolis Hastings steps for posterior computation. Through simulation studies and analyses of real datasets, we show that our method is competitive with various state-of-the-art methods.

stat.ME↗