Searcharxiv⌕ Search

arXiv subjects

Somak Dutta

Publications and source records attributed to Somak Dutta.

14 recordsLinked to original sources
arXiv2025

Transfers and Unstable Degrees in the $\mathbb{A}^{1}$-Brouwer Degrees Package for Macaulay2

We describe a significant update to the Macaulay2 package A1BrouwerDegrees. We extend several methods in the previous version of the package to the setting of finite \'{e}tale algebras, allowing the computation of transfers along finite \'{e}tale extensions. Additionally, we implement a number of new features for the computation of unstable $\mathbb{A}^{1}$-Brouwer degrees and manipulation of classes in the unstable Grothendieck--Witt group.

math.AG↗
arXiv2025

Ridge partial correlation screening for ultrahigh-dimensional data

Variable selection in ultrahigh-dimensional linear regression is challenging due to its high computational cost. Therefore, a screening step is usually conducted before variable selection to significantly reduce the dimension. Here we propose a novel and simple screening method based on ordering the absolute sample ridge partial correlations. The proposed method takes into account not only the ridge regularized estimates of the regression coefficients but also the ridge regularized partial variances of the predictor variables providing sure screening property without strong assumptions on the marginal correlations. Simulation study and a real data analysis show that the proposed method has a competitive performance compared with the existing screening procedures. A publicly available software implementing the proposed screening accompanies the article.

stat.ME↗
arXiv2023

Fast matrix-free methods for model-based personalized synthetic MR imaging

Synthetic Magnetic Resonance (MR) imaging predicts images at new design parameter settings from a few observed MR scans. Model-based methods, that use both the physical and statistical properties underlying the MR signal and its acquisition, can predict images at any setting from as few as three scans, allowing it to be used in individualized patient- and anatomy-specific contexts. However, the estimation problem in model-based synthetic MR imaging is ill-posed and so regularization, in the form of correlated Gaussian Markov Random Fields, is imposed on the voxel-wise spin-lattice relaxation time, spin-spin relaxation time and the proton density underlying the MR image. We develop theoretically sound but computationally practical matrix-free estimation methods for synthetic MR imaging. Our evaluations demonstrate superior performance of our methods in currently-used clinical settings when compared to existing model-based and deep learning methods. Moreover, unlike deep learning approaches, our fast methodology can synthesize needed images during patient visits, with good estimation and prediction accuracy and consistency. An added strength of our model-based approach, also developed and illustrated here, is the accurate estimation of standard errors of regional contrasts in the synthesized images. A R package $symr$ implements our methodology.

stat.AP↗
arXiv2022

On the usefulness of lattice approximations for fractional Gaussian fields

Fractional Gaussian fields provide a rich class of spatial models and have a long history of applications in multiple branches of science. However, estimation and inference for fractional Gaussian fields present significant challenges. This book chapter investigates the use of the fractional Laplacian differencing on regular lattices to approximate to continuum fractional Gaussian fields. Emphasis is given on model based geostatistics and likelihood based computations. For a certain range of the fractional parameter, we demonstrate that there is considerable agreement between the continuum models and their lattice approximations. For that range, the parameter estimates and inferences about the continuum fractional Gaussian fields can be derived from the lattice approximations. Interestingly, regular lattice approximations facilitate fast matrix-free computations and enable anisotropic representations. We illustrate the usefulness of lattice approximations via simulation studies and by analyzing sea surface temperature on the Indian Ocean.

stat.ME↗
arXiv2021

Exploratory Factor Analysis of Data on a Sphere

Data on high-dimensional spheres arise frequently in many disciplines either naturally or as a consequence of preliminary processing and can have intricate dependence structure that needs to be understood. We develop exploratory factor analysis of the projected normal distribution to explain the variability in such data using a few easily interpreted latent factors. Our methodology provides maximum likelihood estimates through a novel fast alternating expectation profile conditional maximization algorithm. Results on simulation experiments on a wide range of settings are uniformly excellent. Our methodology provides interpretable and insightful results when applied to tweets with the $\#MeToo$ hashtag in early December 2018, to time-course functional Magnetic Resonance Images of the average pre-teen brain at rest, to characterize handwritten digits, and to gene expression data from cancerous cells in the Cancer Genome Atlas.

stat.ME↗
arXiv2021

Bayesian iterative screening in ultra-high dimensional linear regressions

Variable selection in ultra-high dimensional linear regression is often preceded by a screening step to significantly reduce the dimension. Here we develop a Bayesian variable screening method (BITS) guided by the posterior model probabilities. BITS can successfully integrate prior knowledge, if any, on effect sizes, and the number of true variables. BITS iteratively includes potential variables with the highest posterior probability accounting for the already selected variables. It is implemented by a fast Cholesky update algorithm and is shown to have the screening consistency property. BITS is built based on a model with Gaussian errors, yet, the screening consistency is proved to hold under more general tail conditions. The notion of posterior screening consistency allows the resulting model to provide a good starting point for further Bayesian variable selection methods. A new screening consistent stopping rule based on posterior probability is developed. Simulation studies and real data examples are used to demonstrate scalability and fine screening performance.

stat.ME↗
arXiv2020

A note on marginal correlation based screening

Independence screening methods such as the two sample $t$-test and the marginal correlation based ranking are among the most widely used techniques for variable selection in ultrahigh dimensional data sets. In this short note, simple examples are used to demonstrate potential problems with the independence screening methods in the presence of correlated predictors. Also, an example is considered where all important variables are independent among themselves and all but one important variables are independent with the unimportant variables. Furthermore, a real data example from a genome wide association study is used to illustrate inferior performance of marginal correlation screening compared to another screening method.

stat.ME↗
arXiv2020

Model Based Screening Embedded Bayesian Variable Selection for Ultra-high Dimensional Settings

We develop a Bayesian variable selection method, called SVEN, based on a hierarchical Gaussian linear model with priors placed on the regression coefficients as well as on the model space. Sparsity is achieved by using degenerate spike priors on inactive variables, whereas Gaussian slab priors are placed on the coefficients for the important predictors making the posterior probability of a model available in explicit form (up to a normalizing constant). The strong model selection consistency is shown to be attained when the number of predictors grows nearly exponentially with the sample size and even when the norm of mean effects solely due to the unimportant variables diverge, which is a novel attractive feature. An appealing byproduct of SVEN is the construction of novel model weight adjusted prediction intervals. Embedding a unique model based screening and using fast Cholesky updates, SVEN produces a highly scalable computational framework to explore gigantic model spaces, rapidly identify the regions of high posterior probabilities and make fast inference and prediction. A temperature schedule guided by our model selection consistency derivations is used to further mitigate multimodal posterior distributions. The performance of SVEN is demonstrated through a number of simulation experiments and a real data example from a genome wide association study with over half a million markers.

stat.ME↗
arXiv2020

Adjusting for Spatial Effects in Genomic Prediction

This paper investigates the problem of adjusting for spatial effects in genomic prediction. Despite being seldomly considered in genomic prediction, spatial effects often affect phenotypic measurements of plants. We consider a Gaussian random field model with an additive covariance structure that incorporates genotype effects, spatial effects and subpopulation effects. An empirical study shows the existence of spatial effects and heterogeneity across different subpopulation families, while simulations illustrate the improvement in selecting genotypically superior plants by adjusting for spatial effects in genomic prediction.

stat.AP↗
arXiv2019

A Matrix--free Likelihood Method for Exploratory Factor Analysis of High-dimensional Gaussian Data

This paper proposes a novel profile likelihood method for estimating the covariance parameters in exploratory factor analysis of high-dimensional Gaussian datasets with fewer observations than number of variables. An implicitly restarted Lanczos algorithm and a limited-memory quasi-Newton method are implemented to develop a matrix-free framework for likelihood maximization. Simulation results show that our method is substantially faster than the expectation-maximization solution without sacrificing accuracy. Our method is applied to fit factor models on data from suicide attempters, suicide ideators and a control group.

stat.ME↗
arXiv2017

A novel sandwich algorithm for empirical Bayes analysis of rank data

Rank data arises frequently in marketing, finance, organizational behavior, and psychology. Most analysis of rank data reported in the literature assumes the presence of one or more variables (sometimes latent) based on whose values the items are ranked. In this paper we analyze rank data using a purely probabilistic model where the observed ranks are assumed to be perturbed versions of the true rank and each perturbation has a specific probability of occurring. We consider the general case when covariate information is present and has an impact on the rankings. An empirical Bayes approach is taken for estimating the model parameters. The Gibbs sampler is shown to converge very slowly to the target posterior distribution and we show that some of the widely used empirical convergence diagnostic tools may fail to detect this lack of convergence. We propose a novel, fast mixing sandwich algorithm for exploring the posterior distribution. An EM algorithm based on Markov chain Monte Carlo (MCMC) sampling is developed for estimating prior hyperparameters. A real life rank data set is analyzed using the methods developed in the paper. The results obtained indicate the usefulness of these methods in analyzing rank data with covariate information.

stat.ME↗
arXiv2013

Markov Chain Monte Carlo Based on Deterministic Transformations

In this article we propose a novel MCMC method based on deterministic transformations T: X x D --> X where X is the state-space and D is some set which may or may not be a subset of X. We refer to our new methodology as Transformation-based Markov chain Monte Carlo (TMCMC). One of the remarkable advantages of our proposal is that even if the underlying target distribution is very high-dimensional, deterministic transformation of a one-dimensional random variable is sufficient to generate an appropriate Markov chain that is guaranteed to converge to the high-dimensional target distribution. Apart from clearly leading to massive computational savings, this idea of deterministically transforming a single random variable very generally leads to excellent acceptance rates, even though all the random variables associated with the high-dimensional target distribution are updated in a single block. Since it is well-known that joint updating of many random variables using Metropolis-Hastings (MH) algorithm generally leads to poor acceptance rates, TMCMC, in this regard, seems to provide a significant advance. We validate our proposal theoretically, establishing the convergence properties. Furthermore, we show that TMCMC can be very effectively adopted for simulating from doubly intractable distributions. TMCMC is compared with MH using the well-known Challenger data, demonstrating the effectiveness of of the former in the case of highly correlated variables. Moreover, we apply our methodology to a challenging posterior simulation problem associated with the geostatistical model of Diggle et al. (1998), updating 160 unknown parameters jointly, using a deterministic transformation of a one-dimensional random variable. Remarkable computational savings as well as good convergence properties and acceptance rates are the results.

stat.CO↗
arXiv2011

Multiplicative random walk Metropolis-Hastings on the real line

In this article we propose multiplication based random walk Metropolis Hastings (MH) algorithm on the real line. We call it the random dive MH (RDMH) algorithm. This algorithm, even if simple to apply, was not studied earlier in Markov chain Monte Carlo literature. The associated kernel is shown to have standard properties like irreducibility, aperiodicity and Harris recurrence under some mild assumptions. These ensure basic convergence (ergodicity) of the kernel. Further the kernel is shown to be geometric ergodic for a large class of target densities on $\mathbb{R}$. This class even contains realistic target densities for which random walk or Langevin MH are not geometrically ergodic. Three simulation studies are given to demonstrate the mixing property and superiority of RDMH to standard MH algorithms on real line. A share-price return data is also analyzed and the results are compared with those available in the literature.

stat.CO↗