arXiv subjects

Subhashis Ghosal

Publications and source records attributed to Subhashis Ghosal.

At least 37 records · Page 2Linked to original sources

arXiv2019

Bayesian Inference on Multivariate Medians and Quantiles

In this paper, we consider Bayesian inference on a class of multivariate median and the multivariate quantile functionals of a joint distribution using a Dirichlet process prior. Since, unlike univariate quantiles, the exact posterior distribution of multivariate median and multivariate quantiles are not obtainable explicitly, we study these distributions asymptotically. We derive a Bernstein-von Mises theorem for the multivariate $\ell_1$-median with respect to general $\ell_p$-norm, which in particular shows that its posterior concentrates around its true value at $n^{-1/2}$-rate and its credible sets have asymptotically correct frequentist coverage. In particular, asymptotic normality results for the empirical multivariate median with general $\ell_p$-norm is also derived in the course of the proof which extends the results from the case $p=2$ in the literature to a general $p$. The technique involves approximating the posterior Dirichlet process by a Bayesian bootstrap process and deriving a conditional Donsker theorem. We also obtain analogous results for an affine equivariant version of the multivariate $\ell_1$-median based on an adaptive transformation and re-transformation technique. The results are extended to a joint distribution of multivariate quantiles. The accuracy of the asymptotic result is confirmed by a simulation study. We also use the results to obtain Bayesian credible regions for multivariate medians for Fisher's iris data, which consists of four features measured for each of three plant species.

math.ST↗

arXiv2019

Bayesian Linear Regression for Multivariate Responses Under Group Sparsity

We study frequentist properties of a Bayesian high-dimensional multivariate linear regression model with correlated responses. The predictors are separated into many groups and the group structure is pre-determined. Two features of the model are unique: (i) group sparsity is imposed on the predictors. (ii) the covariance matrix is unknown and its dimensions can also be high. We choose a product of independent spike-and-slab priors on the regression coefficients and a new prior on the covariance matrix based on its eigendecomposition. Each spike-and-slab prior is a mixture of a point mass at zero and a multivariate density involving a $\ell_{2,1}$-norm. We first obtain the posterior contraction rate, the bounds on the effective dimension of the model with high posterior probabilities. We then show that the multivariate regression coefficients can be recovered under certain compatibility conditions. Finally, we quantify the uncertainty for the regression coefficients with frequentist validity through a Bernstein-von Mises type theorem. The result leads to selection consistency for the Bayesian method. We derive the posterior contraction rate using the general theory by constructing a suitable test from the first principle using moment bounds for certain likelihood ratios. This leads to posterior concentration around the truth with respect to the average Rényi divergence of order 1/2. This technique of obtaining the required tests for posterior contraction rate could be useful in many other problems.

math.ST↗

arXiv2019

Bayesian Analysis of Nonparanormal Graphical Models Using Rank-Likelihood

Gaussian graphical models, where it is assumed that the variables of interest jointly follow a multivariate normal distribution with a sparse precision matrix, have been used to study intrinsic dependence among variables, but the normality assumption may be restrictive in many settings. A nonparanormal graphical model is a semiparametric generalization of a Gaussian graphical model for continuous variables where it is assumed that the variables follow a Gaussian graphical model only after some unknown smooth monotone transformation. We consider a Bayesian approach for the nonparanormal graphical model using a rank-likelihood which remains invariant under monotone transformations, thereby avoiding the need to put a prior on the transformation functions. On the underlying precision matrix of the transformed variables, we consider a horseshoe prior on its Cholesky decomposition and use an efficient posterior Gibbs sampling scheme. We present a posterior consistency result for the precision matrix based on the rank-based likelihood. We study the numerical performance of the proposed method through a simulation study and apply it on a real dataset.

stat.ME↗

arXiv2019

Bayesian Inference in Nonparanormal Graphical Models

Gaussian graphical models have been used to study intrinsic dependence among several variables, but the Gaussianity assumption may be restrictive in many applications. A nonparanormal graphical model is a semiparametric generalization for continuous variables where it is assumed that the variables follow a Gaussian graphical model only after some unknown smooth monotone transformations on each of them. We consider a Bayesian approach in the nonparanormal graphical model by putting priors on the unknown transformations through a random series based on B-splines where the coefficients are ordered to induce monotonicity. A truncated normal prior leads to partial conjugacy in the model and is useful for posterior simulation using Gibbs sampling. On the underlying precision matrix of the transformed variables, we consider a spike-and-slab prior and use an efficient posterior Gibbs sampling scheme. We use the Bayesian Information Criterion to choose the hyperparameters for the spike-and-slab prior. We present a posterior consistency result on the underlying transformation and the precision matrix. We study the numerical performance of the proposed method through an extensive simulation study and finally apply the proposed method on a real data set.

stat.ME↗

arXiv2019

Bayesian Modeling of the Structural Connectome for Studying Alzheimer Disease

We study possible relations between the structure of the connectome, white matter connecting different regions of brain, and Alzheimer disease. Regression models in covariates including age, gender and disease status for the extent of white matter connecting each pair of regions of brain are proposed. Subject We study possible relations between the Alzheimer's disease progression and the structure of the connectome, white matter connecting different regions of brain. Regression models in covariates including age, gender and disease status for the extent of white matter connecting each pair of regions of brain are proposed. Subject inhomogeneity is also incorporated in the model through random effects with an unknown distribution. As there are large number of pairs of regions, we also adopt a dimension reduction technique through graphon (Lovasz and Szegedy (2006)) functions, which reduces functions of pairs of regions to functions of regions. The connecting graphon functions are considered unknown but assumed smoothness allows putting priors of low complexity on them. We pursue a nonparametric Bayesian approach by assigning a Dirichlet process scale mixture of zero mean normal prior on the distributions of the random effects and finite random series of tensor products of B-splines priors on the underlying graphon functions. Markov chain Monte Carlo techniques, for drawing samples for the posterior distributions are developed. The proposed Bayesian method overwhelmingly outperforms similar ANCOVA models in the simulation setup. The proposed Bayesian approach is applied on a dataset of 100 subjects and 83 brain regions and key regions implicated in the changing connectome are identified.

stat.ME↗

arXiv2019

High-dimensional single-index Bayesian modeling of brain atrophy

We propose a model of brain atrophy as a function of high-dimensional genetic information and low dimensional covariates such as gender, age, APOE gene, and disease status. A nonparametric single-index Bayesian model of high dimension is proposed to model the relationship with B-spline series prior on the unknown functions and Dirichlet process scale mixture of centered normal prior on the distributions of the random effects. The posterior rate of contraction without the random effect is established for a fixed number of regions and time points with increasing sample size. We implement an efficient computation algorithm through a Hamiltonian Monte Carlo (HMC) algorithm. The performance of the proposed Bayesian method is compared with the corresponding least square estimator in the linear model with horseshoe prior, LASSO and SCAD penalization on the high-dimensional covariates. The proposed Bayesian method is applied to a dataset on volumes of brain regions recorded over multiple visits of 748 individuals using 620,901 SNPs and 6 other covariates for each individual, to identify factors associated with brain atrophy.

stat.ME↗

arXiv2018

Bayesian Change Point Detection for Functional Data

We propose a Bayesian method to detect change points for functional data. We extract the features of a sequence of functional data by the discrete wavelet transform (DWT), and treat each sequence of feature independently. We believe there is potentially a change in each feature at possibly different time points. The functional data evolves through such changes throughout the sequences of observations. The change point for this sequence of functional data is the cumulative effect of changes in all features. We assign the features with priors which incorporate the characteristic of the wavelet coefficients. Then we compute the posterior distribution of change point for each sequence of feature, and define a matrix where each entry is a measure of similarity between two functional data in this sequence. We compute the ratio of the mean similarity between groups and within groups for all possible partitions, and the change point is where the ratio reaches the minimum. We demonstrate this method using a dataset on climate change.

stat.ME↗

arXiv2018

Bayesian Classification of Multiclass Functional Data

We propose a Bayesian approach to estimating parameters in multiclass functional models. Unordered multinomial probit, ordered multinomial probit and multinomial logistic models are considered. We use finite random series priors based on a suitable basis such as B-splines in these three multinomial models, and classify the functional data using the Bayes rule. We average over models based on the marginal likelihood estimated from Markov Chain Monte Carlo (MCMC) output. Posterior contraction rates for the three multinomial models are computed. We also consider Bayesian linear and quadratic discriminant analyses on the multivariate data obtained by applying a functional principal component technique on the original functional data. A simulation study is conducted to compare these methods on different types of data. We also apply these methods to a phoneme dataset.

stat.ME↗

arXiv2018

Bayesian ROC surface estimation under verification bias

The Receiver Operating Characteristic (ROC) surface is a generalization of ROC curve and is widely used for assessment of the accuracy of diagnostic tests on three categories. A complication called the verification bias, meaning that not all subjects have their true disease status verified often occur in real application of ROC analysis. This is a common problem since the gold standard test, which is used to generate true disease status, can be invasive and expensive. In this paper, we will propose a Bayesian approach for estimating the ROC surface based on continuous data under a semi-parametric trinormality assumption. Our proposed method often adopted in ROC analysis can also be extended to situation in the presence of verification bias. We compute the posterior distribution of the parameters under trinormality assumption by using a rank-based likelihood. Consistency of the posterior under mild conditions is also established. We compare our method with the existing methods for estimating ROC surface and conclude that our method performs well in terms of accuracy.

stat.AP↗

arXiv2018

Bayesian mode and maximum estimation and accelerated rates of contraction

We study the problem of estimating the mode and maximum of an unknown regression function in the presence of noise. We adopt the Bayesian approach by using tensor-product B-splines and endowing the coefficients with Gaussian priors. In the usual fixed-in-advanced sampling plan, we establish posterior contraction rates for mode and maximum and show that they coincide with the minimax rates for this problem. To quantify estimation uncertainty, we construct credible sets for these two quantities that have high coverage probabilities with optimal sizes. If one is allowed to collect data sequentially, we further propose a Bayesian two-stage estimation procedure, where a second stage posterior is built based on samples collected within a credible set constructed from a first stage posterior. Under appropriate conditions on the radius of this credible set, we can accelerate optimal contraction rates from the fixed-in-advanced setting to the minimax sequential rates. A simulation experiment shows that our Bayesian two-stage procedure outperforms single-stage procedure and also slightly improves upon a non-Bayesian two-stage procedure.

math.ST↗

arXiv2018

Bayesian method for causal inference in spatially-correlated multivariate time series

Measuring the causal impact of an advertising campaign on sales is an essential task for advertising companies. Challenges arise when companies run advertising campaigns in multiple stores which are spatially correlated, and when the sales data have a low signal-to-noise ratio which makes the advertising effects hard to detect. This paper proposes a solution to address both of these challenges. A novel Bayesian method is proposed to detect weaker impacts and a multivariate structural time series model is used to capture the spatial correlation between stores through placing a $\mathcal{G}$-Wishart prior on the precision matrix. The new method is to compare two posterior distributions of a latent variable---one obtained by using the observed data from the test stores and the other one obtained by using the data from their counterfactual potential outcomes. The counterfactual potential outcomes are estimated from the data of synthetic controls, each of which is a linear combination of sales figures at many control stores over the causal period. Control stores are selected using a revised Expectation-Maximization variable selection (EMVS) method. A two-stage algorithm is proposed to estimate the parameters of the model. To prevent the prediction intervals from being explosive, a stationarity constraint is imposed on the local linear trend of the model through a recently proposed prior. The benefit of using this prior is discussed in this paper. A detailed simulation study shows the effectiveness of using our proposed method to detect weaker causal impact. The new method is applied to measure the causal effect of an advertising campaign for a consumer product sold at stores of a large national retail chain.

stat.ME↗

arXiv2017

Multivariate Gaussian Network Structure Learning

We consider a graphical model where a multivariate normal vector is associated with each node of the underlying graph and estimate the graphical structure. We minimize a loss function obtained by regressing the vector at each node on those at the remaining ones under a group penalty. We show that the proposed estimator can be computed by a fast convex optimization algorithm. We show that as the sample size increases, the estimated regression coefficients and the correct graphical structure are correctly estimated with probability tending to one. By extensive simulations, we show the superiority of the proposed method over comparable procedures. We apply the technique on two real datasets. The first one is to identify gene and protein networks showing up in cancer cell lines, and the second one is to reveal the connections among different industries in the US.

stat.ML↗

arXiv2016

Analyzing Ozone Concentration by Bayesian Spatio-temporal Quantile Regression

Ground level Ozone is one of the six common air-pollutants on which the EPA has set national air quality standards. In order to capture the spatio-temporal trend of 1-hour and 8-hour average ozone concentration in the US, we develop a method for spatio-temporal simultaneous quantile regression. Unlike existing procedures, in the proposed method, smoothing across the sites is incorporated within modeling assumptions thus allowing borrowing of information across locations, an essential step when the number of samples in each location is low. The quantile function has been assumed to be linear in time and smooth over space and at any given site is given by a convex combination of two monotone increasing functions $ξ_1$ and $ξ_2$ not depending on time. A B-spline basis expansion with increasing coefficients varying smoothly over the space is used to put a prior and a Bayesian analysis is performed. We analyze the average daily 1-hour maximum and 8-hour maximum ozone concentration level data of US and California during 2006-2015 using the proposed method. It is noted that in the last ten years, there is an overall decreasing trend in both 1-hour maximum and 8-hour maximum ozone concentration level over the most parts of the US. In California, an overall a decreasing trend of 1-hour maximum ozone level is observed while no particular overall trend has been observed in the case of 8-hour maximum ozone level.

stat.ME↗

arXiv2016

Bayesian Non-parametric Simultaneous Quantile Regression for Complete and Grid Data

In this paper, we consider Bayesian methods for non-parametric quantile regressions with multiple continuous predictors ranging values in the unit interval. In the first method, the quantile function is assumed to be smooth over the explanatory variable and is expanded in tensor product of B-spline basis functions. While in the second method, the distribution function is assumed to be smooth over the explanatory variable and is expanded in tensor product of B-spline basis functions. Unlike other existing methods of non-parametric quantile regressions, the proposed methods estimate the whole quantile function instead of estimating on a grid of quantiles. Priors on the B-spline coefficients are put in such a way that the monotonicity of the estimated quantile levels are maintained unlike local polynomial quantile regression methods. The proposed methods have also been modified for quantile grid data where only the percentile range of each response observations are known. Simulations studies have been provided for both complete and quantile grid data. The proposed method has been used to estimate the quantiles of US household income data and North Atlantic hurricane intensity data.

stat.ME↗

arXiv2016

Bayesian Detection of Image Boundaries

Detecting boundary of an image based on noisy observations is a fundamental problem of image processing and image segmentation. For a $d$-dimensional image ($d = 2, 3, \ldots$), the boundary can often be described by a closed smooth $(d - 1)$-dimensional manifold. In this paper, we propose a nonparametric Bayesian approach based on priors indexed by $\mathbb{S}^{d - 1}$, the unit sphere in $\mathbb{R}^d$. We derive optimal posterior contraction rates using Gaussian processes or finite random series priors using basis functions such as trigonometric polynomials for 2-dimensional images and spherical harmonics for 3-dimensional images. For 2-dimensional images, we show a rescaled squared exponential Gaussian process on $\mathbb{S}^1$ achieves four goals of guaranteed geometric restriction, (nearly) minimax rate optimal and adaptive to the smoothness level, convenient for joint inference and computationally efficient. We conduct an extensive study of its reproducing kernel Hilbert space, which may be of interest by its own and can also be used in other contexts. Simulations confirm excellent performance of the proposed method and indicate its robustness under model misspecification at least under the simulated settings.

math.ST↗

arXiv2016

Efficient Bayesian estimation and uncertainty quantification in ordinary differential equation models

Often the regression function is specified by a system of ordinary differential equations (ODEs) involving some unknown parameters. Typically analytical solution of the ODEs is not available, and hence likelihood evaluation at many parameter values by numerical solution of equations may be computationally prohibitive. Bhaumik and Ghosal (2015) considered a Bayesian two-step approach by embedding the model in a larger nonparametric regression model, where a prior is put through a random series based on B-spline basis functions. A posterior on the parameter is induced from the regression function by minimizing an integrated weighted squared distance between the derivative of the regression function and the derivative suggested by the ODEs. Although this approach is computationally fast, the Bayes estimator is not asymptotically efficient. In this paper we suggest a modification of the two-step method by directly considering the distance between the function in the nonparametric model and that obtained from a four stage Runge-Kutta (RK4) method. We also study the asymptotic behavior of the posterior distribution of the parameter based on an approximate likelihood obtained from an RK4 numerical solution of the ODEs. We establish a Bernstein-von Mises theorem for both methods which assures that Bayesian uncertainty quantification matches with the frequentist one and the Bayes estimator is asymptotically efficient.

math.ST↗

arXiv2016

Adaptive Bayesian density regression for high-dimensional data

Density regression provides a flexible strategy for modeling the distribution of a response variable $Y$ given predictors $\mathbf{X}=(X_1,\ldots,X_p)$ by letting that the conditional density of $Y$ given $\mathbf{X}$ as a completely unknown function and allowing its shape to change with the value of $\mathbf{X}$. The number of predictors $p$ may be very large, possibly much larger than the number of observations $n$, but the conditional density is assumed to depend only on a much smaller number of predictors, which are unknown. In addition to estimation, the goal is also to select the important predictors which actually affect the true conditional density. We consider a nonparametric Bayesian approach to density regression by constructing a random series prior based on tensor products of spline functions. The proposed prior also incorporates the issue of variable selection. We show that the posterior distribution of the conditional density contracts adaptively at the truth nearly at the optimal oracle rate, determined by the unknown sparsity and smoothness levels, even in the ultra high-dimensional settings where $p$ increases exponentially with $n$. The result is also extended to the anisotropic case where the degree of smoothness can vary in different directions, and both random and deterministic predictors are considered. We also propose a technique to calculate posterior moments of the conditional density function without requiring Markov chain Monte Carlo methods.

math.ST↗

arXiv2015

Supremum Norm Posterior Contraction and Credible Sets for Nonparametric Multivariate Regression

In the setting of nonparametric multivariate regression with unknown error variance, we study asymptotic properties of a Bayesian method for estimating a regression function f and its mixed partial derivatives. We use a random series of tensor product of B-splines with normal basis coefficients as a prior for f, and the error variance is either estimated using the empirical Bayes approach or is endowed with a suitable prior in a hierarchical Bayes approach. We establish pointwise, L2 and supremum norm posterior contraction rates for f and its mixed partial derivatives, and show that they coincide with the minimax rates. Our results cover even the anisotropic situation, where the true regression function may have different smoothness in different directions. Using the convergence bounds, we show that pointwise, L2 and supremum norm credible sets for f and its mixed partial derivatives have guaranteed frequentist coverage with optimal size. New results on tensor products of B-splines are also obtained in the course.

math.ST↗