arXiv subjects

Anirban Bhattacharya

Publications and source records attributed to Anirban Bhattacharya.

At least 73 records · Page 4Linked to original sources

arXiv2018

Revisiting the proton-radius problem using constrained Gaussian processes

Background: The "proton radius puzzle" refers to an eight-year old problem that highlights major inconsistencies in the extraction of the charge radius of the proton from muonic Lamb-shift experiments as compared against experiments using elastic electron scattering. For the latter, the determination of the charge radius involves an extrapolation of the experimental form factor to zero momentum transfer. Purpose: To estimate the proton radius by introducing a novel non-parametric approach to model the electric form factor of the proton. Methods: Within a Bayesian paradigm, we develop a model flexible enough to fit the data without any parametric assumptions on the form factor. The Bayesian estimation is guided by imposing only two physical constraints on the form factor: (a) its value at zero momentum transfer (normalization) and (b) its overall shape, assumed to be a monotonically decreasing function of the momentum transfer. Variants of these assumptions are explored to assess the impact of these constraints. Results: So far our results are inconclusive in regard to the proton puzzle, as they depend on both, the assumed constrains and the range of experimental data used. For example, if only low momentum-transfer data is used, adopting only the normalization constraint provides a value compatible with the smaller muonic result, while imposing only the shape constraint favors the larger electronic value. Conclusions: We have presented a novel technique to estimate the proton radius from electron scattering data based on a non-parametric Gaussian process. We have shown the impact of the physical constraints imposed on the form factor and of the range of experimental data used. In this regard, we are hopeful that as this technique is refined and with the anticipated new results from the PRad experiment, we will get closer to resolve of the puzzle.

nucl-th↗

arXiv2018

Probabilistic community detection with unknown number of communities

A fundamental problem in network analysis is clustering the nodes into groups which share a similar connectivity pattern. Existing algorithms for community detection assume the knowledge of the number of clusters or estimate it a priori using various selection criteria and subsequently estimate the community structure. Ignoring the uncertainty in the first stage may lead to erroneous clustering, particularly when the community structure is vague. We instead propose a coherent probabilistic framework for simultaneous estimation of the number of communities and the community structure, adapting recently developed Bayesian nonparametric techniques to network models. An efficient Markov chain Monte Carlo (MCMC) algorithm is proposed which obviates the need to perform reversible jump MCMC on the number of clusters. The methodology is shown to outperform recently developed community detection algorithms in a variety of synthetic data examples and in benchmark real-datasets. Using an appropriate metric on the space of all configurations, we develop non-asymptotic Bayes risk bounds even when the number of clusters is unknown. Enroute, we develop concentration properties of non-linear functions of Bernoulli random variables, which may be of independent interest.

stat.ME↗

arXiv2018

$α$-Variational Inference with Statistical Guarantees

We propose a family of variational approximations to Bayesian posterior distributions, called $α$-VB, with provable statistical guarantees. The standard variational approximation is a special case of $α$-VB with $α=1$. When $α\in(0,1]$, a novel class of variational inequalities are developed for linking the Bayes risk under the variational approximation to the objective function in the variational optimization problem, implying that maximizing the evidence lower bound in variational inference has the effect of minimizing the Bayes risk within the variational density family. Operating in a frequentist setup, the variational inequalities imply that point estimates constructed from the $α$-VB procedure converge at an optimal rate to the true parameter in a wide range of problems. We illustrate our general theory with a number of examples, including the mean-field variational approximation to (low)-high-dimensional Bayesian linear regression with spike and slab priors, mixture of Gaussian models, latent Dirichlet allocation, and (mixture of) Gaussian variational approximation in regular parametric models.

math.ST↗

arXiv2017

On Statistical Optimality of Variational Bayes

The article addresses a long-standing open problem on the justification of using variational Bayes methods for parameter estimation. We provide general conditions for obtaining optimal risk bounds for point estimates acquired from mean-field variational Bayesian inference. The conditions pertain to the existence of certain test functions for the distance metric on the parameter space and minimal assumptions on the prior. A general recipe for verification of the conditions is outlined which is broadly applicable to existing Bayesian models with or without latent variables. As illustrations, specific applications to Latent Dirichlet Allocation and Gaussian mixture models are discussed.

math.ST↗

arXiv2017

Frequentist coverage and sup-norm convergence rate in Gaussian process regression

Gaussian process (GP) regression is a powerful interpolation technique due to its flexibility in capturing non-linearity. In this paper, we provide a general framework for understanding the frequentist coverage of point-wise and simultaneous Bayesian credible sets in GP regression. As an intermediate result, we develop a Bernstein von-Mises type result under supremum norm in random design GP regression. Identifying both the mean and covariance function of the posterior distribution of the Gaussian process as regularized $M$-estimators, we show that the sampling distribution of the posterior mean function and the centered posterior distribution can be respectively approximated by two population level GPs. By developing a comparison inequality between two GPs, we provide exact characterization of frequentist coverage probabilities of Bayesian point-wise credible intervals and simultaneous credible bands of the regression function. Our results show that inference based on GP regression tends to be conservative; when the prior is under-smoothed, the resulting credible intervals and bands have minimax-optimal sizes, with their frequentist coverage converging to a non-degenerate value between their nominal level and one. As a byproduct of our theory, we show that the GP regression also yields minimax-optimal posterior contraction rate relative to the supremum norm, which provides a positive evidence to the long standing problem on optimal supremum norm contraction rate in GP regression.

math.ST↗

arXiv2017

Compressed Covariance Estimation With Automated Dimension Learning

We propose a method for estimating a covariance matrix that can be represented as a sum of a low-rank matrix and a diagonal matrix. The proposed method compresses high-dimensional data, computes the sample covariance in the compressed space, and lifts it back to the ambient space via a decompression operation. A salient feature of our approach relative to existing literature on combining sparsity and low-rank structures in covariance matrix estimation is that we do not require the low-rank component to be sparse. A principled framework for estimating the compressed dimension using Stein's Unbiased Risk Estimation theory is demonstrated. Experimental simulation results demonstrate the efficacy and scalability of our proposed approach.

stat.ME↗

arXiv2017

Empirical Bayes, SURE and Sparse Normal Mean Models

This paper studies the sparse normal mean models under the empirical Bayes framework. We focus on the mixture priors with an atom at zero and a density component centered at a data driven location determined by maximizing the marginal likelihood or minimizing the Stein Unbiased Risk Estimate. We study the properties of the corresponding posterior median and posterior mean. In particular, the posterior median is a thresholding rule and enjoys the multi-direction shrinkage property that shrinks the observation toward either the origin or the data-driven location. The idea is extended by considering a finite mixture prior, which is flexible to model the cluster structure of the unknown means. We further generalize the results to heteroscedastic normal mean models. Specifically, we propose a semiparametric estimator which can be calculated efficiently by combining the familiar EM algorithm with the Pool-Adjacent-Violators algorithm for isotonic regression. The effectiveness of our methods is demonstrated via extensive numerical studies.

stat.ME↗

arXiv2017

Adaptive posterior convergence rates in non-linear latent variable models

Non-linear latent variable models have become increasingly popular in a variety of applications. However, there has been little study on theoretical properties of these models. In this article, we study rates of posterior contraction in univariate density estimation for a class of non-linear latent variable models where unobserved U(0,1) latent variables are related to the response variables via a random non-linear regression with an additive error. Our approach relies on characterizing the space of densities induced by the above model as kernel convolutions with a general class of continuous mixing measures. The literature on posterior rates of contraction in density estimation almost entirely focuses on finite or countably infinite mixture models. We develop approximation results for our class of continuous mixing measures. Using an appropriate Gaussian process prior on the unknown regression function, we obtain the optimal frequentist rate up to a logarithmic factor under standard regularity conditions on the true density.

math.ST↗

arXiv2017

Scalable Bayesian Variable Selection Using Nonlocal Prior Densities in Ultrahigh-Dimensional Settings

Bayesian model selection procedures based on nonlocal alternative prior densities are extended to ultrahigh dimensional settings and compared to other variable selection procedures using precision-recall curves. Variable selection procedures included in these comparisons include methods based on $g$-priors, reciprocal lasso, adaptive lasso, scad, and minimax concave penalty criteria. The use of precision-recall curves eliminates the sensitivity of our conclusions to the choice of tuning parameters. We find that Bayesian variable selection procedures based on nonlocal priors are competitive to all other procedures in a range of simulation scenarios, and we subsequently explain this favorable performance through a theoretical examination of their consistency properties. When certain regularity conditions apply, we demonstrate that the nonlocal procedures are consistent for linear models even when the number of covariates $p$ increases sub-exponentially with the sample size $n$. A model selection procedure based on Zellner's $g$-prior is also found to be competitive with penalized likelihood methods in identifying the true model, but the posterior distribution on the model space induced by this method is much more dispersed than the posterior distribution induced on the model space by the nonlocal prior methods. We investigate the asymptotic form of the marginal likelihood based on the nonlocal priors and show that it attains a unique term that cannot be derived from the other Bayesian model selection procedures. We also propose a scalable and efficient algorithm called Simplified Shotgun Stochastic Search with Screening (S5) to explore the enormous model space, and we show that S5 dramatically reduces the computing time without losing the capacity to search the interesting region in the model space. The S5 algorithm is available in an \verb R ~package {\it BayesS5} on \texttt{CRAN}.

stat.ME↗

arXiv2016

Bayesian fractional posteriors

We consider the fractional posterior distribution that is obtained by updating a prior distribution via Bayes theorem with a fractional likelihood function, a usual likelihood function raised to a fractional power. First, we analyze the contraction property of the fractional posterior in a general misspecified framework. Our contraction results only require a prior mass condition on certain Kullback-Leibler (KL) neighborhood of the true parameter (or the KL divergence minimizer in the misspecified case), and obviate constructions of test functions and sieves commonly used in the literature for analyzing the contraction property of a regular posterior. We show through a counterexample that some condition controlling the complexity of the parameter space is necessary for the regular posterior to contract, rendering additional flexibility on the choice of the prior for the fractional posterior. Second, we derive a novel Bayesian oracle inequality based on a PAC-Bayes inequality in misspecified models. Our derivation reveals several advantages of averaging based Bayesian procedures over optimization based frequentist procedures. As an application of the Bayesian oracle inequality, we derive a sharp oracle inequality in the convex regression problem under an arbitrary dimension. We also illustrate the theory in Gaussian process regression and density estimation problems.

math.ST↗

arXiv2016

Partitioned Cross-Validation for Divide-and-Conquer Density Estimation

We present an efficient method to estimate cross-validation bandwidth parameters for kernel density estimation in very large datasets where ordinary cross-validation is rendered highly inefficient, both statistically and computationally. Our approach relies on calculating multiple cross-validation bandwidths on partitions of the data, followed by suitable scaling and averaging to return a partitioned cross-validation bandwidth for the entire dataset. The partitioned cross-validation approach produces substantial computational gains over ordinary cross-validation. We additionally show that partitioned cross-validation can be statistically efficient compared to ordinary cross-validation. We derive analytic expressions for the asymptotically optimal number of partitions and study its finite sample accuracy through a detailed simulation study. We additionally propose a permuted version of partitioned cross-validation which attains even higher efficiency. Theoretical properties of the estimators are studied and the methodology is applied to the Higgs Boson dataset with 11 million observations

stat.ME↗

arXiv2016

Fast sampling with Gaussian scale-mixture priors in high-dimensional regression

We propose an efficient way to sample from a class of structured multivariate Gaussian distributions which routinely arise as conditional posteriors of model parameters that are assigned a conditionally Gaussian prior. The proposed algorithm only requires matrix operations in the form of matrix multiplications and linear system solutions. We exhibit that the computational complexity of the proposed algorithm grows linearly with the dimension unlike existing algorithms relying on Cholesky factorizations with cubic orders of complexity. The algorithm should be broadly applicable in settings where Gaussian scale mixture priors are used on high dimensional model parameters. We provide an illustration through posterior sampling in a high dimensional regression setting with a horseshoe prior on the vector of regression coefficients.

stat.CO↗

arXiv2016

Sub-optimality of some continuous shrinkage priors

Two-component mixture priors provide a traditional way to induce sparsity in high-dimensional Bayes models. However, several aspects of such a prior, including computational complexities in high-dimensions, interpretation of exact zeros and non-sparse posterior summaries under standard loss functions, has motivated an amazing variety of continuous shrinkage priors, which can be expressed as global-local scale mixtures of Gaussians. Interestingly, we demonstrate that many commonly used shrinkage priors, including the Bayesian Lasso, do not have adequate posterior concentration in high-dimensional settings.

math.ST↗

arXiv2015

Optimal Gaussian approximations to the posterior for log-linear models with Diaconis-Ylvisaker priors

In contingency table analysis, sparse data is frequently encountered for even modest numbers of variables, resulting in non-existence of maximum likelihood estimates. A common solution is to obtain regularized estimates of the parameters of a log-linear model. Bayesian methods provide a coherent approach to regularization, but are often computationally intensive. Conjugate priors ease computational demands, but the conjugate Diaconis-Ylvisaker priors for the parameters of log-linear models do not give rise to closed form credible regions, complicating posterior inference. Here we derive the optimal Gaussian approximation to the posterior for log-linear models with Diaconis-Ylvisaker priors, and provide convergence rate and finite-sample bounds for the Kullback-Leibler divergence between the exact posterior and the optimal Gaussian approximation. We demonstrate empirically in simulations and a real data application that the approximation is highly accurate, even in relatively small samples. The proposed approximation provides a computationally scalable and principled approach to regularized estimation and approximate Bayesian inference for log-linear models.

stat.ME↗

arXiv2015

Posterior contraction in Gaussian process regression using Wasserstein approximations

We study posterior rates of contraction in Gaussian process regression with unbounded covariate domain. Our argument relies on developing a Gaussian approximation to the posterior of the leading coefficients of a Karhunen--Loéve expansion of the Gaussian process. The salient feature of our result is deriving such an approximation in the $L^2$ Wasserstein distance and relating the speed of the approximation to the posterior contraction rate using a coupling argument. Specific illustrations are provided for the Gaussian or squared-exponential covariance kernel.

math.ST↗

arXiv2015

Optical Properties of Mono-Dispersed AlGaN Nanowires in the Single-Prong Growth Mechanism

Growth of mono-dispersed AlGaN nanowires of ternary wurtzite phase is reported using chemical vapour deposition technique in the vapour-liquid-solid process. The role of distribution of Au catalyst nanoparticles on the size and the shape of AlGaN nanowires are discussed. These variations in the morphology of the nanowires are understood invoking Ostwald ripening of Au catalyst nanoparticles at high temperature followed by the effect of single and multi-prong growth mechanism. Energy-filtered transmission electron microscopy is used as an evidence for the presence of Al in the as-prepared samples. A significant blue shift of the band gap, in the absence of quantum confinement effect in the nanowires with diameter about 100 nm, is used as a supportive evidence for the AlGaN alloy formation. Polarized resonance Raman spectroscopy with strong electron-phonon coupling along with optical confinement due to the dielectric contrast of nanowire with respect to that of surrounding media are adopted to understand the crystalline orientation of a single nanowire in the sub-diffraction limit of about 100 nm using 325 nm wavelength, for the first time. The results are compared with the structural analysis using high resolution transmission microscopic study.

cond-mat.mtrl-sci↗

arXiv2015

Direct Evidence of Mg Incorporation Pathway in Vapor-Liquid-Solid Grown p-type Nonpolar GaN Nanowires

Doping of III-nitride based compound semiconductor nanowires is still a challenging issue to have a control over the dopant distribution in precise locations of the nanowire optoelectronic devices. Knowledge of the dopant incorporation and its pathways in nanowires for such devices is limited by the growth methods. We report the direct evidence of incorporation pathway for Mg dopants in p-type nonpolar GaN nanowires grown via vapour-liquid-solid (VLS) method in a chemical vapour deposition technique for the first time. Mg incorporation is confirmed using X-ray photoelectron (XPS) and electron energy loss spectroscopic (EELS) measurements. Energy filtered transmission electron microscopic (EFTEM) studies are used for finding the Mg incorporation pathway in the GaN nanowire. Photoluminescence studies on Mg doped GaN nanowires along with the electrical characterization on heterojunction formed between nanowires and n-Si confirm the activation of Mg atoms as p-type dopants in nonpolar GaN nanowires.

cond-mat.mtrl-sci↗

arXiv2015

Optimal Bayesian estimation in stochastic block models

With the advent of structured data in the form of social networks, genetic circuits and protein interaction networks, statistical analysis of networks has gained popularity over recent years. Stochastic block model constitutes a classical cluster-exhibiting random graph model for networks. There is a substantial amount of literature devoted to proposing strategies for estimating and inferring parameters of the model, both from classical and Bayesian viewpoints. Unlike the classical counterpart, there is however a dearth of theoretical results on the accuracy of estimation in the Bayesian setting. In this article, we undertake a theoretical investigation of the posterior distribution of the parameters in a stochastic block model. In particular, we show that one obtains optimal rates of posterior convergence with routinely used multinomial-Dirichlet priors on cluster indicators and uniform priors on the probabilities of the random edge indicators. En route, we develop geometric embedding techniques to exploit the lower dimensional structure of the parameter space which may be of independent interest.

math.ST↗