arXiv subjects

Sourabh Bhattacharya

Publications and source records attributed to Sourabh Bhattacharya.

At least 37 records · Page 2Linked to original sources

arXiv2021

IID Sampling from Intractable Multimodal and Variable-Dimensional Distributions

Bhattacharya (2021b) has introduced a novel methodology for generating iid realizations from any target distribution on the Euclidean space, irrespective of dimensionality. In this article, our purpose is two-fold. We first extend the method for obtaining iid realizations from general multimodal distributions, and illustrate with a mixture of two 50-dimensional normal distributions. Then we extend the iid sampling method for fixed-dimensional distributions to variable-dimensional situations and illustrate with a variable-dimensional normal mixture modeling of the well-known "acidity data", with further demonstration of the applicability of the iid sampling method developed for multimodal distributions.

stat.ME↗

arXiv2021

IID Sampling from Doubly Intractable Distributions

Intractable posterior distributions of parameters with intractable normalizing constants depending upon the parameters are known as doubly intractable posterior distributions. The terminology itself indicates that obtaining Bayesian inference from such posteriors is doubly difficult compared to traditional intractable posteriors where the normalizing constants are tractable and admit traditional Markov Chain Monte Carlo (MCMC) solutions. As can be anticipated, a plethora of MCMC-based methods have originated in the literature to deal with doubly intractable distributions. Yet, it remains very much unclear if any of the methods can satisfactorily sample from such posteriors, particularly in high-dimensional setups. In this article, we consider efficient Monte Carlo and importance sampling approximations of the intractable normalizing constant for a few values of the parameters, and Gaussian process interpolations for the remaining values of the parameters, using the approximations. We then incorporate this strategy within the exact iid sampling framework developed in Bhattacharya (2021a) and Bhattacharya (2021b), and illustrate the methodology with simulation experiments comprising a two-dimensional normal-gamma posterior, a two-dimensional Ising model posterior, a two-dimensional Strauss process posterior and a 100-dimensional autologistic model posterior. In each case we demonstrate great accuracy of our methodology, which is also computationally extremely efficient, often taking only a few minutes for generating 10, 000 iid realizations on 80 processors.

stat.CO↗

arXiv2021

Bayes Factor Asymptotics for Variable Selection in the Gaussian Process Framework

Although variable selection is one of the most popular areas of modern statistical research, much of its development has taken place in the classical paradigm compared to the Bayesian counterpart. Somewhat surprisingly, both the paradigms have focussed almost completely on linear models, in spite of the vast scope offered by the model liberation movement brought about by modern advancements in studying real, complex phenomena. In this article, we investigate general Bayesian variable selection in models driven by Gaussian processes, which allows us to treat linear, non-linear and nonparametric models, in conjunction with even dependent setups, in the same vein. We consider the Bayes factor route to variable selection, and develop a general asymptotic theory for the Gaussian process framework in the "large p, large n" settings even with p>>n, establishing almost sure exponential convergence of the Bayes factor under appropriately mild conditions. The fixed p setup is included as a special case. To illustrate, we apply our general result to variable selection in linear regression, Gaussian process model with squared exponential covariance function accommodating the covariates, and a first order autoregressive process with time-varying covariates. We also follow up our theoretical investigations with ample simulation experiments in the above regression contexts and variable selection in a real, riboflavin data consisting of 71 observations but 4088 covariates. For implementation of variable selection using Bayes factors, we develop a novel and effective general-purpose transdimensional, transformation based Markov chain Monte Carlo algorithm, which has played a crucial role in our simulated and real data applications.

math.ST↗

arXiv2020

Deep Multi-view Image Fusion for Soybean Yield Estimation in Breeding Applications Deep Multi-view Image Fusion for Soybean Yield Estimation in Breeding Applications

Reliable seed yield estimation is an indispensable step in plant breeding programs geared towards cultivar development in major row crops. The objective of this study is to develop a machine learning (ML) approach adept at soybean [\textit{Glycine max} L. (Merr.)] pod counting to enable genotype seed yield rank prediction from in-field video data collected by a ground robot. To meet this goal, we developed a multi-view image-based yield estimation framework utilizing deep learning architectures. Plant images captured from different angles were fused to estimate the yield and subsequently to rank soybean genotypes for application in breeding decisions. We used data from controlled imaging environment in field, as well as from plant breeding test plots in field to demonstrate the efficacy of our framework via comparing performance with manual pod counting and yield estimation. Our results demonstrate the promise of ML models in making breeding decisions with significant reduction of time and human effort, and opening new breeding methods avenues to develop cultivars.

cs.CV↗

arXiv2020

Planning for Aerial Robot Teams for Wide-Area Biometric and Phenotypic Data Collection

This work presents an efficient and implementable solution to the problem of joint task allocation and path planning in a multi-UAV platform deployed for biometric data collection in-the-wild. The sensing requirement associated with the task gives rise to an uncanny variant of the traditional vehicle routing problem with coverage/sensing constraints. As is the case in several multi-robot path-planning problems, our problem reduces to an $m$TSP problem. In order to tame the computational challenges associated with the problem, we propose a hierarchical solution that decouples the vehicle routing problem from the target allocation problem. As a tangible solution to the allocation problem, we use a clustering-based technique that incorporates temporal uncertainty in the cardinality and position of the robots. Finally, we implement the proposed techniques on our multi-quadcopter platforms.

cs.RO↗

arXiv2020

Bayesian Appraisal of Random Series Convergence with Application to Climate Change

Roy and Bhattacharya (2020) provided Bayesian characterization of infinite series, and their most important application, namely, to the Dirichlet series characterizing the (in)famous Riemann Hypothesis, revealed insights that are not in support of the most celebrated conjecture for over 150 years. In contrast with deterministic series considered by Roy and Bhattacharya (2020), in this article we take up random infinite series for our investigation. Remarkably, our method does not require any simplifying assumption. Albeit the Bayesian characterization theory for random series is no different from that for the deterministic setup, construction of effective upper bounds for partial sums, required for implementation, turns out to be a challenging undertaking in the random setup. In this article, we construct parametric and nonparametric upper bound forms for the partial sums of random infinite series and demonstrate the generality of the latter in comparison to the former. Simulation studies exhibit high accuracy and efficiency of the nonparametric bound in all the setups that we consider. Finally, exploiting the property that the summands tend to zero in the case of series convergence, we consider application of our nonparametric bound driven Bayesian method to global climate change analysis. Specifically, analyzing the global average temperature record over the years 1850--2016 and Holocene global average temperature reconstruction data 12,000 years before present, we conclude, in spite of the current global warming situation, that global climate dynamics is subject to temporary variability only, the current global warming being an instance, and long term global warming or cooling either in the past or in the future, are highly unlikely.

math.PR↗

arXiv2020

On the Characterization of Saddle Point Equilibrium for Security Games with Additive Utility

In this work, we investigate a security game between an attacker and a defender, originally proposed in \cite{emadi2019security}. As is well known, the combinatorial nature of security games leads to a large cost matrix. Therefore, computing the value and optimal strategy for the players becomes computationally expensive. In this work, we analyze a special class of zero-sum games in which the payoff matrix has a special structure which results from the {\it additive property} of the utility function. Based on variational principles, we present structural properties of optimal attacker as well as defender's strategy. We propose a linear-time algorithm to compute the value based on the structural properties, which is an improvement from our previous result in \cite{emadi2019security}, especially in the context of large-scale zero-sum games.

cs.GT↗

arXiv2020

A Bayesian Multiple Testing Paradigm for Model Selection in Inverse Regression Problems

In this article, we propose a novel Bayesian multiple testing formulation for model and variable selection in inverse setups, judiciously embedding the idea of inverse reference distributions proposed by Bhattacharya (2013) in a mixture framework consisting of the competing models. We develop the theory and methods in the general context encompassing parametric and nonparametric competing models, dependent data, as well as misspecifications. Our investigation shows that asymptotically the multiple testing procedure almost surely selects the best possible inverse model that minimizes the minimum Kullback-Leibler divergence from the true model. We also show that the error rates, namely, versions of the false discovery rate and the false non-discovery rate converge to zero almost surely as the sample size goes to infinity. Asymptotic α-control of versions of the false discovery rate and its impact on the convergence of false non-discovery rate versions, are also investigated. Our simulation experiments involve small sample based selection among inverse Poisson log regression and inverse geometric logit and probit regression, where the regressions are either linear or based on Gaussian processes. Additionally, variable selection is also considered. Our multiple testing results turn out to be very encouraging in the sense of selecting the best models in all the non-misspecified and misspecified cases.

math.ST↗

arXiv2020

Convergence of Pseudo-Bayes Factors in Forward and Inverse Regression Problems

In the Bayesian literature on model comparison, Bayes factors play the leading role. In the classical statistical literature, model selection criteria are often devised used cross-validation ideas. Amalgamating the ideas of Bayes factor and cross-validation Geisser and Eddy (1979) created the pseudo-Bayes factor. The usage of cross-validation inculcates several theoretical advantages, computational simplicity and numerical stability in Bayes factors as the marginal density of the entire dataset is replaced with products of cross-validation densities of individual data points. However, the popularity of pseudo-Bayes factors is still negligible in comparison with Bayes factors, with respect to both theoretical investigations and practical applications. In this article, we establish almost sure exponential convergence of pseudo-Bayes factors for large samples under a general setup consisting of dependent data and model misspecifications. We particularly focus on general parametric and nonparametric regression setups in both forward and inverse contexts. We illustrate our theoretical results with various examples, providing explicit calculations. We also supplement our asymptotic theory with simulation experiments in small sample situations of Poisson log regression and geometric logit and probit regression, additionally addressing the variable selection problem. We consider both linear and nonparametric regression modeled by Gaussian processes for our purposes. Our simulation results provide quite interesting insights into the usage of pseudo-Bayes factors in forward and inverse setups.

math.ST↗

arXiv2020

Asymptotic Theory of Dependent Bayesian Multiple Testing Procedures Under Possible Model Misspecification

We study asymptotic properties of Bayesian multiple testing procedures and provide sufficient conditions for strong consistency under general dependence structure. We also consider a novel Bayesian multiple testing procedure and associated error measures that coherently accounts for the dependence structure present in the model. We advocate posterior versions of FDR and FNR as appropriate error rates and show that their asymptotic convergence rates are directly associated with the Kullback-Leibler divergence from the true model. Our results hold even when the class of postulated models is misspecified. We illustrate our results in a variable selection problem with autoregressive response variables, and compare the new Bayesian procedure with some existing methods through extensive simulation studies in the variable selection problem. Superior performance of the new procedure compared to the others vindicate that proper exploitation of the dependence structure by multiple testing methods is indeed important. Moreover, we obtain encouraging results in a real, maize data context, where we select influential marker variables.

math.ST↗

arXiv2020

A Fully Bayesian Approach to Assessment of Model Adequacy in Inverse Problems

We consider the problem of assessing goodness of fit of a single Bayesian model to the observed data in the inverse problem context. A novel procedure of goodness of fit test is proposed, based on construction of reference distributions using the `inverse' part of the given model. This is motivated by an example from palaeoclimatology in which it is of interest to reconstruct past climates using information obtained from fossils deposited in lake sediment. Technically, given a model $f(Y\mid X,θ)$, where $Y$ is the observed data and $X$ is a set of (non-random) covariates, we obtain reference distributions based on the posterior $π(\tilde X\mid Y)$, where $\tilde X$ must be interpreted as the {\it unobserved} random vector corresponding to the {\it observed} covariates $X$. Put simply, if the posterior distribution $π(\tilde X\mid Y)$ gives high density to the observed covariates $X$, or equivalently, if the posterior distribution of $T(\tilde X)$ gives high density to $T(X)$, where $T$ is any appropriate statistic, then we say that the model fits the data. Otherwise the model in question is not adequate. We provide decision-theoretic justification of our proposed approach and discuss other theoretical and computational advantages. We demonstrate our methodology with many simulated examples and three complex, high-dimensional, realistic palaeoclimate problems, including the motivating palaeoclimate problem.

stat.ME↗

arXiv2020

Nonstationary, Nonparametric, Nonseparable Bayesian Spatio-Temporal Modeling Using Kernel Convolution of Order Based Dependent Dirichlet Process

In this article, using kernel convolution of order based dependent Dirichlet process (Griffin and Steel (2006)) we construct a nonstationary, nonseparable, nonparametric space-time process, which, as we show, satisfies desirable properties, and includes the stationary, separable, parametric processes as special cases. We also investigate the smoothness properties of our proposed model. Since our model entails an infinite random series, for Bayesian model fitting purpose we must either truncate the series or more appropriately consider a random number of summands, which renders the model dimension a random variable. We attack the variable dimensionality problem using Transdimensional Transformation based Markov Chain Monte Carlo introduced by Das and Bhattacharya (2019b), which can update all the variables and also change dimensions in a single block using essentially a single random variable drawn from some arbitrary density defined on a relevant support. For the sake of completeness we also address the problem of truncating the infinite series by providing a uniform bound on the error incurred by truncating the infinite series. We illustrate the effectiveness of our model and methodologies on a simulated data set and demonstrate that our approach significantly outperforms that of Fuentes and Reich (2013) which is based on principles somewhat similar to ours. We also fit two real, spatial and spatio-temporal datasets with our approach and obtain quite encouraging results in both the cases.

stat.ME↗

arXiv2020

On Classical and Bayesian Asymptotics in Stochastic Differential Equations with Random Effects having Mixture Normal Distributions

Delattre et al. (2013) considered a system of stochastic differential equations (SDEs) in a random effects setup. Under the independent and identical (iid) situation, and assuming normal distribution of the random effects, they established weak consistency of the maximum likelihood estimators (M LEs) of the population parameters of the random effects. In this article, respecting the increasing importance and versatility of normal mixtures and their ability to approximate any standard distribution, we consider the random effects having mixture of normal distributions and prove asymptotic results associated with the MLEs in both independent and identical (iid) and independent but not identical (non-iid) situations. Besides, we consider iid and non-iid setups under the Bayesian paradigm and establish posterior consistency and asymptotic normality of the posterior distribution of the population parameters, even when the number of mixture components is unknown and treated as a random variable. Although ours is an independent work, we later noted that Delattre et al. (2016) also assumed the SDE setup with normal mixture distribution of the random effect parameters but considered only the iid case and proved only weak consistency of the M LE under an extra, strong assumption as opposed to strong consistency that we are able to prove without the extra assumption. Furthermore, they did not deal with asymptotic normality of M LE or the Bayesian asymptotics counterpart which we investigate in details. Ample simulation experiments and application to a real, stock market data set reveal the importance and usefulness of our methods even for small samples.

math.ST↗

arXiv2020

A Non-Gaussian, Nonparametric Structure for Gene-Gene and Gene-Environment Interactions in Case-Control Studies Based on Hierarchies of Dirichlet Processes

It is becoming increasingly clear that complex interactions among genes and environmental factors play crucial roles in triggering complex diseases. Thus, understanding such interactions is vital, which is possible only through statistical models that adequately account for such intricate, albeit unknown, dependence structures. Bhattacharya & Bhattacharya (2016b) attempt such modeling, relating finite mixtures composed of Dirichlet processes that represent unknown number of genetic sub-populations through a hierarchical matrix-normal structure that incorporates gene-gene interactions, and possible mutations, induced by environmental variables. However, the product dependence structure implied by their matrix-normal model seems to be too simple to be appropriate for general complex, realistic situations. In this article, we propose and develop a novel nonparametric Bayesian model for case-control genotype data using hierarchies of Dirichlet processes that offers a more realistic and nonparametric dependence structure between the genes, induced by the environmental variables. In this regard, we propose a novel and highly parallelisable MCMC algorithm that is rendered quite efficient by the combination of modern parallel computing technology, effective Gibbs sampling steps, retrospective sampling and Transformation based Markov Chain Monte Carlo (TMCMC). We use appropriate Bayesian hypothesis testing procedures to detect the roles of genes and environment in case-control studies. We apply our ideas to 5 biologically realistic case-control genotype datasets simulated under distinct set-ups, and obtain encouraging results in each case. We finally apply our ideas to a real, myocardial infarction dataset, and obtain interesting results on gene-gene and gene-environment interaction, while broadly agreeing with the results reported in the literature.

stat.AP↗

arXiv2020

Posterior Convergence of Gaussian and General Stochastic Process Regression Under Possible Misspecifications

In this article, we investigate posterior convergence in nonparametric regression models where the unknown regression function is modeled by some appropriate stochastic process. In this regard, we consider two setups. The first setup is based on Gaussian processes, where the covariates are either random or non-random and the noise may be either normally or double-exponentially distributed. In the second setup, we assume that the underlying regression function is modeled by some reasonably smooth, but unspecified stochastic process satisfying reasonable conditions. The distribution of the noise is also left unspecified, but assumed to be thick-tailed. As in the previous studies regarding the same problems, we do not assume that the truth lies in the postulated parameter space, thus explicitly allowing the possibilities of misspecification. We exploit the general results of Shalizi (2009) for our purpose and establish not only posterior consistency, but also the rates at which the posterior probabilities converge, which turns out to be the Kullback-Leibler divergence rate. We also investigate the more familiar posterior convergence rates. Interestingly, we show that the posterior predictive distribution can accurately approximate the best possible predictive distribution in the sense that the Hellinger distance, as well as the total variation distance between the two distributions can tend to zero, in spite of misspecifications.

math.ST↗

arXiv2020

Posterior Convergence of Nonparametric Binary and Poisson Regression Under Possible Misspecifications

In this article, we investigate posterior convergence of nonparametric binary and Poisson regression under possible model misspecification, assuming general stochastic process prior with appropriate properties. Our model setup and objective for binary regression is similar to that of Ghosal and Roy (2006) where the authors have used the approach of entropy bound and exponentially consistent tests with the sieve method to achieve consistency with respect to their Gaussian process prior. In contrast, for both binary and Poisson regression, using general stochastic process prior, our approach involves verification of asymptotic equipartition property along with the method of sieve, which is a manoeuvre of the general results of Shalizi (2009), useful even for misspecified models. Moreover, we will establish not only posterior consistency but also the rates at which the posterior probabilities converge, which turns out to be the Kullback-Leibler divergence rate. We also investgate the traditional posterior convergence rates. Interestingly, from subjective Bayesian viewpoint we will show that the posterior predictive distribution can accurately approximate the best possible predictive distribution in the sense that the Hellinger distance, as well as the total variation distance between the two distributions can tend to zero, in spite of misspecifications.

math.ST↗

arXiv2020

Posterior Consistency of Bayesian Inverse Regression and Inverse Reference Distributions

We consider Bayesian inference in inverse regression problems where the objective is to infer about unobserved covariates from observed responses and covariates. We establish posterior consistency of such unobserved covariates in Bayesian inverse regression problemsunder appropriate priors in a leave-one-out cross-validation setup. We relate this to posterior consistency of inverse reference distributions (Bhattacharya (2013)) for assessing model adequacy. We illustrate our theory and methods with various examples of Bayesian inverse regression, along with adequate simulation experiments.

math.ST↗

arXiv2020

Bayesian Characterizations of Properties of Stochastic Processes with Applications

In this article, we primarily propose a novel Bayesian characterization of stationary and nonstationary stochastic processes. In practice, this theory aims to distinguish between global stationarity and nonstationarity for both parametric and nonparametric stochastic processes. Interestingly, our theory builds on our previous work on Bayesian characterization of infinite series, which was applied to verification of the (in)famous Riemann Hypothesis. Thus, there seems to be interesting and important connections between pure mathematics and Bayesian statistics, with respect to our proposed ideas. We validate our proposed method with simulation and real data experiments associated with different setups. In particular, applications of our method include stationarity and nonstationarity determination in various time series models, spatial and spatio-temporal setups, and convergence diagnostics of Markov Chain Monte Carlo. Our results demonstrate very encouraging performance, even in very subtle situations. Using similar principles, we also provide a novel Bayesian characterization of mutual independence among any number of random variables, using which we characterize the properties of point processes, including characterizations of Poisson point processes, complete spatial randomness, stationarity and nonstationarity. Applications to simulation experiments with ample Poisson and non-Poisson point process models again indicate quite encouraging performance of our proposed ideas. We further propose a novel recursive Bayesian method for determination of frequencies of oscillatory stochastic processes, based on our general principle. Simulation studies and real data experiments with varieties of time series models consisting of single and multiple frequencies bring out the worth of our method.

math.ST↗