Scalable methods for Bayesian selective inference

Abstract

Modeled along the truncated approach in Panigrahi (2016), selection-adjusted inference in a Bayesian regime is based on a selective posterior. Such a posterior is determined together by a generative model imposed on data and the selection event that enforces a truncation on the assumed law. The effective difference between the selective posterior and the usual Bayesian framework is reflected in the use of a truncated likelihood. The normalizer of the truncated law in the adjusted framework is the probability of the selection event; this is typically intractable and it leads to the computational bottleneck in sampling from such a posterior. The current work lays out a primal-dual approach of solving an approximating optimization problem to provide valid post-selective Bayesian inference. The selection procedures are posed as data-queries that solve a randomized version of a convex learning program which have the advantage of preserving more left-over information for inference. We propose a randomization scheme under which the optimization has separable constraints that result in a partially separable objective in lower dimensions for many commonly used selective queries to approximate the otherwise intractable selective posterior. We show that the approximating optimization under a Gaussian randomization gives a valid exponential rate of decay for the selection probability on a large deviation scale. We offer a primal-dual method to solve the optimization problem leading to an approximate posterior; this allows us to exploit the usual merits of a Bayesian machinery in both low and high dimensional regimes where the underlying signal is effectively sparse. We show that the adjusted estimates empirically demonstrate better frequentist properties in comparison to the unadjusted estimates based on the usual posterior, when applied to a wide range of constrained, convex data queries.