arXiv subjects

Jackson Loper

Publications and source records attributed to Jackson Loper.

9 recordsLinked to original sources

arXiv2025

A new tail bound for the sum of bounded independent random variables

We construct a new tail bound for the sum of independent random variables for situations in which the expected value of the sum is known and each random variable lies within a specified interval, which may be different for each variable. This new bound can be computed by solving a two-dimensional convex optimization problem. Simulations demonstrate that the new bound is often substantially tighter than Hoeffding's inequality for cases in which both bounds are applicable.

math.PR↗

arXiv2025

Globally Convergent Variational Inference

In variational inference (VI), an approximation of the posterior distribution is selected from a family of distributions through numerical optimization. With the most common variational objective function, known as the evidence lower bound (ELBO), only convergence to a local optimum can be guaranteed. In this work, we instead establish the global convergence of a particular VI method. This VI method, which may be considered an instance of neural posterior estimation (NPE), minimizes an expectation of the inclusive (forward) KL divergence to fit a variational distribution that is parameterized by a neural network. Our convergence result relies on the neural tangent kernel (NTK) to characterize the gradient dynamics that arise from considering the variational objective in function space. In the asymptotic regime of a fixed, positive-definite neural tangent kernel, we establish conditions under which the variational objective admits a unique solution in a reproducing kernel Hilbert space (RKHS). Then, we show that the gradient descent dynamics in function space converge to this unique function. In ablation studies and practical problems, we demonstrate that our results explain the behavior of NPE in non-asymptotic finite-neuron settings, and show that NPE outperforms ELBO-based optimization, which often converges to shallow local optima.

stat.ML↗

arXiv2024

Sequential Monte Carlo for Inclusive KL Minimization in Amortized Variational Inference

For training an encoder network to perform amortized variational inference, the Kullback-Leibler (KL) divergence from the exact posterior to its approximation, known as the inclusive or forward KL, is an increasingly popular choice of variational objective due to the mass-covering property of its minimizer. However, minimizing this objective is challenging. A popular existing approach, Reweighted Wake-Sleep (RWS), suffers from heavily biased gradients and a circular pathology that results in highly concentrated variational distributions. As an alternative, we propose SMC-Wake, a procedure for fitting an amortized variational approximation that uses likelihood-tempered sequential Monte Carlo samplers to estimate the gradient of the inclusive KL divergence. We propose three gradient estimators, all of which are asymptotically unbiased in the number of iterations and two of which are strongly consistent. Our method interleaves stochastic gradient updates, SMC samplers, and iterative improvement to an estimate of the normalizing constant to reduce bias from self-normalization. In experiments with both simulated and real datasets, SMC-Wake fits variational distributions that approximate the posterior more accurately than existing methods.

cs.LG↗

arXiv2023

Improving Accuracy in Cell-Perturbation Experiments by Leveraging Auxiliary Information

Modern cell-perturbation experiments expose cells to panels of hundreds of stimuli, such as cytokines or CRISPR guides that perform gene knockouts. These experiments are designed to investigate whether a particular gene is upregulated or downregulated by exposure to each treatment. However, due to high levels of experimental noise, typical estimators of whether a gene is up- or down-regulated make many errors. In this paper, we make two contributions. Our first contribution is a new estimator of regulatory effect that makes use of Gaussian processes and factor analysis to leverage auxiliary information about similarities among treatments, such as the chemical similarity among the drugs used to perturb cells. The new estimator typically has lower variance than unregularized estimators, which do not use auxiliary information, but higher bias. To assess whether this new estimator improves accuracy (i.e., achieves a favorable trade-off between bias and variance), we cannot simply compute its error on heldout data as ``ground truth'' about the effects of treatments is unavailable. Our second contribution is a novel data-splitting method to evaluate error rates. This data-splitting method produces valid error bounds using ``sign-valid'' estimators, which by definition have the correct sign more often than not. Using this data-splitting method, through a series of case studies we find that our new estimator, which leverages auxiliary information, can yield a three-fold reduction in type S error rate.

stat.AP↗

arXiv2023

Variational Inference with Coverage Guarantees in Simulation-Based Inference

Amortized variational inference is an often employed framework in simulation-based inference that produces a posterior approximation that can be rapidly computed given any new observation. Unfortunately, there are few guarantees about the quality of these approximate posteriors. We propose Conformalized Amortized Neural Variational Inference (CANVI), a procedure that is scalable, easily implemented, and provides guaranteed marginal coverage. Given a collection of candidate amortized posterior approximators, CANVI constructs conformalized predictors based on each candidate, compares the predictors using a metric known as predictive efficiency, and returns the most efficient predictor. CANVI ensures that the resulting predictor constructs regions that contain the truth with a user-specified level of probability. CANVI is agnostic to design decisions in formulating the candidate approximators and only requires access to samples from the forward model, permitting its use in likelihood-free settings. We prove lower bounds on the predictive efficiency of the regions produced by CANVI and explore how the quality of a posterior approximation relates to the predictive efficiency of prediction regions based on that approximation. Finally, we demonstrate the accurate calibration and high predictive efficiency of CANVI on a suite of simulation-based inference benchmark tasks and an important scientific task: analyzing galaxy emission spectra.

stat.ME↗

arXiv2022

Model-free Sign Estimation for High-Throughput Screenings

In high-throughput screenings, it is common to estimate the effects of many treatments using a small number of independent trials of each. Because little is known about the distributional properties of the measurements from these trials, it is challenging to identify plausible assumptions that can serve as a basis for inferential statistics in this setting. In this article, we develop a method based on minimal assumptions to infer signs of treatment effects (positive or negative). The proposed method controls the number of misestimated signs by using the number of sign disagreements between measurements of the same treatment as a proxy for the number of sign errors. In simulations, the proposed method compares favorably with the Benjamini-Hochberg procedure applied to invalid $p$-values, which is currently considered best practice for many high-throughput screenings. For real data from the L1000 cell-perturbation platform, the proposed method outperforms existing practices, which fail to control error at the nominal level in some cases and are needlessly conservative in others.

stat.AP↗

arXiv2020

Linear-time inference for Gaussian Processes on one dimension

Gaussian Processes (GPs) provide powerful probabilistic frameworks for interpolation, forecasting, and smoothing, but have been hampered by computational scaling issues. Here we investigate data sampled on one dimension (e.g., a scalar or vector time series sampled at arbitrarily-spaced intervals), for which state-space models are popular due to their linearly-scaling computational costs. It has long been conjectured that state-space models are general, able to approximate any one-dimensional GP. We provide the first general proof of this conjecture, showing that any stationary GP on one dimension with vector-valued observations governed by a Lebesgue-integrable continuous kernel can be approximated to any desired precision using a specifically-chosen state-space model: the Latent Exponentially Generated (LEG) family. This new family offers several advantages compared to the general state-space model: it is always stable (no unbounded growth), the covariance can be computed in closed form, and its parameter space is unconstrained (allowing straightforward estimation via gradient descent). The theorem's proof also draws connections to Spectral Mixture Kernels, providing insight about this popular family of kernels. We develop parallelized algorithms for performing inference and learning in the LEG model, test the algorithm on real and synthetic data, and demonstrate scaling to datasets with billions of samples.

stat.ML↗

arXiv2018

Capacities and the Free Passage of Entropic Barriers

We propose an approach for estimating the probability that a given small target, among many, will be the first to be reached in a molecular dynamics simulation. Reaching small targets out of a vast number of possible configurations constitutes an entropic barrier. Experimental evidence suggests that entropic barriers are ubiquitous in biomolecular systems, and often characterize the rate-limiting step of biomolecular processes. Presumably for the same reasons, they often characterize the rate-limiting step in simulations. To the extent that first-passage probabilities can be computed without requiring direct simulation, the process of traversing entropic barriers can replaced by a single choice from the computed ("first-passage") distribution. We will show that in the presence of certain entropic barriers, first-passage probabilities are approximately invariant to the initial configuration, provided that it is modestly far away from each of the targets. We will further show that as a consequence of this invariance, the first-passage distribution can be well-approximated in terms of "capacities" of local sets around the targets. Using these theoretical results and a Monte Carlo mechanism for approximating capacities, we provide a method for estimating the hitting probabilities of small targets in the presence of entropic barriers. In numerical experiments with an idealized ("golf-course") potential, the estimates are as accurate as the results of direct simulations, but far faster to compute.

physics.comp-ph↗

arXiv2018

Uniform Ergodicity for Brownian Motion in a Bounded Convex Set

We consider an n-dimensional Brownian Motion trapped inside a bounded convex set by normally-reflecting boundaries. It is well-known that this process is uniformly ergodic. However, the rates of this ergodicity are not well-understood, especially in the regime of very high-dimensional sets. Here we present new bounds on these rates for convex sets with a given diameter. Our bounds do not depend upon the smoothness of the boundary nor the value of the ambient dimension, n.

math.PR↗