arXiv subjects

James G. Scott

Publications and source records attributed to James G. Scott.

At least 19 recordsLinked to original sources

arXiv2026

Conformal C2ST: Turning weak classifiers into strong two-sample tests

The two-sample testing problem, a fundamental task in statistics and machine learning, seeks to determine whether two sets of samples, drawn from underlying distributions $p$ and $q$, are in fact identically distributed (i.e. whether $p=q$). A popular and intuitive approach is the classifier two-sample test (C2ST), where a classifier is trained to distinguish between samples from $p$ and $q$. Yet despite simplicity of the C2ST, its reliability hinges on access to a near-Bayes-optimal classifier, a requirement that is rarely met and difficult to verify. This raises a major open question: can a weak classifier still be useful for two-sample testing? We show that the answer is a definitive yes. Building on the work of Hu and Lei (2024), we analyze two conformal variants of the C2ST that convert the scores from any trained classifier -- even if weak, biased, or overfit -- into exact, finite-sample p-values. We establish two key theoretical properties of the conformal C2ST: (i) finite-sample Type-I error control, and (ii) non-trivial power that degrades gently in tandem with the error of the trained classifier. The upshot is that even poorly performing classifiers can yield powerful and reliable two-sample tests. This general framework finds a powerful application in Bayesian inference, particularly for validating Neural Posterior Estimation (NPE) models, where the task of comparing a learned posterior approximation $q(θ\mid y)$ to the true posterior $p(θ\mid y)$ can be framed as a two-sample test. Empirically, the Conformal C2ST outperforms classical discriminative tests across a wide range of benchmarks for this task. Our results establish the conformal C2ST as a practical, theoretically grounded diagnostic tool.

stat.ML↗

arXiv2025

CoLT: The conditional localization test for assessing the accuracy of neural posterior estimates

We consider the problem of validating whether a neural posterior estimate $ q(θ\mid x) $ is an accurate approximation to the true, unknown true posterior $ p(θ\mid x) $. Existing methods for evaluating the quality of an NPE estimate are largely derived from classifier-based tests or divergence measures, but these suffer from several practical drawbacks. As an alternative, we introduce the \emph{Conditional Localization Test} (CoLT), a principled method designed to detect discrepancies between $ p(θ\mid x) $ and $ q(θ\mid x) $ across the full range of conditioning inputs. Rather than relying on exhaustive comparisons or density estimation at every $ x $, CoLT learns a localization function that adaptively selects points $θ_l(x)$ where the neural posterior $q$ deviates most strongly from the true posterior $p$ for that $x$. This approach is particularly advantageous in typical simulation-based inference settings, where only a single draw $ θ\sim p(θ\mid x) $ from the true posterior is observed for each conditioning input, but where the neural posterior $ q(θ\mid x) $ can be sampled an arbitrary number of times. Our theoretical results establish necessary and sufficient conditions for assessing distributional equality across all $ x $, offering both rigorous guarantees and practical scalability. Empirically, we demonstrate that CoLT not only performs better than existing methods at comparing $p$ and $q$, but also pinpoints regions of significant divergence, providing actionable insights for model refinement. These properties position CoLT as a state-of-the-art solution for validating neural posterior estimates.

stat.ML↗

arXiv2025

Conditional diffusions for amortized neural posterior estimation

Neural posterior estimation (NPE), a simulation-based computational approach for Bayesian inference, has shown great success in approximating complex posterior distributions. Existing NPE methods typically rely on normalizing flows, which approximate a distribution by composing many simple, invertible transformations. But flow-based models, while state of the art for NPE, are known to suffer from several limitations, including training instability and sharp trade-offs between representational power and computational cost. In this work, we demonstrate the effectiveness of conditional diffusions coupled with high-capacity summary networks for amortized NPE. Conditional diffusions address many of the challenges faced by flow-based methods. Our results show that, across a highly varied suite of benchmarking problems for NPE architectures, diffusions offer improved stability, superior accuracy, and faster training times, even with simpler, shallower models. Building on prior work on diffusions for NPE, we show that these gains persist across a variety of different summary network architectures. Code is available at https://github.com/TianyuCodings/cDiff.

stat.ML↗

arXiv2021

How Likely are Ride-share Drivers to Earn a Living Wage? Large-scale Spatio-temporal Density Smoothing with the Graph-fused Elastic Net

Ride-sourcing or transportation network companies (TNCs) provide on-demand transportation service for compensation, connecting drivers of personal vehicles with passengers through smartphone applications. In this study, we consider the problem of estimating a spatiotemporally varying probability distribution for the productivity of a TNC driver, using data on more than 1.2 million TNC trips in Austin, Texas. We propose a graph-based smoothing approach that allows for distinct spatial and temporal dynamics, including different degrees of smoothness, spatio-temporal interactions, and interpolation in regions with little or no data. For such a goal, we introduce the Graph-fused Elastic Net (GFEN) and use it in combination with a dyadic tree decomposition for density estimation. In addition, we present an optimization-driven approach for fast point estimates scalable to massive graphs. Bayesian inference and uncertainty quantification with MCMC are also illustrated. The main results demonstrate that the optimization strategy is an effective exploration tool for selecting adequate regularization schemes using Bayesian optimization of the cross-validation loss. Two key empirical findings made possible by our method include: 1) the probability that a TNC driver can expect to earn a living wage in Austin exhibits high variability in space and time, from as low as 25% to as high as 85%; and 2) some drivers suffer considerable "tail risk", with the bottom 10% of the earnings distribution falling below $10 per hour -- grossly below a living wage in Austin for a single adult -- for specific times and locations. All code and data for the paper are publicly available, as a Shiny app for visualizing the results and a software package in Julia for implementing the GFEN.

stat.AP↗

arXiv2020

Optimal post-selection inference for sparse signals: a nonparametric empirical-Bayes approach

Many recently developed Bayesian methods have focused on sparse signal detection. However, much less work has been done addressing the natural follow-up question: how to make valid inferences for the magnitude of those signals after selection. Ordinary Bayesian credible intervals suffer from selection bias, owing to the fact that the target of inference is chosen adaptively. Existing Bayesian approaches for correcting this bias produce credible intervals with poor frequentist properties, while existing frequentist approaches require sacrificing the benefits of shrinkage typical in Bayesian methods, resulting in confidence intervals that are needlessly wide. We address this gap by proposing a nonparametric empirical-Bayes approach for constructing optimal selection-adjusted confidence sets. Our method produces confidence sets that are as short as possible on average, while both adjusting for selection and maintaining exact frequentist coverage uniformly over the parameter space. Our main theoretical result establishes an important consistency property of our procedure: that under mild conditions, it asymptotically converges to the results of an oracle-Bayes analysis in which the prior distribution of signal sizes is known exactly. Across a series of examples, the method outperforms existing frequentist techniques for post-selection inference, producing confidence sets that are notably shorter but with the same coverage guarantee.

stat.ME↗

arXiv2020

Targeted Smooth Bayesian Causal Forests: An analysis of heterogeneous treatment effects for simultaneous versus interval medical abortion regimens over gestation

We introduce Targeted Smooth Bayesian Causal Forests (tsBCF), a nonparametric Bayesian approach for estimating heterogeneous treatment effects which vary smoothly over a single covariate in the observational data setting. The tsBCF method induces smoothness by parameterizing terminal tree nodes with smooth functions, and allows for separate regularization of treatment effects versus prognostic effect of control covariates. Smoothing parameters for prognostic and treatment effects can be chosen to reflect prior knowledge or tuned in a data-dependent way. We use tsBCF to analyze a new clinical protocol for early medical abortion. Our aim is to assess relative effectiveness of simultaneous versus interval administration of mifepristone and misoprostol over the first nine weeks of gestation. The model reflects our expectation that the relative effectiveness varies smoothly over gestation, but not necessarily over other covariates. We demonstrate the performance of the tsBCF method on benchmarking experiments. Software for tsBCF is available at https://github.com/jestarling/tsbcf/.

stat.AP↗

arXiv2019

Monotone function estimation in the presence of extreme data coarsening: Analysis of preeclampsia and birth weight in urban Uganda

This paper proposes a Bayesian hierarchical model to characterize the relationship between birth weight and maternal pre-eclampsia across gestation at a large maternity hospital in urban Uganda. Key scientific questions we investigate include: 1) how pre-eclampsia compares to other maternal-fetal covariates as a predictor of birth weight; and 2) whether the impact of pre-eclampsia on birthweight varies across gestation. Our model addresses several key statistical challenges: it correctly encodes the prior medical knowledge that birth weight should vary smoothly and monotonically with gestational age, yet it also avoids assumptions about functional form along with assumptions about how birth weight varies with other covariates. Our model also accounts for the fact that a high proportion (83%) of birth weights in our data set are rounded to the nearest 100 grams. Such extreme data coarsening is rare in maternity hospitals in high resource obstetrics settings but common for data sets collected in low and middle-income countries (LMICs); this introduces a substantial extra layer of uncertainty into the problem and is a major reason why we adopt a Bayesian approach. Our proposed non-parametric regression model, which we call Projective Smooth BART (psBART), builds upon the highly successful Bayesian Additive Regression Tree (BART) framework. This model captures complex nonlinear relationships and interactions, induces smoothness and monotonicity in a single target covariate, and provides a full posterior for uncertainty quantification. The results of our analysis show that pre-eclampsia is a dominant predictor of birth weight in this urban Ugandan setting, and therefore an important risk factor for perinatal mortality.

stat.AP↗

arXiv2019

Evaluation of Ride-Sourcing Search Frictions and Driver Productivity: A Spatial Denoising Approach

This paper considers the problem of measuring spatial and temporal variation in driver productivity on ride-sourcing trips. This variation is especially important from a driver's perspective: if a platform's drivers experience systematic disparities in earnings because of variation in their riders' destinations, they may perceive the pricing model as inequitable. This perception can exacerbate search frictions if it leads drivers to avoid locations where they believe they may be assigned "unlucky" fares. To characterize any such systematic disparities in productivity, we develop an analytic framework with three key components. First, we propose a productivity metric that looks two consecutive trips ahead, thus capturing the effect on expected earnings of market conditions at drivers' drop-off locations. Second, we develop a natural experiment by analyzing trips with a common origin but varying destinations, thus isolating purely spatial effects on productivity. Third, we apply a spatial denoising method that allows us to work with raw spatial information exhibiting high levels of noise and sparsity, without having to aggregate data into large, low-resolution spatial zones. By applying our framework to data on more than 1.4 million rides in Austin, Texas, we find significant spatial variation in ride-sourcing driver productivity and search frictions. Drivers at the same location experienced disparities in productivity after being dispatched on trips with different destinations, with origin-based surge pricing increasing these earnings disparities. Our results show that trip distance is the dominant factor in driver productivity: short trips yielded lower productivity, even when ending in areas with high demand. These findings suggest that new pricing strategies are required to minimize random disparities in driver earnings.

stat.AP↗

arXiv2019

BART with Targeted Smoothing: An analysis of patient-specific stillbirth risk

This article introduces BART with Targeted Smoothing, or tsBART, a new Bayesian tree-based model for nonparametric regression. The goal of tsBART is to introduce smoothness over a single target covariate t, while not necessarily requiring smoothness over other covariates x. TsBART is based on the Bayesian Additive Regression Trees (BART) model, an ensemble of regression trees. TsBART extends BART by parameterizing each tree's terminal nodes with smooth functions of t, rather than independent scalars. Like BART, tsBART captures complex nonlinear relationships and interactions among the predictors. But unlike BART, tsBART guarantees that the response surface will be smooth in the target covariate. This improves interpretability and helps regularize the estimate. After introducing and benchmarking the tsBART model, we apply it to our motivating example: pregnancy outcomes data from the National Center for Health Statistics. Our aim is to provide patient-specific estimates of stillbirth risk across gestational age (t), based on maternal and fetal risk factors (x). Obstetricians expect stillbirth risk to vary smoothly over gestational age, but not necessarily over other covariates, and tsBART has been designed precisely to reflect this structural knowledge. The results of our analysis show the clear superiority of the tsBART model for quantifying stillbirth risk, thereby providing patients and doctors with better information for managing the risk of perinatal mortality. All methods described here are implemented in the R package tsbart.

stat.ME↗

arXiv2018

Socioeconomic bias in influenza surveillance

Individuals in low socioeconomic brackets are considered at-risk for developing influenza-related complications and often exhibit higher than average influenza-related hospitalization rates. This disparity has been attributed to various factors, including restricted access to preventative and therapeutic health care, limited sick leave, and household structure. Adequate influenza surveillance in these at-risk populations is a critical precursor to accurate risk assessments and effective intervention. However, the United States of America's primary national influenza surveillance system (ILINet) monitors outpatient healthcare providers, which may be largely inaccessible to lower socioeconomic populations. Recent initiatives to incorporate internet-source and hospital electronic medical records data into surveillance systems seek to improve the timeliness, coverage, and accuracy of outbreak detection and situational awareness. Here, we use a flexible statistical framework for integrating multiple surveillance data sources to evaluate the adequacy of traditional (ILINet) and next generation (BioSense 2.0 and Google Flu Trends) data for situational awareness of influenza across poverty levels. We find that zip codes in the highest poverty quartile are a critical blind-spot for ILINet that the integration of next generation data fails to ameliorate.

stat.AP↗

arXiv2017

Interpretable Low-Dimensional Regression via Data-Adaptive Smoothing

We consider the problem of estimating a regression function in the common situation where the number of features is small, where interpretability of the model is a high priority, and where simple linear or additive models fail to provide adequate performance. To address this problem, we present Maximum Variance Total Variation denoising (MVTV), an approach that is conceptually related both to CART and to the more recent CRISP algorithm, a state-of-the-art alternative method for interpretable nonlinear regression. MVTV divides the feature space into blocks of constant value and fits the value of all blocks jointly via a convex optimization routine. Our method is fully data-adaptive, in that it incorporates highly robust routines for tuning all hyperparameters automatically. We compare our approach against CART and CRISP via both a complexity-accuracy tradeoff metric and a human study, demonstrating that that MVTV is a more powerful and interpretable method.

stat.ML↗

arXiv2017

A deconvolution path for mixtures

We propose a class of estimators for deconvolution in mixture models based on a simple two-step "bin-and-smooth" procedure applied to histogram counts. The method is both statistically and computationally efficient: by exploiting recent advances in convex optimization, we are able to provide a full deconvolution path that shows the estimate for the mixing distribution across a range of plausible degrees of smoothness, at far less cost than a full Bayesian analysis. This enables practitioners to conduct a sensitivity analysis with minimal effort. This is especially important for applied data analysis, given the ill-posed nature of the deconvolution problem. Our results establish the favorable theoretical properties of our estimator and show that it offers state-of-the-art performance when compared to benchmark methods across a range of scenarios.

stat.ME↗

arXiv2017

The DFS Fused Lasso: Linear-Time Denoising over General Graphs

The fused lasso, also known as (anisotropic) total variation denoising, is widely used for piecewise constant signal estimation with respect to a given undirected graph. The fused lasso estimate is highly nontrivial to compute when the underlying graph is large and has an arbitrary structure. But for a special graph structure, namely, the chain graph, the fused lasso---or simply, 1d fused lasso---can be computed in linear time. In this paper, we establish a surprising connection between the total variation of a generic signal defined over an arbitrary graph, and the total variation of this signal over a chain graph induced by running depth-first search (DFS) over the nodes of the graph. Specifically, we prove that for any signal, its total variation over the induced chain graph is no more than twice its total variation over the original graph. This connection leads to several interesting theoretical and computational conclusions. Denoting by $m$ and $n$ the number of edges and nodes, respectively, of the graph in question, our result implies that for an underlying signal with total variation $t$ over the graph, the fused lasso achieves a mean squared error rate of \smash{$t^{2/3} n^{-2/3}$}. Moreover, precisely the same mean squared error rate is achieved by running the 1d fused lasso on the induced chain graph from running DFS. Importantly, the latter estimator is simple and computationally cheap, requiring only $O(m)$ operations for constructing the DFS-induced chain and $O(n)$ operations for computing the 1d fused lasso solution over this chain. Further, for trees that have bounded max degree, the error rate of \smash{$t^{2/3} n^{-2/3}$} cannot be improved, in the sense that it is the minimax rate for signals that have total variation $t$ over the tree.

math.ST↗

arXiv2017

Deep Nonparametric Estimation of Discrete Conditional Distributions via Smoothed Dyadic Partitioning

We present an approach to deep estimation of discrete conditional probability distributions. Such models have several applications, including generative modeling of audio, image, and video data. Our approach combines two main techniques: dyadic partitioning and graph-based smoothing of the discrete space. By recursively decomposing each dimension into a series of binary splits and smoothing over the resulting distribution using graph-based trend filtering, we impose a strict structure to the model and achieve much higher sample efficiency. We demonstrate the advantages of our model through a series of benchmarks on both synthetic and real-world datasets, in some cases reducing the error by nearly half in comparison to other popular methods in the literature. All of our models are implemented in Tensorflow and publicly available at https://github.com/tansey/sdp .

stat.ML↗

arXiv2017

GapTV: Accurate and Interpretable Low-Dimensional Regression and Classification

We consider the problem of estimating a regression function in the common situation where the number of features is small, where interpretability of the model is a high priority, and where simple linear or additive models fail to provide adequate performance. To address this problem, we present GapTV, an approach that is conceptually related both to CART and to the more recent CRISP algorithm, a state-of-the-art alternative method for interpretable nonlinear regression. GapTV divides the feature space into blocks of constant value and fits the value of all blocks jointly via a convex optimization routine. Our method is fully data-adaptive, in that it incorporates highly robust routines for tuning all hyperparameters automatically. We compare our approach against CART and CRISP and demonstrate that GapTV finds a much better trade-off between accuracy and interpretability.

stat.ML↗

arXiv2016

Sequential nonparametric tests for a change in distribution: an application to detecting radiological anomalies

We propose a sequential nonparametric test for detecting a change in distribution, based on windowed Kolmogorov--Smirnov statistics. The approach is simple, robust, highly computationally efficient, easy to calibrate, and requires no parametric assumptions about the underlying null and alternative distributions. We show that both the false-alarm rate and the power of our procedure are amenable to rigorous analysis, and that the method outperforms existing sequential testing procedures in practice. We then apply the method to the problem of detecting radiological anomalies, using data collected from measurements of the background gamma-radiation spectrum on a large university campus. In this context, the proposed method leads to substantial improvements in time-to-detection for the kind of radiological anomalies of interest in law-enforcement and border-security applications.

stat.ME↗

arXiv2016

Diet2Vec: Multi-scale analysis of massive dietary data

Smart phone apps that enable users to easily track their diets have become widespread in the last decade. This has created an opportunity to discover new insights into obesity and weight loss by analyzing the eating habits of the users of such apps. In this paper, we present diet2vec: an approach to modeling latent structure in a massive database of electronic diet journals. Through an iterative contract-and-expand process, our model learns real-valued embeddings of users' diets, as well as embeddings for individual foods and meals. We demonstrate the effectiveness of our approach on a real dataset of 55K users of the popular diet-tracking app LoseIt\footnote{http://www.loseit.com/}. To the best of our knowledge, this is the largest fine-grained diet tracking study in the history of nutrition and obesity research. Our results suggest that diet2vec finds interpretable results at all levels, discovering intuitive representations of foods, meals, and diets.

stat.ML↗

arXiv2016

False discovery rate smoothing

We present false discovery rate smoothing, an empirical-Bayes method for exploiting spatial structure in large multiple-testing problems. FDR smoothing automatically finds spatially localized regions of significant test statistics. It then relaxes the threshold of statistical significance within these regions, and tightens it elsewhere, in a manner that controls the overall false-discovery rate at a given level. This results in increased power and cleaner spatial separation of signals from noise. The approach requires solving a non-standard high-dimensional optimization problem, for which an efficient augmented-Lagrangian algorithm is presented. In simulation studies, FDR smoothing exhibits state-of-the-art performance at modest computational cost. In particular, it is shown to be far more robust than existing methods for spatially dependent multiple testing. We also apply the method to a data set from an fMRI experiment on spatial working memory, where it detects patterns that are much more biologically plausible than those detected by standard FDR-controlling methods. All code for FDR smoothing is publicly available in Python and R.

stat.ME↗