arXiv subjects

Jeffrey W. Miller

Publications and source records attributed to Jeffrey W. Miller.

At least 19 recordsLinked to original sources

arXiv2026

Compressive Bayesian non-negative matrix factorization for mutational signatures analysis

Non-negative matrix factorization (NMF) is a popular tool for dimensionality reduction, especially for count matrices. However, inferring an appropriate number of factors is challenging. Existing approaches based on information criteria or nonparametric sparsity-inducing priors tend to be computationally burdensome or highly sensitive to prior choices. Moreover, theoretical properties of the posterior distribution of Poisson NMF parameters endowed with shrinkage priors remain under-explored. This paper introduces a novel Bayesian NMF method that automatically infers the number of factors while also incorporating information on the latent factors from previous studies. This is achieved using compressive hyperpriors, which are hierarchical priors that make the sample-specific weights of unneeded factors concentrate near zero in the posterior. We provide novel distribution theory for posterior inference to elucidate this shrinkage mechanism, both in finite samples and asymptotically. We apply our method to mutational signatures analysis in cancer genomics, in simulations and on real data from breast cancer. Compared to state-of-the-art alternatives, our method is more robust to mild overdispersion and improves detection and estimation of signatures aligned with prior information.

stat.ME↗

arXiv2026

Poisson process factorization for modeling mutational processes along cancer genomes

Cancer cells acquire DNA mutations through many processes, such as environmental exposures and dysregulated repair mechanisms, and each process consistently produces distinct mutation types at characteristic frequencies, referred to as its signature. The usual approach for inferring these signatures is to decompose the matrix of mutation counts from a sample of tumors via non-negative matrix factorization (NMF). However, existing methods do not model the heterogeneity of mutation rates along the genome, which is driven in part by observed genomic features. In this paper, we introduce Poisson process factorization (PPF), which addresses this limitation by employing an inhomogeneous Poisson point process model to infer mutational signatures and their activities as they vary across the genome. PPF generalizes the baseline NMF model by representing a patient's exposure to each signature as a locus-specific function that depends on genomic covariates and patient-specific copy numbers via a log-linear model. We apply PPF to a sample of 113 breast tumors with $707{,}104$ mutations, using covariates representing histone modifications, cell replication timing, nucleosome positioning, and DNA methylation. Our analysis quantifies the joint effects of these features on the mutational processes in breast cancer, and estimates patient-specific activities of each signature as a function of genomic position.

stat.ME↗

arXiv2026

Reduced-rank Generalized Bilinear Models

Dimensionality reduction and effect estimation are central tasks in the analysis of high-dimensional data such as in genomics. Generalized bilinear models (GBMs) provide a versatile framework for these tasks, however, the statistical and computational efficiency of GBMs degrades rapidly as the number of sample covariates grows. To address this limitation, we introduce reduced-rank generalized bilinear models (RR-GBMs), which employ a reduced-rank sample coefficient matrix to model the effect of a large number of covariates without requiring an excessive number of parameters. In simulation studies, we find that when the true sample coefficient matrix is reduced rank or close to reduced rank, RR-GBM outperforms the standard full-rank GBM both statistically and computationally, providing more accurate estimates with lower computational burden. We develop a data thinning approach for model selection in the RR-GBM framework, facilitating rank selection. Furthermore, RR-GBM enables a new approach to visualizing the relationships among covariates and among features. We demonstrate the method in an application to Perturb-seq data for pancreatic cancer.

stat.ME↗

arXiv2026

Local graph estimation with pathwise false discovery control

Many datasets include a small set of variables, such as biomarkers or clinical outcomes, whose relationships to the broader system are of primary scientific interest. Estimating the full network of inter-variable relationships in such settings often obscures local structures around these targets, limiting interpretability. To address this fundamental problem, we introduce local graph estimation, a statistical framework for inferring substructures around target variables. We show that traditional graph estimation methods often fail to recover local structure, and present pathwise feature selection (PFS) as an effective alternative. PFS estimates local subgraphs by iteratively applying feature selection and propagating uncertainty along network paths, providing rigorous finite-sample false discovery control even in settings with mixed variable types and nonlinear dependencies. In four distinct applications spanning environmental and public health, multiomics, brain connectomics, and single-nucleus RNA sequencing, PFS recovers interpretable networks consistent with domain knowledge, highlighting its ability to uncover established mechanisms and generate novel hypotheses.

stat.ME↗

arXiv2026

Robust model selection using likelihood as data

Model selection is a central task in statistics, but standard methods are not robust in misspecified settings where the true data-generating process (DGP) is not in the set of candidate models. The key limitation is that existing methods -- including information criteria and Bayesian posteriors -- do not quantify uncertainty about how well each candidate model approximates the true DGP. In this paper, we introduce a novel approach to model selection based on modeling the likelihood values themselves. Specifically, given $K$ candidate models and $n$ observations, we view the $n\times K$ matrix of negative log-likelihood values as a random data matrix and observe that the expectation of each row is equal to the vector of Kullback--Leibler divergences between the $K$ models and the true DGP, up to an additive constant. We use a multivariate normal model to estimate and quantify uncertainty in this expectation, providing calibrated inferences for robust model selection under misspecification. The procedure is easy to compute, interpretable, and comes with theoretical guarantees, including consistency.

stat.ME↗

arXiv2025

A powerful goodness-of-fit test using the probability integral transform of order statistics

Goodness-of-fit (GoF) tests are a fundamental component of statistical practice, essential for checking model assumptions and testing scientific hypotheses. Despite their widespread use, popular GoF tests exhibit surprisingly low statistical power against substantial departures from the null hypothesis. To address this, we introduce PITOS, a novel GoF test based on applying the probability integral transform (PIT) to the $j$th order statistic (OS) given the $i$th order statistic for selected pairs $i,j$. Under the null, for any pair $i,j$, this yields a $\mathrm{Uniform}(0,1)$ random variable, which we map to a p-value via $u\mapsto 2\min(u, 1-u)$. We compute these p-values for a structured collection of pairs $i,j$ generated via a discretized transformed Halton sequence, and aggregate them using the Cauchy combination technique to obtain the PITOS p-value. Our method maintains approximately valid Type I error control, has an efficient $O(n \log n)$ runtime, and can be used with any null distribution via the Rosenblatt transform. In empirical demonstrations, we find that PITOS has much higher power than popular GoF tests on distributions characterized by local departures from the null, while maintaining competitive power across all distributions tested.

stat.ME↗

arXiv2025

Integrated path stability selection

Stability selection is a popular method for improving feature selection algorithms. One of its key attributes is that it provides theoretical upper bounds on the expected number of false positives, E(FP), enabling false positive control in practice. However, stability selection often selects few features because existing bounds on E(FP) are relatively loose. In this paper, we introduce a novel approach to stability selection based on integrating stability paths rather than maximizing over them. This yields upper bounds on E(FP) that are much stronger than previous bounds, leading to significantly more true positives in practice for the same target E(FP). Furthermore, our method requires no more computation than the original stability selection algorithm. We demonstrate the method on simulations and real data from two cancer studies.

stat.ME↗

arXiv2025

Nonparametric IPSS: Fast, flexible feature selection with false discovery control

Feature selection is a critical task in machine learning and statistics. However, existing feature selection methods either (i) rely on parametric methods such as linear or generalized linear models, (ii) lack theoretical false discovery control, or (iii) identify few true positives. Here, we introduce a general feature selection method with finite-sample false discovery control based on applying integrated path stability selection (IPSS) to arbitrary feature importance scores. The method is nonparametric whenever the importance scores are nonparametric, and it estimates q-values, which are better suited to high-dimensional data than p-values. We focus on two special cases using importance scores from gradient boosting (IPSSGB) and random forests (IPSSRF). Extensive nonlinear simulations with RNA sequencing data show that both methods accurately control the false discovery rate and detect more true positives than existing methods. Both methods are also efficient, running in under 20 seconds when there are 500 samples and 5000 features. We apply IPSSGB and IPSSRF to detect microRNAs and genes related to cancer, finding that they yield better predictions with fewer features than existing approaches.

stat.ML↗

arXiv2025

Bayesian model criticism using uniform parametrization checks

Models are often misspecified in practice, making model criticism a key part of Bayesian analysis. It is important to detect not only when a model is wrong, but which aspects are wrong, and to do so in a computationally convenient and statistically rigorous way. We introduce a novel method for model criticism based on the fact that if the parameters are drawn from the prior, and the dataset is generated according to the assumed likelihood, then a sample from the posterior will be distributed according to the prior. Thus, departures from the assumed likelihood or prior can be detected by testing whether a posterior sample could plausibly have been generated by the prior. Building upon this idea, we propose to reparametrize all random elements of the likelihood and prior in terms of independent uniform random variables, or u-values. This makes it possible to aggregate across arbitrary subsets of the u-values for data points and parameters to test for model departures using classical hypothesis tests for dependence or non-uniformity. We demonstrate empirically how this method of uniform parametrization checks (UPCs) facilitates model criticism in several examples, and we develop supporting theoretical results.

stat.ME↗

arXiv2024

Improved control of Dirichlet location and scale near the boundary

Dirichlet distributions are commonly used for modeling vectors in a probability simplex. When used as a prior or a proposal distribution, it is natural to set the mean of a Dirichlet to be equal to the location where one wants the distribution to be centered. However, if the mean is near the boundary of the probability simplex, then a Dirichlet distribution becomes highly concentrated either (i) at the mean or (ii) extremely close to the boundary. Consequently, centering at the mean provides poor control over the location and scale near the boundary. In this article, we introduce a method for improved control over the location and scale of Beta and Dirichlet distributions. Specifically, given a target location point and a desired scale, we maximize the density at the target location point while constraining a specified measure of scale. We consider various choices of scale constraint, such as fixing the concentration parameter, the mean cosine error, or the variance in the Beta case. In several examples, we show that this maximum density method provides superior performance for constructing priors, defining Metropolis-Hastings proposals, and generating simulated probability vectors.

stat.ME↗

arXiv2023

Reproducible Parameter Inference Using Bagged Posteriors

Under model misspecification, it is known that Bayesian posteriors often do not properly quantify uncertainty about true or pseudo-true parameters. Even more fundamentally, misspecification leads to a lack of reproducibility in the sense that the same model will yield contradictory posteriors on independent data sets from the true distribution. To define a criterion for reproducible uncertainty quantification under misspecification, we consider the probability that two confidence sets constructed from independent data sets have nonempty overlap, and we establish a lower bound on this overlap probability that holds for any valid confidence sets. We prove that credible sets from the standard posterior can strongly violate this bound, particularly in high-dimensional settings (i.e., with dimension increasing with sample size), indicating that it is not internally coherent under misspecification. To improve reproducibility in an easy-to-use and widely applicable way, we propose to apply bagging to the Bayesian posterior ("BayesBag"'); that is, to use the average of posterior distributions conditioned on bootstrapped datasets. We motivate BayesBag from first principles based on Jeffrey conditionalization and show that the bagged posterior typically satisfies the overlap lower bound. Further, we prove a Bernstein--Von Mises theorem for the bagged posterior, establishing its asymptotic normal distribution. We demonstrate the benefits of BayesBag via simulation experiments and an application to crime rate prediction.

stat.ME↗

arXiv2023

Strong uniform laws of large numbers for bootstrap means and other randomly weighted sums

This article establishes novel strong uniform laws of large numbers for randomly weighted sums such as bootstrap means. By leveraging recent advances, these results extend previous work in their general applicability to a wide range of weighting procedures and in their flexibility with respect to the effective bootstrap sample size. In addition to the standard multinomial bootstrap and the m-out-of-n bootstrap, our results apply to a large class of randomly weighted sums involving negatively orthant dependent (NOD) weights, including the Bayesian bootstrap, jackknife, resampling without replacement, simple random sampling with over-replacement, independent weights, and multivariate Gaussian weighting schemes. Weights are permitted to be non-identically distributed and possibly even negative. Our proof technique is based on extending a proof of the i.i.d. strong uniform law of large numbers to employ strong laws for randomly weighted sums; in particular, we exploit a recent Marcinkiewicz--Zygmund strong law for NOD weighted sums.

math.PR↗

arXiv2023

Principal variables analysis for non-Gaussian data

Principal variables analysis (PVA) is a technique for selecting a subset of variables that capture as much of the information in a dataset as possible. Existing approaches for PVA are based on the Pearson correlation matrix, which is not well-suited to describing the relationships between non-Gaussian variables. We propose a generalized approach to PVA enabling the use of different types of correlation, and we explore using Spearman, Gaussian copula, and polychoric correlations as alternatives to Pearson correlation when performing PVA. We compare performance in simulation studies varying the form of the true multivariate distribution over a wide range of possibilities. Our results show that on continuous non-Gaussian data, using generalized PVA with Gaussian copula or Spearman correlations provides a major improvement in performance compared to Pearson. Meanwhile, on ordinal data, generalized PVA with polychoric correlations outperforms the rest by a wide margin. We apply generalized PVA to a dataset of 102 clinical variables measured on individuals with X-linked dystonia parkinsonism (XDP), a rare neurodegenerative disorder, and we find that using different types of correlation yields substantively different sets of principal variables.

stat.ME↗

arXiv2023

Consistent Model-based Clustering: using the Quasi-Bernoulli Stick-Breaking Process

In mixture modeling and clustering applications, the number of components and clusters is often not known. A stick-breaking mixture model, such as the Dirichlet process mixture model, is an appealing construction that assumes infinitely many components, while shrinking the weights of most of the unused components to near zero. However, it is well-known that this shrinkage is inadequate: even when the component distribution is correctly specified, spurious weights appear and give an inconsistent estimate of the number of clusters. In this article, we propose a simple solution: when breaking each mixture weight stick into two pieces, the length of the second piece is multiplied by a quasi-Bernoulli random variable, taking value one or a small constant close to zero. This effectively creates a soft-truncation and further shrinks the unused weights. Asymptotically, we show that as long as this small constant diminishes to zero at a rate faster than $o(1/n^2)$, with $n$ the sample size, the posterior distribution will converge to the true number of clusters. In comparison, we rigorously explore Dirichlet process mixture models using a concentration parameter that is either constant or rapidly diminishes to zero -- both of which lead to inconsistency for the number of clusters. Our proposed model is easy to implement, requiring only a small modification of a standard Gibbs sampler for mixture models. In simulations and a data application of clustering brain networks, our proposed method recovers the ground-truth number of clusters, and leads to a small number of clusters.

stat.ME↗

arXiv2022

Consistency of mixture models with a prior on the number of components

This article establishes general conditions for posterior consistency of Bayesian finite mixture models with a prior on the number of components. That is, we provide sufficient conditions under which the posterior concentrates on neighborhoods of the true parameter values when the data are generated from a finite mixture over the assumed family of component distributions. Specifically, we establish almost sure consistency for the number of components, the mixture weights, and the component parameters, up to a permutation of the component labels. The approach taken here is based on Doob's theorem, which has the advantage of holding under extraordinarily general conditions, and the disadvantage of only guaranteeing consistency at a set of parameter values that has probability one under the prior. However, we show that in fact, for commonly used choices of prior, this yields consistency at Lebesgue-almost all parameter values -- which is satisfactory for most practical purposes. We aim to formulate the results in a way that maximizes clarity, generality, and ease of use.

math.ST↗

arXiv2021

Reproducible Model Selection Using Bagged Posteriors

Bayesian model selection is premised on the assumption that the data are generated from one of the postulated models. However, in many applications, all of these models are incorrect (that is, there is misspecification). When the models are misspecified, two or more models can provide a nearly equally good fit to the data, in which case Bayesian model selection can be highly unstable, potentially leading to self-contradictory findings. To remedy this instability, we propose to use bagging on the posterior distribution ("BayesBag") -- that is, to average the posterior model probabilities over many bootstrapped datasets. We provide theoretical results characterizing the asymptotic behavior of the posterior and the bagged posterior in the (misspecified) model selection setting. We empirically assess the BayesBag approach on synthetic and real-world data in (i) feature selection for linear regression and (ii) phylogenetic tree reconstruction. Our theory and experiments show that, when all models are misspecified, BayesBag (a) provides greater reproducibility and (b) places posterior mass on optimal models more reliably, compared to the usual Bayesian posterior; on the other hand, under correct specification, BayesBag is slightly more conservative than the usual posterior, in the sense that BayesBag posterior probabilities tend to be slightly farther from the extremes of zero and one. Overall, our results demonstrate that BayesBag provides an easy-to-use and widely applicable approach that improves upon Bayesian model selection by making it more stable and reproducible.

stat.ME↗

arXiv2021

Bayesian data selection

Insights into complex, high-dimensional data can be obtained by discovering features of the data that match or do not match a model of interest. To formalize this task, we introduce the "data selection" problem: finding a lower-dimensional statistic - such as a subset of variables - that is well fit by a given parametric model of interest. A fully Bayesian approach to data selection would be to parametrically model the value of the statistic, nonparametrically model the remaining "background" components of the data, and perform standard Bayesian model selection for the choice of statistic. However, fitting a nonparametric model to high-dimensional data tends to be highly inefficient, statistically and computationally. We propose a novel score for performing both data selection and model selection, the "Stein volume criterion", that takes the form of a generalized marginal likelihood with a kernelized Stein discrepancy in place of the Kullback-Leibler divergence. The Stein volume criterion does not require one to fit or even specify a nonparametric background model, making it straightforward to compute - in many cases it is as simple as fitting the parametric model of interest with an alternative objective function. We prove that the Stein volume criterion is consistent for both data selection and model selection, and we establish consistency and asymptotic normality (Bernstein-von Mises) of the corresponding generalized posterior on parameters. We validate our method in simulation and apply it to the analysis of single-cell RNA sequencing datasets using probabilistic principal components analysis and a spin glass model of gene regulation.

stat.ME↗

arXiv2021

Asymptotic normality, concentration, and coverage of generalized posteriors

Generalized likelihoods are commonly used to obtain consistent estimators with attractive computational and robustness properties. Formally, any generalized likelihood can be used to define a generalized posterior distribution, but an arbitrarily defined "posterior" cannot be expected to appropriately quantify uncertainty in any meaningful sense. In this article, we provide sufficient conditions under which generalized posteriors exhibit concentration, asymptotic normality (Bernstein-von Mises), an asymptotically correct Laplace approximation, and asymptotically correct frequentist coverage. We apply our results in detail to generalized posteriors for a wide array of generalized likelihoods, including pseudolikelihoods in general, the Gaussian Markov random field pseudolikelihood, the fully observed Boltzmann machine pseudolikelihood, the Ising model pseudolikelihood, the Cox proportional hazards partial likelihood, and a median-based likelihood for robust inference of location. Further, we show how our results can be used to easily establish the asymptotics of standard posteriors for exponential families and generalized linear models. We make no assumption of model correctness so that our results apply with or without misspecification.

math.ST↗