arXiv subjects

Justin D. Silverman

Publications and source records attributed to Justin D. Silverman.

6 recordsLinked to original sources

arXiv2025

Sparse Bayesian Partially Identified Models for Sequence Count Data

In genomics, differential abundance and expression analyses are complicated by the compositional nature of sequence count data, which reflect only relative-not absolute-abundances or expression levels. Many existing methods attempt to address this limitation through data normalizations, but we have shown that such approaches imply strong, often biologically implausible assumptions about total microbial load or total gene expression. Even modest violations of these assumptions can inflate Type I and Type II error rates to over 70%. Sparse estimators have been proposed as an alternative, leveraging the assumption that only a small subset of taxa (or genes) change between conditions. However, we show that current sparse methods suffer from similar pathologies because they treat sparsity assumptions as fixed and ignore the uncertainty inherent in these assumptions. We introduce a sparse Bayesian Partially Identified Model (PIM) that addresses this limitation by explicitly modeling uncertainty in sparsity assumptions. Our method extends the Scale-Reliant Inference (SRI) framework to the sparse setting, providing a principled approach to differential analysis under scale uncertainty. We establish theoretical consistency of the proposed estimator and, through extensive simulations and real data analyses, demonstrate substantial reductions in both Type I and Type II errors compared to existing methods.

stat.ME↗

arXiv2025

Scalable Inference for Bayesian Multinomial Logistic-Normal Dynamic Linear Models

Many scientific fields collect longitudinal count compositional data. Each observation is a multivariate count vector, where the total counts are arbitrary, and the information lies in the relative frequency of the counts. Multiple authors have proposed Bayesian Multinomial Logistic-Normal Dynamic Linear Models (MLN-DLMs) as a flexible approach to modeling these data. However, adoption of these methods has been limited by computational challenges. This article develops an efficient and accurate approach to posterior state estimation, called $\textit{Fenrir}$. Our approach relies on a novel algorithm for MAP estimation and an accurate approximation to a key posterior marginal of the model. As there are no equivalent methods against which we can compare, we also develop an optimized Stan implementation of MLN-DLMs. Our experiments suggest that Fenrir can be three orders of magnitude more efficient than Stan and can even be incorporated into larger sampling schemes for joint inference of model hyperparameters. Our methods are made available to the community as a user-friendly software library written in C++ with an R interface.

stat.AP↗

arXiv2025

Scalable Bayesian Semiparametric Additive Regression Models For Microbiome Studies

Statistical analysis of microbiome data is challenging. Bayesian multinomial logistic-normal (MLN) models have gained popularity due to their ability to account for the count compositional nature of these data, but existing approaches are either computationally intractable or restricted to purely parametric or non-parametric methods, which limit their flexibility and scalability. In this work, we introduce \textit{MultiAddGPs}, a novel semi-parametric framework that integrates additive Gaussian Process (GP) regression within a Bayesian MLN model to disentangle linear and non-linear covariate effects, including non-stationary dynamics. Our approach builds on the computationally efficient Collapse-Uncollapse (CU) sampler and additive GP regression, introducing a novel back-sampling algorithm and marginal likelihood approximation for efficient inference and hyperparameter estimation. Our models are over 240,000 times faster than alternatives while simultaneously producing more accurate posterior estimates. Additionally, we incorporate non-stationary kernel functions designed to model treatment interventions and disease effects. We demonstrate our approach using simulated and real data studies and produce novel biological insights from a previously published human gut microbiome study. Our methods are publicly available as part of the \textit{fido} software package on CRAN \footnotemark.

stat.ME↗

arXiv2024

Scale Reliant Inference

Many scientific fields, including human gut microbiome science, collect multivariate count data where the sum of the counts is unrelated to the scale of the underlying system being measured (e.g., total microbial load in a subject's colon). This disconnect complicates downstream analyses such as differential analysis in case-control studies. This article is motivated by a novel study of in vitro human gut microbiome models. Popular tools for analyzing these data led to dramatically elevated rates of both false positives and false negatives. To understand those failures, we provide a formal problem statement that frames these challenges of scale in terms of the classical theory of identifiability. We call this the problem of Scale Reliant Inference (SRI). We use this formulation to prove fundamental limits on SRI in terms of criteria such as consistency and type-I error control. We show that the failures of existing methods stem from a fundamental failure to properly quantify uncertainty in the system scale. We demonstrate that a particular type of Bayesian model called a Bayesian Partially Identified Model (PIMs) can correctly quantify uncertainty in SRI. We introduce Scale Simulation Random Variables (SSRVs) as a flexible and efficient approach to specifying and inferring Bayesian PIMs. In the context of both real and simulated data, we find SSRVs drastically decrease type-I and type-II error rates.

stat.ME↗

arXiv2019

Bayesian Multinomial Logistic Normal Models through Marginally Latent Matrix-T Processes

Bayesian multinomial logistic-normal (MLN) models are popular for the analysis of sequence count data (e.g., microbiome or gene expression data) due to their ability to model multivariate count data with complex covariance structure. However, existing implementations of MLN models are limited to handling small data sets due to the non-conjugacy of the multinomial and logistic-normal distributions. We introduce MLN models which can be written as marginally latent matrix-t process (LTP) models. Marginally LTP models describe a flexible class of generalized linear regression, non-linear regression, and time series models. We develop inference schemes for Marginally LTP models and, through application to MLN models, demonstrate that our inference schemes are both highly accurate and often 4-5 orders of magnitude faster than MCMC.

stat.ME↗

arXiv2017

The Bayesian Sorting Hat: A Decision-Theoretic Approach to Size-Constrained Clustering

Size-constrained clustering (SCC) refers to the dual problem of using observations to determine latent cluster structure while at the same time assigning observations to the unknown clusters subject to an analyst defined constraint on cluster sizes. While several approaches have been proposed, SCC remains a difficult problem due to the combinatorial dependency between observations introduced by the size-constraints. Here we reformulate SCC as a decision problem and introduce a novel loss function to capture various types of size constraints. As opposed to prior work, our approach is uniquely suited to situations in which size constraints reflect and external limitation or desire rather than an internal feature of the data generation process. To demonstrate our approach, we develop a Bayesian mixture model for clustering respondents using both simulated and real categorical survey data. Our motivation for the development of this decision theoretic approach to SCC was to determine optimal team assignments for a Harry Potter themed scavenger hunt based on categorical survey data from participants.

stat.AP↗