arXiv subjects

Jarad Niemi

Publications and source records attributed to Jarad Niemi.

16 recordsLinked to original sources

arXiv2025

Quantile Forecast Matching with a Bayesian Quantile Gaussian Process Model

A set of probabilities along with corresponding quantiles are often used to define predictive distributions or probabilistic forecasts. These quantile predictions offer easily interpreted uncertainty of an event, and quantiles are generally straightforward to estimate using standard statistical and machine learning methods. However, compared to a distribution defined by a probability density or cumulative distribution function, a set of quantiles has less distributional information. When given estimated quantiles, it may be desirable to estimate a fully defined continuous distribution function. Many researchers do so to make evaluation or ensemble modeling simpler. Most existing methods for fitting a distribution to quantiles lack accurate representation of the inherent uncertainty from quantile estimation or are limited in their applications. In this manuscript, we present a Gaussian process model, the quantile Gaussian process, which is based on established theory of quantile functions and sample quantiles, to construct a probability distribution given estimated quantiles. A Bayesian application of the quantile Gaussian process is evaluated for parameter inference and distribution approximation in simulation studies. The quantile Gaussian process is used to approximate the distributions of quantile forecasts from the 2023-24 US Centers for Disease Control collaborative flu forecasting initiative. The simulation studies and data analysis show that the quantile Gaussian process leads to accurate inference on model parameters, estimation of a continuous distribution, and uncertainty quantification of sample quantiles.

stat.ME↗

arXiv2025

Bayesian Stacking via Proper Scoring Rule Optimization using a Gibbs Posterior

In collaborative forecast projects, the combining of multiple probabilistic forecasts into an ensemble is standard practice, with linear pooling being a common combination method. The weighting scheme of a linear pool should be tailored to the specific research question, and weight selection is often performed via optimizing a proper scoring rule. This is known as optimal linear pooling. Besides optimal linear pooling, Bayesian predictive synthesis has emerged as a model probability updating scheme which is more flexible than standard Bayesian model averaging and which provides a Bayesian solution to selecting model weights for a linear pool. In many problems, equally weighted linear pool forecasts often outperform forecasts constructed using sophisticated weight selection methods. Thus regularization to an equal weighting of forecasts may be a valuable addition to any weight selection method. In this manuscript, we introduce an optimal linear pool based on a Gibbs posterior over stacked model weights optimized over a proper scoring rule. The Gibbs posterior extends stacking into a Bayesian framework by allowing for optimal weight solutions to be influenced by a prior distribution, and it also provides uncertainty quantification of weights in the form of a probability distribution. We compare ensemble forecast performance with model averaging methods and equal weighted models in simulation studies and in a real data example from the 2023-24 US Centers for Disease Control FluSight competition. In both the simulation studies and the FluSight analysis, the stacked Gibbs posterior produces ensemble forecasts which often outperform the ensembles of other methods.

stat.ME↗

arXiv2025

Forecasting Influenza Hospitalizations Using a Bayesian Hierarchical Nonlinear Model with Discrepancy

The annual influenza outbreak leads to significant public health and economic burdens making it desirable to have prompt and accurate probabilistic forecasts of the disease spread. The United States Centers for Disease Control and Prevention (CDC) hosts annually a national flu forecasting competition which has led to the development of a variety of flu forecast modeling methods. Beginning in 2013, the target to be forecast was weekly percentage of patients with an influenza-like illness (ILI), but in 2021 the target was changed to weekly hospitalizations. Reliable hospitalization data has only been available since 2021, but ILI data has been available since 2010 and has been successfully forecast for several seasons. In this manuscript, we introduce a two component modeling framework for forecasting hospitalizations utilizing both hospitalization and ILI data. The first component is for modeling ILI data using a nonlinear Bayesian model. The second component is for modeling hospitalizations as a function of ILI. For hospitalization forecasts, ILI is first forecast then hospitalizations are forecast with ILI forecasts used as a predictor. In a simulation study, the hospitalization forecast model is assessed and two previously successful ILI forecast models are compared. Also assessed is the usefulness of including a systematic model discrepancy term in the ILI model. Forecasts of state and national hospitalizations for the 2023-24 flu season are made, and different modeling decisions are compared. We found that including a discrepancy component in the ILI model tends to improve forecasts during certain weeks of the year. We also found that other modeling decisions such as the exact nonlinear function to be used in the ILI model or the error distribution for hospitalization models may or may not be better than other decisions, depending on the season, location, or week of the forecast.

stat.AP↗

arXiv2023

Mixture distributions for probabilistic forecasts of disease outbreaks

Collaboration among multiple teams has played a major role in probabilistic forecasting events of influenza outbreaks, the COVID-19 pandemic, other disease outbreaks, and in many other fields. When collecting forecasts from individual teams, ensuring that each team's model represents forecast uncertainty according to the same format allows for direct comparison of forecasts as well as methods of constructing multi-model ensemble forecasts. This paper outlines several common probabilistic forecast representation formats including parametric distributions, sample distributions, bin distributions, and quantiles and compares their use in the context of collaborative projects. We propose the use of a discrete mixture distribution format in collaborative forecasting in place of other formats. The flexibility in distribution shape, the ease for scoring and building ensemble models, and the reasonably low level of computer storage required to store such a forecast make the discrete mixture distribution an attractive alternative to the other representation formats.

stat.AP↗

arXiv2022

Specifying Prior Distributions in Reliability Applications

Especially when facing reliability data with limited information (e.g., a small number of failures), there are strong motivations for using Bayesian inference methods. These include the option to use information from physics-of-failure or previous experience with a failure mode in a particular material to specify an informative prior distribution. Another advantage is the ability to make statistical inferences without having to rely on specious (when the number of failures is small) asymptotic theory needed to justify non-Bayesian methods. Users of non-Bayesian methods are faced with multiple methods of constructing uncertainty intervals (Wald, likelihood, and various bootstrap methods) that can give substantially different answers when there is little information in the data. For Bayesian inference, there is only one method of constructing equal-tail credible intervals-but it is necessary to provide a prior distribution to fully specify the model. Much work has been done to find default prior distributions that will provide inference methods with good (and in some cases exact) frequentist coverage properties. This paper reviews some of this work and provides, evaluates, and illustrates principled extensions and adaptations of these methods to the practical realities of reliability data (e.g., non-trivial censoring).

stat.ME↗

arXiv2022

Valid predictions of random quantities in linear mixed models

In applications of linear mixed-effects models, experimenters often desire uncertainty quantification for random quantities, like predicted treatment effects for unobserved individuals or groups. For example, consider an agricultural experiment measuring a response on animals receiving different treatments and residing on different farms. A farmer deciding whether to adopt the treatment is most interested in farm-level uncertainty quantification, for example, the range of plausible treatment effects predicted at a new farm. The two-stage linear mixed-effects model is often used to model this type of data. However, standard techniques for linear mixed model-based prediction do not produce calibrated uncertainty quantification. In general, the prediction intervals used in practice are not valid -- they do not meet or exceed their nominal coverage level over repeated sampling. We propose new methods for constructing prediction intervals within the two-stage model framework based on an inferential model (IM). The IM method generates prediction intervals that are guaranteed valid for any sample size. Simulation experiments suggest variations of the IM method that are both valid and efficient, a major improvement over existing methods. We illustrate the use of the IM method using two agricultural data sets, including an on-farm study where the IM-based prediction intervals suggest a higher level of uncertainty in farm-specific effects compared to the standard Student-$t$ based intervals, which are not valid.

stat.ME↗

arXiv2022

The RITAS algorithm: a constructive yield monitor data processing algorithm

Yield monitor datasets are known to contain a high percentage of unreliable records. The current tool set is mostly limited to observation cleaning procedures based on heuristic or empirically-motivated statistical rules for extreme value identification and removal. We propose a constructive algorithm for handling well-documented yield monitor data artifacts without resorting to data deletion. The four-step Rectangle creation, Intersection assignment and Tessellation, Apportioning, and Smoothing (RITAS) algorithm models sample observations as overlapping, unequally-shaped, irregularly-sized, time-ordered, areal spatial units to better replicate the nature of the destructive sampling process. Positional data is used to create rectangular areal spatial units. Time-ordered intersecting area tessellation and harvested mass apportioning generate regularly-shaped and -sized polygons partitioning the entire harvested area. Finally, smoothing via a Gaussian process is used to provide map users with spatial-trend visualization. The intermediate steps as well as the algorithm output are illustrated in maize and soybean grain yield maps for five years of yield monitor data collected at a research agricultural site located in the US Fish and Wildlife Service Neal Smith National Wildlife Refuge.

stat.ME↗

arXiv2022

Automatic Dynamic Relevance Determination for Gaussian process regression with high-dimensional functional inputs

In the context of Gaussian process regression with functional inputs, it is common to treat the input as a vector. The parameter space becomes prohibitively complex as the number of functional points increases, effectively becoming a hindrance for automatic relevance determination in high-dimensional problems. Generalizing a framework for time-varying inputs, we introduce the asymmetric Laplace functional weight (ALF): a flexible, parametric function that drives predictive relevance over the index space. Automatic dynamic relevance determination (ADRD) is achieved with three unknowns per input variable and enforces smoothness over the index space. Additionally, we discuss a screening technique to assess under complete absence of prior and model information whether ADRD is reasonably consistent with the data. Such tool may serve for exploratory analyses and model diagnostics. ADRD is applied to remote sensing data and predictions are generated in response to atmospheric functional inputs. Fully Bayesian estimation is carried out to identify relevant regions of the functional input space. Validation is performed to benchmark against traditional vector-input model specifications. We find that ADRD outperforms models with input dimension reduction via functional principal component analysis. Furthermore, the predictive power is comparable to high-dimensional models, in terms of both mean prediction and uncertainty, with 10 times fewer tuning parameters. Enforcing smoothness on the predictive relevance profile rules out erratic patterns associated with vector-input models.

stat.ME↗

arXiv2020

Score-based likelihood ratios to evaluate forensic pattern evidence

In 2016, the European Network of Forensic Science Institutes (ENFSI) published guidelines for the evaluation, interpretation and reporting of scientific evidence. In the guidelines, ENFSI endorsed the use of the likelihood ratio (LR) as a means to represent the probative value of most types of evidence. While computing the value of a LR is practical in several forensic disciplines, calculating an LR for pattern evidence such as fingerprints, firearm and other toolmarks is particularly challenging because standard statistical approaches are not applicable. Recent research suggests that machine learning algorithms can summarize a potentially large set of features into a single score which can then be used to quantify the similarity between pattern samples. It is then possible to compute a score-based likelihood ratio (SLR) and obtain an approximation to the value of the evidence, but research has shown that the SLR can be quite different from the LR not only in size but also in direction. We provide theoretical and empirical arguments that under reasonable assumptions, the SLR can be a practical tool for forensic evaluations.

stat.AP↗

arXiv2020

Knot Selection in Sparse Gaussian Processes with a Variational Objective Function

Sparse, knot-based Gaussian processes have enjoyed considerable success as scalable approximations to full Gaussian processes. Certain sparse models can be derived through specific variational approximations to the true posterior, and knots can be selected to minimize the Kullback-Leibler divergence between the approximate and true posterior. While this has been a successful approach, simultaneous optimization of knots can be slow due to the number of parameters being optimized. Furthermore, there have been few proposed methods for selecting the number of knots, and no experimental results exist in the literature. We propose using a one-at-a-time knot selection algorithm based on Bayesian optimization to select the number and locations of knots. We showcase the competitive performance of this method relative to simultaneous optimization of knots on three benchmark data sets, but at a fraction of the computational cost.

stat.ML↗

arXiv2020

Knot Selection in Sparse Gaussian Processes

Knot-based, sparse Gaussian processes have enjoyed considerable success as scalable approximations to full Gaussian processes. Problems can occur, however, when knot selection is done by optimizing the marginal likelihood. For example, the marginal likelihood surface is highly multimodal, which can cause suboptimal knot placement where some knots serve practically no function. This is especially a problem when many more knots are used than are necessary, resulting in extra computational cost for little to no gains in accuracy. We propose a one-at-a-time knot selection algorithm to select both the number and placement of knots. Our algorithm uses Bayesian optimization to efficiently propose knots that are likely to be good and largely avoids the pathologies encountered when using the marginal likelihood as the objective function. We provide empirical results showing improved accuracy and speed over the current standard approaches.

stat.ML↗

arXiv2017

Assessing the impacts of time to detection distribution assumptions on detection probability estimation

Abundance estimates from animal point-count surveys require accurate estimates of detection probabilities. The standard model for estimating detection from removal-sampled point-count surveys assumes that organisms at a survey site are detected at a constant rate; however, this assumption is often not justified. We consider a class of N-mixture models that allows for detection heterogeneity over time through a flexibly defined time-to-detection distribution (TTDD) and allows for fixed and random effects for both abundance and detection. Our model is thus a combination of survival time-to-event analysis with unknown-N, unknown-p abundance estimation. We specifically explore two-parameter families of TTDDs, e.g. gamma, that can additionally include a mixture component to model increased probability of detection in the initial observation period. We find that modeling a TTDD by using a two-parameter family is necessary when data have a chance of arising from a distribution of this nature. In addition, models with a mixture component can outperform non-mixture models even when the truth is non-mixture. Finally, we analyze an Overbird data set from the Chippewa National Forest using mixed effect models for both abundance and detection. We demonstrate that the effects of explanatory variables on abundance and detection are consistent across mixture TTDDs but that flexible TTDDs result in lower estimated probabilities of detection and therefore higher estimates of abundance.

q-bio.PE↗

arXiv2016

Bayesian inference for a covariance matrix

Covariance matrix estimation arises in multivariate problems including multivariate normal sampling models and regression models where random effects are jointly modeled, e.g. random-intercept, random-slope models. A Bayesian analysis of these problems requires a prior on the covariance matrix. Here we assess, through a simulation study and a real data set, the impact this prior choice has on posterior inference of the covariance matrix. Inverse Wishart distribution is the natural choice for a covariance matrix prior because its conjugacy on normal model and simplicity, is usually available in Bayesian statistical software. However inverse Wishart distribution presents some undesirable properties from a modeling point of view. It can be too restrictive because assume the same amount of prior information about every variance parameters and, more important, it shows a prior relationship between the variances and correlations. Some alternatives distributions has been proposed. The scaled inverse Wishart distribution, which give more flexibility on the variance priors conserving the conjugacy property but does not eliminate the prior relationship between variances and correlations. Secondly, it is possible to fit separate priors for individual correlations and standard deviations. This strategy eliminates any prior relationship within the covariance matrix parameters, but it is not conjugate and therefore computationally slow.

stat.ME↗

arXiv2016

A fully Bayesian strategy for high-dimensional hierarchical modeling using massively parallel computing

Markov chain Monte Carlo (MCMC) is the predominant tool used in Bayesian parameter estimation for hierarchical models. When the model expands due to an increasing number of hierarchical levels, number of groups at a particular level, or number of observations in each group, a fully Bayesian analysis via MCMC can easily become computationally demanding, even intractable. We illustrate how the steps in an MCMC for hierarchical models are predominantly one of two types: conditionally independent draws or low-dimensional draws based on summary statistics of parameters at higher levels of the hierarchy. Parallel computing can increase efficiency by performing embarrassingly parallel computations for conditionally independent draws and calculating the summary statistics using parallel reductions. During the MCMC algorithm, we record running means and means of squared parameter values to allow convergence diagnosis and posterior inference while avoiding the costly memory transfer bottleneck. We demonstrate the effectiveness of the algorithm on a model motivated by next generation sequencing data, and we release our implementation in R packages fbseq and fbseqCUDA.

stat.CO↗

arXiv2014

Massively parallel approximate Gaussian process regression

We explore how the big-three computing paradigms -- symmetric multi-processor (SMC), graphical processing units (GPUs), and cluster computing -- can together be brought to bare on large-data Gaussian processes (GP) regression problems via a careful implementation of a newly developed local approximation scheme. Our methodological contribution focuses primarily on GPU computation, as this requires the most care and also provides the largest performance boost. However, in our empirical work we study the relative merits of all three paradigms to determine how best to combine them. The paper concludes with two case studies. One is a real data fluid-dynamics computer experiment which benefits from the local nature of our approximation; the second is a synthetic data example designed to find the largest design for which (accurate) GP emulation can performed on a commensurate predictive set under an hour.

stat.CO↗

arXiv2011

Efficient Bayesian inference in stochastic chemical kinetic models using graphical processing units

A goal of systems biology is to understand the dynamics of intracellular systems. Stochastic chemical kinetic models are often utilized to accurately capture the stochastic nature of these systems due to low numbers of molecules. Collecting system data allows for estimation of stochastic chemical kinetic rate parameters. We describe a well-known, but typically impractical data augmentation Markov chain Monte Carlo algorithm for estimating these parameters. The impracticality is due to the use of rejection sampling for latent trajectories with fixed initial and final endpoints which can have diminutive acceptance probability. We show how graphical processing units can be efficiently utilized for parameter estimation in systems that hitherto were inestimable. For more complex systems, we show the efficiency gain over traditional CPU computing is on the order of 200. Finally, we show a Bayesian analysis of a system based on Michaelis-Menton kinetics.

stat.CO↗