arXiv subjects

David Kohns

Publications and source records attributed to David Kohns.

10 recordsLinked to original sources

arXiv2026

Proximal R2 Priors for Structured Variance Allocation

Scientific domain knowledge in regression often concerns structural relationships among model terms, which are typically encoded as parameter constraints. Since such relationships rarely hold exactly in practice, prior distributions should encourage them while at the same time retaining support for scientifically plausible departures. Accounting for these relationships becomes more pressing in modern regression, where high-dimensionality is usually present and regularisation is essential. $R^2$ decomposition priors provide an interpretable foundation for regularisation by placing a prior on total explained variance and distributing the implied signal across model terms. We introduce the Proximal R2 prior, which uses proximal relaxations to encode these relationships as soft constraints on the variance allocation. Our formulation yields differentiable posteriors compatible with standard gradient based samplers. Theoretically, we prove that our prior preserves the characteristic marginal properties of global-local shrinkage priors. Moreover, we establish posterior consistency and contraction rates in growing dimensions, showing that the inclusion of structural relationships and proximal methods do not alter the shrinkage properties. Notably, these results demonstrate that the structural constraints in the model do not hinder posterior recovery. Simulation studies and real-data applications to interaction and generalised additive models demonstrate that our prior improves parameter recovery, out-of-sample prediction, and sampling efficiency over existing constrained and unconstrained shrinkage priors, even under structural misspecification.

stat.ME↗

arXiv2026

R2-GP: Variance-Calibrated Priors for Additive Gaussian Processes

We develop R2-GP, an R2D2-type prior for Gaussian process (GP) generalised additive models (GAMs) that calibrates explained variance ($R^2$) for Matérn covariance kernels. The function variance for standard Matérn kernels depends jointly on the effect magnitude and length-scale. As such, smooth and rough components with identical amplitudes contribute very different amounts of variation. With independent priors on component amplitudes, this variation results in variance overestimation, especially in higher dimensions. We overcome this by taking direct control over each component's expected contribution to prior explained variance. We do this by rescaling each component based on its length scale and observed inputs, and modelling each rescaled variance as a share of the function variance induced by a global $R^2$ prior. This allows R2-GP to scale well to many components. To also scale to large sample sizes, we extend R2-GP to Hilbert space approximation methods, a framework for kernels exploiting their spectral decompositions for fast and efficient inference. Our simulations and real-world case studies demonstrate superior generalisation and parameter recovery over independent variance priors.

stat.ML↗

arXiv2026

To select or not to select: predictively consistent priors instead of model selection

Bayesian modelling workflows often consider multiple candidate models of varying complexity. Model selection is commonly used to navigate potential trade-offs between model complexity and generalisability to new data. We study when model selection is unnecessary or can even be harmful for predictive performance in finite data regimes and find that the need for selecting simpler models can depend on prior choice. We formalise predictively consistent priors, which keep prior predictive implications stable as model complexity increases. Across examples and numerical experiments, including adding covariates in linear and logistic regression, forward variable selection, and nonlinear modelling, flexible models with predictively consistent priors typically match or outperform selected simpler models in out-of-sample predictive performance. When selection helps, it can indicate poor joint prior implications, such as excessive prior mass on implausible predictive values. Based on our findings, we propose replacing the notion of sparsity or parsimony at the level of model components with specifying priors that remain sensible in predictive space as models become more complex.

stat.ME↗

arXiv2025

Joint Quantile Shrinkage: A State-Space Approach toward Non-Crossing Bayesian Quantile Models

Crossing of fitted conditional quantiles is a prevalent problem for quantile regression models. We propose a new Bayesian modelling framework that penalises multiple quantile regression functions toward the desired non-crossing space. We achieve this by estimating multiple quantiles jointly with a prior on variation across quantiles, a fused shrinkage prior with quantile adaptivity. The posterior is derived from a decision-theoretic general Bayes perspective, whose form yields a natural state-space interpretation aligned with Time-Varying Parameter (TVP) models. Taken together our approach leads to a Quantile-Varying Parameter (QVP) model, for which we develop efficient sampling algorithms. We demonstrate that our proposed modelling framework provides superior parameter recovery and predictive performance compared to competing Bayesian and frequentist quantile regression estimators in simulated experiments and a real-data application to multivariate quantile estimation in macroeconomics.

stat.ME↗

arXiv2025

R2 priors for Grouped Variance Decomposition in High-dimensional Regression

We introduce the Group-R2 decomposition prior, a hierarchical shrinkage prior that extends R2-based priors to structured regression settings with known groups of predictors. By decomposing the prior distribution of the coefficient of determination R2 in two stages, first across groups, then within groups, the prior enables interpretable control over model complexity and sparsity. We derive theoretical properties of the prior, including marginal distributions of coefficients, tail behavior, and connections to effective model complexity. Through simulation studies, we evaluate the conditions under which grouping improves predictive performance and parameter recovery compared to priors that do not account for groups. Our results provide practical guidance for prior specification and highlight both the strengths and limitations of incorporating grouping into R2-based shrinkage priors.

stat.ME↗

arXiv2025

The ARR2 prior: flexible predictive prior definition for Bayesian auto-regressions

We present the ARR2 prior, a joint prior over the auto-regressive components in Bayesian time-series models and their induced $R^2$. Compared to other priors designed for times-series models, the ARR2 prior allows for flexible and intuitive shrinkage. We derive the prior for pure auto-regressive models, and extend it to auto-regressive models with exogenous inputs, and state-space models. Through both simulations and real-world modelling exercises, we demonstrate the efficacy of the ARR2 prior in improving sparse and reliable inference, while showing greater inference quality and predictive performance than other shrinkage priors. An open-source implementation of the prior is provided.

stat.CO↗

arXiv2023

Bayesian order identification of ARMA models with projection predictive inference

Auto-regressive moving-average (ARMA) models are ubiquitous forecasting tools. Parsimony in such models is highly valued for their interpretability and computational tractability, and as such the identification of model orders remains a fundamental task. We propose a novel method of ARMA order identification through projection predictive inference, which benefits from improved stability through the use of a reference model. The procedure consists of two steps: in the first, the practitioner incorporates their understanding of underlying data-generating process into a reference model, which we latterly project onto possibly parsimonious submodels. These submodels are optimally inferred to best replicate the predictive performance of the reference model. We further propose a search heuristic amenable to the ARMA framework. We show that the submodels selected by our procedure exhibit predictive performance at least as good as those chosen by AIC over simulated and real-data experiments, and in some cases out-perform the latter. Finally we show that our procedure is robust to noise, and scales well to larger data.

stat.ME↗

arXiv2022

Nowcasting Growth using Google Trends Data: A Bayesian Structural Time Series Model

This paper investigates the benefits of internet search data in the form of Google Trends for nowcasting real U.S. GDP growth in real time through the lens of mixed frequency Bayesian Structural Time Series (BSTS) models. We augment and enhance both model and methodology to make these better amenable to nowcasting with large number of potential covariates. Specifically, we allow shrinking state variances towards zero to avoid overfitting, extend the SSVS (spike and slab variable selection) prior to the more flexible normal-inverse-gamma prior which stays agnostic about the underlying model size, as well as adapt the horseshoe prior to the BSTS. The application to nowcasting GDP growth as well as a simulation study demonstrate that the horseshoe prior BSTS improves markedly upon the SSVS and the original BSTS model with the largest gains in dense data-generating-processes. Our application also shows that a large dimensional set of search terms is able to improve nowcasts early in a specific quarter before other macroeconomic data become available. Search terms with high inclusion probability have good economic interpretation, reflecting leading signals of economic anxiety and wealth effects.

econ.EM↗

arXiv2021

Decoupling Shrinkage and Selection for the Bayesian Quantile Regression

This paper extends the idea of decoupling shrinkage and sparsity for continuous priors to Bayesian Quantile Regression (BQR). The procedure follows two steps: In the first step, we shrink the quantile regression posterior through state of the art continuous priors and in the second step, we sparsify the posterior through an efficient variant of the adaptive lasso, the signal adaptive variable selection (SAVS) algorithm. We propose a new variant of the SAVS which automates the choice of penalisation through quantile specific loss-functions that are valid in high dimensions. We show in large scale simulations that our selection procedure decreases bias irrespective of the true underlying degree of sparsity in the data, compared to the un-sparsified regression posterior. We apply our two-step approach to a high dimensional growth-at-risk (GaR) exercise. The prediction accuracy of the un-sparsified posterior is retained while yielding interpretable quantile specific variable selection results. Our procedure can be used to communicate to policymakers which variables drive downside risk to the macro economy.

econ.EM↗

arXiv2021

Horseshoe Prior Bayesian Quantile Regression

This paper extends the horseshoe prior of Carvalho et al. (2010) to Bayesian quantile regression (HS-BQR) and provides a fast sampling algorithm for computation in high dimensions. The performance of the proposed HS-BQR is evaluated on Monte Carlo simulations and a high dimensional Growth-at-Risk (GaR) forecasting application for the U.S. The Monte Carlo design considers several sparsity and error structures. Compared to alternative shrinkage priors, the proposed HS-BQR yields better (or at worst similar) performance in coefficient bias and forecast error. The HS-BQR is particularly potent in sparse designs and in estimating extreme quantiles. As expected, the simulations also highlight that identifying quantile specific location and scale effects for individual regressors in dense DGPs requires substantial data. In the GaR application, we forecast tail risks as well as complete forecast densities using the McCracken and Ng (2020) database. Quantile specific and density calibration score functions show that the HS-BQR provides the best performance, especially at short and medium run horizons. The ability to produce well calibrated density forecasts and accurate downside risk measures in large data contexts makes the HS-BQR a promising tool for nowcasting applications and recession modelling.

econ.EM↗