arXiv subjects

Reinhard Furrer

Publications and source records attributed to Reinhard Furrer.

At least 19 recordsLinked to original sources

arXiv2026

Scalable non-separable spatio-temporal Gaussian process models for large-scale short-term weather prediction

Monitoring daily weather fields is critical for climate science, agriculture, and environmental planning, yet fully probabilistic spatio-temporal models become computationally prohibitive at continental scale. We present a case study on short-term forecasting of daily maximum temperature and precipitation across the conterminous United States using novel scalable spatio-temporal Gaussian process methodology. Building on three approximation families - inducing-point methods (FITC), Vecchia approximations, and a hybrid Vecchia-inducing-point full-scale approach (VIF) - we introduce three extensions that address key bottlenecks in large space-time settings: (i) a scalable correlation-based neighbor selection strategy for Vecchia approximations with point-referenced data, enabling accurate conditioning under complex dependence structures, (ii) a space-time kMeans++ inducing-point selection algorithm, and (iii) GPU-accelerated implementations of computationally expensive operations, including matrix operations and neighbor searches. Using both synthetic experiments and a large NOAA station dataset containing more than one million space-time observations, we analyze the models with respect to predictive performance, parameter estimation, and computational efficiency. Our results demonstrate that scalable Gaussian process models can yield accurate continental-scale forecasts while remaining computationally feasible, offering practical tools for weather applications.

stat.AP↗

arXiv2025

Time-series Random Process Complexity Ranking Using a Bound on Conditional Differential Entropy

Conditional differential entropy provides an intuitive measure for relatively ranking time-series complexity by quantifying uncertainty in future observations given past context. However, its direct computation for high-dimensional processes from unknown distributions is often intractable. This paper builds on the information theoretic prediction error bounds established by Fang et al. \cite{fang2019generic}, which demonstrate that the conditional differential entropy \textbf{$h(X_k \mid X_{k-1},...,X_{k-m})$} is upper bounded by a function of the determinant of the covariance matrix of next-step prediction errors for any next step prediction model. We add to this theoretical framework by further increasing this bound by leveraging Hadamard's inequality and the positive semi-definite property of covariance matrices. To see if these bounds can be used to rank the complexity of time series, we conducted two synthetic experiments: (1) controlled linear autoregressive processes with additive Gaussian noise, where we compare ordinary least squares prediction error entropy proxies to the true entropies of various additive noises, and (2) a complexity ranking task of bio-inspired synthetic audio data with unknown entropy, where neural network prediction errors are used to recover the known complexity ordering. This framework provides a computationally tractable method for time-series complexity ranking using prediction errors from next-step prediction models, that maintains a theoretical foundation in information theory.

eess.SP↗

arXiv2025

Vecchia-Inducing-Points Full-Scale Approximations for Gaussian Processes

Gaussian processes are flexible, probabilistic, non-parametric models widely used in machine learning and statistics. However, their scalability to large data sets is limited by computational constraints. To overcome these challenges, we propose Vecchia-inducing-points full-scale (VIF) approximations combining the strengths of global inducing points and local Vecchia approximations. Vecchia approximations excel in settings with low-dimensional inputs and moderately smooth covariance functions, while inducing point methods are better suited to high-dimensional inputs and smoother covariance functions. Our VIF approach bridges these two regimes by using an efficient correlation-based neighbor-finding strategy for the Vecchia approximation of the residual process, implemented via a modified cover tree algorithm. We further extend our framework to non-Gaussian likelihoods by introducing iterative methods that substantially reduce computational costs for training and prediction by several orders of magnitudes compared to Cholesky-based computations when using a Laplace approximation. In particular, we propose and compare novel preconditioners and provide theoretical convergence results. Extensive numerical experiments on simulated and real-world data sets show that VIF approximations are both computationally efficient as well as more accurate and numerically stable than state-of-the-art alternatives. All methods are implemented in the open source C++ library GPBoost with high-level Python and R interfaces.

stat.ML↗

arXiv2024

Harnessing the Potential of Spatial Statistics for Spatial Omics Data with pasta

Spatial omics assays allow for the molecular characterisation of cells in their spatial context. Notably, the two main technological streams, imaging-based and high-throughput sequencing-based, can give rise to very different data modalities. The characteristics of the two data types are well known in adjacent fields such as spatial statistics as point patterns and lattice data, and there is a wide range of tools available. This paper discusses the application of spatial statistics to spatially-resolved omics data and in particular, discusses various advantages, challenges, and nuances. This work is accompanied by a vignette, pasta, that showcases the usefulness of spatial statistics in biology using several R packages.

q-bio.QM↗

arXiv2024

A class of modular and flexible covariate-based covariance functions for nonstationary spatial modeling

Paradoxically, while the assumptions of second-order stationarity and isotropy appear outdated in light of modern spatial data, they remain remarkably robust in practice, as nonstationary methods often provide marginal improvements in predictive performance. This limitation reflects a fundamental trade-off: nonparametric approaches, while offering extreme flexibility, require substantial tuning to avoid overfitting and numerical challenges in practice, while parametric approaches are more robust against overfitting but are constrained in flexibility, often facing considerable numerical challenges as flexibility increases. In this article we introduce a parametric class of covariance functions that extends the use of parametric nonstationary spatial models, aiming to compete with the flexibility and local adaptability of nonparametric approaches. The covariance function is modular in the sense that allows for separate parametric structures for different sources of nonstationarity, such as marginal standard deviation, geometric anisotropy, and smoothness. The proposed covariance function retains the practical identifiability and computational stability of parametric forms while closing the performance gap with fully nonparametric methods. A Mat\'ern stationary isotropic model is nested within the complex model and can be adapted such that it is computationally feasible for handling thousands of observations. A two-stage approach can be employed for model selection. We explore the statistical properties of the presented approach, demonstrate its compatibility with the frequentist paradigm, and highlight the interpretability of its parameters. We illustrate its prediction capabilities as well as interpretability through an analysis of Swiss monthly precipitation data, showing that Gaussian process models with the presented covariance function, while remaining robust against overfitting, provide quantitative and qualitative improvements over existing approaches.

stat.ME↗

arXiv2024

More than Formulas -- Integrity, Communication, Computing and Reproducibility in Statistics Education

This paper introduces a novel course design in the Master Program in Biostatistics at the University of Zurich that integrates computing skills, effective communication, reproducibility, and scientific integrity within one course. Utilizing a flipped classroom model, the course aims to equip students with the necessary competencies to handle real-world data analysis challenges and effective statistical practice in general. The curriculum includes practical tools such as version control with Git, dynamic reporting, unit testing and containerization to foster reproducibility, and integrity in statistical practice. Feedback gathered from both staff and students post-implementation indicates that the course significantly enhances student readiness for professional and academic environments, demonstrating the effectiveness of this educational approach.

stat.OT↗

arXiv2024

Iterative Methods for Full-Scale Gaussian Process Approximations for Large Spatial Data

Gaussian processes are flexible probabilistic regression models which are widely used in statistics and machine learning. However, a drawback is their limited scalability to large data sets. To alleviate this, full-scale approximations (FSAs) combine predictive process methods and covariance tapering, thus approximating both global and local structures. We show how iterative methods can be used to reduce computational costs in calculating likelihoods, gradients, and predictive distributions with FSAs. In particular, we introduce a novel preconditioner and show theoretically and empirically that it accelerates the conjugate gradient method's convergence speed and mitigates its sensitivity with respect to the FSA parameters and the eigenvalue structure of the original covariance matrix, and we demonstrate empirically that it outperforms a state-of-the-art pivoted Cholesky preconditioner. Furthermore, we introduce an accurate and fast way to calculate predictive variances using stochastic simulation and iterative methods. In addition, we show how our newly proposed fully independent training conditional (FITC) preconditioner can also be used in iterative methods for Vecchia approximations. In our experiments, it outperforms existing state-of-the-art preconditioners for Vecchia approximations. All methods are implemented in a free C++ software library with high-level Python and R packages.

stat.ME↗

arXiv2023

Compatibility of Space-Time Kernels with Full, Dynamical, or Compact Support

We deal with the comparison of space-time covariance kernels having, either, full, spatially dynamical, or space-time compact support. Such a comparison is based on compatibility of these covariance models under fixed domain asymptotics, having a theoretical background that is substantially coming from equivalence or orthogonality of Gaussian measures. In turn, such a theory is intimately related to the tails of the spectral densities associated with the three models. Models with space-time compact support are still elusive. We taper the temporal part of a model with dynamical support, obtaining a space-time compact support. The spectrum related to such a construction is obtained through temporal convolution of the spatially dynamical spectrum with the spectrum associated with the temporal taper. The solution of such a challenge opens the door to the compatibility-based comparison. Our findings show that indeed these three models can be compatible under some suitable parametric restrictions. As a corollary, we deduce implications in terms of maximum likelihood estimation and misspecified kriging prediction under fixed domain asymptotics.

math.ST↗

arXiv2023

BiasBed -- Rigorous Texture Bias Evaluation

The well-documented presence of texture bias in modern convolutional neural networks has led to a plethora of algorithms that promote an emphasis on shape cues, often to support generalization to new domains. Yet, common datasets, benchmarks and general model selection strategies are missing, and there is no agreed, rigorous evaluation protocol. In this paper, we investigate difficulties and limitations when training networks with reduced texture bias. In particular, we also show that proper evaluation and meaningful comparisons between methods are not trivial. We introduce BiasBed, a testbed for texture- and style-biased training, including multiple datasets and a range of existing algorithms. It comes with an extensive evaluation protocol that includes rigorous hypothesis testing to gauge the significance of the results, despite the considerable training instability of some style bias methods. Our extensive experiments, shed new light on the need for careful, statistically founded evaluation protocols for style bias (and beyond). E.g., we find that some algorithms proposed in the literature do not significantly mitigate the impact of style bias at all. With the release of BiasBed, we hope to foster a common understanding of consistent and meaningful comparisons, and consequently faster progress towards learning methods free of texture bias. Code is available at https://github.com/D1noFuzi/BiasBed

cs.CV↗

arXiv2022

On the orthogonality of zero-mean Gaussian measures: Sufficiently dense sampling

For a stationary random function $\xi$, sampled on a subset $D$ of $\mathbb{R}^{d}$, we examine the equivalence and orthogonality of two zero-mean Gaussian measures $\mathbb{P}_{1}$ and $\mathbb{P}_{2}$ associated with $\xi$. We give the isotropic analog to the result that the equivalence of $\mathbb{P}_{1}$ and $\mathbb{P}_{2}$ is linked with the existence of a square-integrable extension of the difference between the covariance functions of $\mathbb{P}_{1}$ and $\mathbb{P}_{2}$ from $D$ to $\mathbb{R}^{d}$. We show that the orthogonality of $\mathbb{P}_{1}$ and $\mathbb{P}_{2}$ can be recovered when the set of distances from points of $D$ to the origin is dense in the set of non-negative real numbers.

math.PR↗

arXiv2022

Dominant-feature identification in data from Gaussian processes applied to Finnish forest inventory records

In spatial data, location-dependent variation leads to connected structures known as features. Variations occur at different spatial scales and possibly originate from distinct underlying processes. Each of these scales is characterized by its own dominant features. Here we introduce a statistical method for identifying these scales and their dominant features in data from Gaussian processes. This identification involves credibly recognizing the dominant features by scale-space decomposition and assessing feature attributes by estimating covariance function parameters of the underlying processes and their associations to potential drivers. We analyze Finnish forest inventory data from the 1920s using this dominant-feature identification method and identify the scales of variation in basal area estimates of most common Finnish trees, including Scots pine, Norway spruce, birch, and other native deciduous trees. Comparing the resulting scale-dependent features and their attributes in these tree species, we identify the different effects of edaphic and anthropogenic drivers on the spatial distribution of their basal areas. These data are analyzed for the first time in terms of their scale of variation, and the resulting scale-dependent maps and estimates are an essential contribution to the historical forest ecology of Fennoscandia. Until now, this analysis was not possible with conventional methods.

stat.ME↗

arXiv2022

EggCounts: a Bayesian hierarchical toolkit to model faecal egg count reductions

This is a vignette for the R package eggCounts version 2.0. The package implements a suite of Bayesian hierarchical models dealing with faecal egg count reductions. The models are designed for a variety of practical situations, including individual treatment efficacy, zero inflation, small sample size (less than 10) and potential outliers. The functions are intuitive to use and their output are easy to interpret, such that users are protected from being exposed to complex Bayesian hierarchical modelling tasks. In addition, the package includes plotting functions to display data and results in a visually appealing manner. The models are implemented in Stan modelling language, which provides efficient sampling technique to obtain posterior samples. This vignette briefly introduces different models, and provides a short walk-through analysis with example data.

stat.CO↗

arXiv2021

Asymptotic analysis of ML-covariance parameter estimators based on covariance approximations

Given a zero-mean Gaussian random field with a covariance function that belongs to a parametric family of covariance functions, we introduce a new notion of likelihood approximations, termed truncated-likelihood functions. Truncated-likelihood functions are based on direct functional approximations of the presumed family of covariance functions. For compactly supported covariance functions, within an increasing-domain asymptotic framework, we provide sufficient conditions under which consistency and asymptotic normality of estimators based on truncated-likelihood functions are preserved. We apply our result to the family of generalized Wendland covariance functions and discuss several examples of Wendland approximations. For families of covariance functions that are not compactly supported, we combine our results with the covariance tapering approach and show that ML estimators, based on truncated-tapered likelihood functions, asymptotically minimize the Kullback-Leibler divergence, when the taper range is fixed.

math.ST↗

arXiv2021

Discussion on Competition for Spatial Statistics for Large Datasets

We discuss the experiences and results of the AppStatUZH team's participation in the comprehensive and unbiased comparison of different spatial approximations conducted in the Competition for Spatial Statistics for Large Datasets. In each of the different sub-competitions, we estimated parameters of the covariance model based on a likelihood function and predicted missing observations with simple kriging. We approximated the covariance model either with covariance tapering or a compactly supported Wendland covariance function.

stat.ME↗

arXiv2021

varycoef: An R Package for Gaussian Process-based Spatially Varying Coefficient Models

Gaussian processes (GPs) are well-known tools for modeling dependent data with applications in spatial statistics, time series analysis, or econometrics. In this article, we present the R package varycoef that implements estimation, prediction, and variable selection of linear models with spatially varying coefficients (SVC) defined by GPs, so called GP-based SVC models. Such models offer a high degree of flexibility while being relatively easy to interpret. Using varycoef, we show versatile applications of (spatially) varying coefficient models on spatial and time series data. This includes model and coefficient estimation with predictions and variable selection. The package uses state-of-the-art computational statistics techniques like parallelization, model-based optimization, and covariance tapering. This allows the user to work with (S)VC models in a computationally efficient manner, i.e., model estimation on large data sets is possible in a feasible amount of time.

stat.CO↗

arXiv2021

Joint Variable Selection of both Fixed and Random Effects for Gaussian Process-based Spatially Varying Coefficient Models

Spatially varying coefficient (SVC) models are a type of regression model for spatial data where covariate effects vary over space. If there are several covariates, a natural question is which covariates have a spatially varying effect and which not. We present a new variable selection approach for Gaussian process-based SVC models. It relies on a penalized maximum likelihood estimation (PMLE) and allows variable selection both with respect to fixed effects and Gaussian process random effects. We validate our approach both in a simulation study as well as a real world data set. Our novel approach shows good selection performance in the simulation study. In the real data application, our proposed PMLE yields sparser SVC models and achieves a smaller information criterion than classical MLE. In a cross-validation applied on the real data, we show that sparser PML estimated SVC models are on par with ML estimated SVC models with respect to predictive performance.

stat.ME↗

arXiv2020

Identification of Dominant Features in Spatial Data

Dominant features of spatial data are connected structures or patterns that emerge from location-based variation and manifest at specific scales or resolutions. To identify dominant features, we propose a sequential application of multiresolution decomposition and variogram function estimation. Multiresolution decomposition separates data into additive components, and in this way enables the recognition of their dominant features. A dedicated multiresolution decomposition method is developed for arbitrary gridded spatial data, where the underlying model includes a precision and spatial-weight matrix to capture spatial correlation. The data are separated into their components by smoothing on different scales, such that larger scales have longer spatial correlation ranges. Moreover, our model can handle missing values, which is often useful in applications. Variogram function estimation can be used to describe properties in spatial data. Such functions are therefore estimated for each component to determine its effective range, which assesses the width-extent of the dominant feature. Finally, Bayesian analysis enables the inference of identified dominant features and to judge whether these are credibly different. The efficient implementation of the method relies mainly on a sparse-matrix data structure and algorithms. By applying the method to simulated data we demonstrate its applicability and theoretical soundness. In disciplines that use spatial data, this method can lead to new insights, as we exemplify by identifying the dominant features in a forest dataset. In that application, the width-extents of the dominant features have an ecological interpretation, namely the species interaction range, and their estimates support the derivation of ecosystem properties such as biodiversity indices.

stat.ME↗

arXiv2020

Maximum Likelihood Estimation of Spatially Varying Coefficient Models for Large Data with an Application to Real Estate Price Prediction

In regression models for spatial data, it is often assumed that the marginal effects of covariates on the response are constant over space. In practice, this assumption might often be questionable. In this article, we show how a Gaussian process-based spatially varying coefficient (SVC) model can be estimated using maximum likelihood estimation (MLE). In addition, we present an approach that scales to large data by applying covariance tapering. We compare our methodology to existing methods such as a Bayesian approach using the stochastic partial differential equation (SPDE) link, geographically weighted regression (GWR), and eigenvector spatial filtering (ESF) in both a simulation study and an application where the goal is to predict prices of real estate apartments in Switzerland. The results from both the simulation study and application show that the MLE approach results in increased predictive accuracy and more precise estimates. Since we use a model-based approach, we can also provide predictive variances. In contrast to existing model-based approaches, our method scales better to data where both the number of spatial points is large and the number of spatially varying covariates is moderately-sized, e.g., above ten.

stat.ME↗