arXiv subjects

Robert J. Taggart

Publications and source records attributed to Robert J. Taggart.

8 recordsLinked to original sources

arXiv2026

Evaluating time-to-event forecasts under fixed-horizon right-censoring

Time-to-event forecasts frequently arise in applications with finite prediction or decision horizons, including in operational meteorology, hydrology, warranty analysis and insurance. In such settings, forecasts, observations, or both may only be available through their right-censored values relative to a common fixed censoring time. This article develops a framework for evaluating such forecasts. For probabilistic forecasts, recent localization results for proper scoring rules imply coherent evaluation methods based on threshold-weighted versions of familiar scores, including the continuous ranked probability score (CRPS) and logarithmic score. These methods remain valid when forecasts themselves are right-censored to the prediction horizon. For single-valued and interval-valued forecasts, we develop corresponding notions of local consistency and local elicitability under fixed-horizon right-censoring. We show that quantiles and prediction intervals whose endpoints are quantiles remain locally elicitable through threshold-weighted variants of quantile and interval scores. In contrast, the expectation functional is not locally elicitable, implying that mean-valued time-to-event forecasts do not admit analogous theoretically justified evaluation methods. Several familiar diagnostic tools extend naturally to this setting. Threshold-weighted CRPS and quantile scores retain useful diagnostic representations through Brier-score decompositions and Murphy diagrams, while reliability diagrams based on quantile isotonic regression support diagnosis of conditional biases and forecast recalibration. A synthetic forecasting experiment together with operational flood and wind forecasting applications illustrates the methodology.

math.ST↗

arXiv2026

On equitable scoring functions and optimal forecasting behaviour

Equitable scoring functions have a long history in meteorological forecast verification and have recently gained renewed prominence through the use of the Stable Equitable Error in Probability Space (SEEPS) score for evaluating precipitation forecasts from numerical and machine-learning weather prediction systems. This paper provides a systematic analysis of equitable scoring functions through the optimal forecasts they induce. We introduce the generalized diagonal score, which defines a broad class of equitable scoring functions that includes, up to equivalence, the SEEPS, Gerrity, Peirce, Barnston and diagonal scores. Within this framework, we show that optimal single-valued and categorical forecasts are characterized by crossing points between the predictive and climatological cumulative distribution functions. Moreover, we show that the generalized diagonal score, when evaluating predictive distributions, is proper but not strictly proper, and is insensitive to substantial forecast misspecification away from crossing points. The framework also yields scoring rules that are both proper and equitable for probabilistic forecasts with categorical outcomes. For single-valued and categorical forecasts, the crossing-point characterization shows that optimal forecasts can vary across climatologies even when the underlying predictive distribution is unchanged. Consequently, identical predictive distributions may lead to substantially different optimal forecasts under equitable scores, with important implications for assessing the suitability of such scores for any given application.

math.ST↗

arXiv2025

Warnings based on risk matrices: a coherent framework with consistent evaluation

Risk matrices are widely used across a range of fields and have found increasing utility in warning decision practices globally. However, their application in this context presents challenges, which range from potentially perverse warning outcomes to a lack of objective verification (i.e., evaluation) methods. This paper introduces a coherent framework for generating multi-level warnings from risk matrices to address these challenges. The proposed framework is general, is based on probabilistic forecasts of hazard severity or impact and is compatible with the Common Alerting Protocol (CAP). Moreover, it includes a family of consistent scoring functions for objectively evaluating the predictive performance of risk matrix assessments and the warnings they produce. These scoring functions enable the ranking of forecasters or warning systems and the tracking of system improvements by rewarding accurate probabilistic forecasts and compliance with warning service directives. A synthetic experiment demonstrates the efficacy of these scoring functions, while the framework is illustrated through warnings for heavy rainfall based on operational ensemble prediction system forecasts for Tropical Cyclone Jasper (Queensland, Australia, 2023). This work establishes a robust foundation for enhancing the reliability and verifiability of risk-based warning systems.

stat.AP↗

arXiv2024

scores: A Python package for verifying and evaluating models and predictions with xarray

`scores` is a Python package containing mathematical functions for the verification, evaluation and optimisation of forecasts, predictions or models. It supports labelled n-dimensional (multidimensional) data, which is used in many scientific fields and in machine learning. At present, `scores` primarily supports the geoscience communities; in particular, the meteorological, climatological and oceanographic communities. `scores` not only includes common scores (e.g., Mean Absolute Error), it also includes novel scores not commonly found elsewhere (e.g., FIxed Risk Multicategorical (FIRM) score, Flip-Flop Index), complex scores (e.g., threshold-weighted continuous ranked probability score), and statistical tests (such as the Diebold Mariano test). It also contains isotonic regression which is becoming an increasingly important tool in forecast verification and can be used to generate stable reliability diagrams. Additionally, it provides pre-processing tools for preparing data for scores in a variety of formats including cumulative distribution functions (CDF). At the time of writing, `scores` includes over 50 metrics, statistical techniques and data processing tools. All of the scores and statistical techniques in this package have undergone a thorough scientific and software review. Every score has a companion Jupyter Notebook tutorial that demonstrates its use in practice. `scores` supports `xarray` datatypes, allowing it to work with Earth system data in a range of formats including NetCDF4, HDF5, Zarr and GRIB among others. `scores` uses Dask for scaling and performance. Support for `pandas` is being introduced. The `scores` software repository can be found at https://github.com/nci/scores/

physics.ao-ph↗

arXiv2022

Point forecasting and forecast evaluation with generalized Huber loss

Huber loss, its asymmetric variants and their associated functionals (here named Huber functionals) are studied in the context of point forecasting and forecast evaluation. The Huber functional of a distribution is the set of minimizers of the expected (asymmetric) Huber loss, is an intermediary between a quantile and corresponding expectile, and also arises in M-estimation. Each Huber functional is elicitable, generating the precise set of minimizers of an expected score, subject to weak regularity conditions on the class of probability distributions, and has a complete characterization of its consistent scoring functions. Such scoring functions admit a mixture representation as a weighted average of elementary scoring functions. Each elementary score can be interpreted as the relative economic loss of using a particular forecast for a class of investment decisions where profits and losses are capped. The relevance of this theory for comparative assessment of weather forecasts is also discussed.

math.ST↗

arXiv2021

Evaluation of point forecasts for extreme events using consistent scoring functions

We present a method for comparing point forecasts in a region of interest, such as the tails or centre of a variable's range. This method cannot be hedged, in contrast to conditionally selecting events to evaluate and then using a scoring function that would have been consistent (or proper) prior to event selection. Our method also gives decompositions of scoring functions that are consistent for the mean or a particular quantile or expectile. Each member of each decomposition is itself a consistent scoring function that emphasises performance over a selected region of the variable's range. The score of each member of the decomposition has a natural interpretation rooted in optimal decision theory. It is the weighted average of economic regret over user decision thresholds, where the weight emphasises those decision thresholds in the corresponding region of interest.

stat.AP↗

arXiv2012

Potential maps, Hardy spaces, and tent spaces on special Lipschitz domains

Suppose that $Ω$ is the open region in $\mathbb{R}^n$ above a Lipschitz graph and let $d$ denote the exterior derivative on $\mathbb{R}^n$. We construct a convolution operator $T $ which preserves support in $\bar{Ω$}, is smoothing of order 1 on the homogeneous function spaces, and is a potential map in the sense that $dT$ is the identity on spaces of exact forms with support in $\barΩ$. Thus if $f$ is exact and supported in $\barΩ$, then there is a potential $u$, given by $u=Tf$, of optimal regularity and supported in $\barΩ$, such that $du=f$. This has implications for the regularity in homogeneous function spaces of the de Rham complex on $Ω$ with or without boundary conditions. The operator $T$ is used to obtain an atomic characterisation of Hardy spaces $H^p$ of exact forms with support in $\barΩ$ when $n/(n+1)<p\leq1$. This is done via an atomic decomposition of functions in the tent spaces $\mathcal T^p(\mathbb{R}^n\times\mathbb{R}^+)$ with support in a tent $T(Ω)$ as a sum of atoms with support away from the boundary of $Ω$. This new decomposition of tent spaces is useful, even for scalar valued functions.

math.AP↗

arXiv2008

Pointwise convergence for semigroups in vector-valued $L^p$ spaces

Suppose that T_t is a symmetric diffusion semigroup on L^2(X) and consider its tensor product extension to the Bochner space L^p(X,B), where B belongs to a certain broad class of UMD spaces. We prove a vector-valued version of the Hopf--Dunford--Schwartz ergodic theorem and show that this extends to a maximal theorem for analytic continuations of the semigroup's extension to L^p(X,B). As an application, we show that such continuations exhibit pointwise convergence.

math.FA↗