arXiv subjects

Jordan Richards

Publications and source records attributed to Jordan Richards.

At least 19 recordsLinked to original sources

arXiv2026

Optimising Temporary Accommodation Placement Across London with AI-Powered SaaS in E-Governance Systems

Temporary accommodation has become a major fiscal and administrative pressure for English local authorities, particularly in London, where demand and costs have risen sharply. This paper documents the creation and use of DOMUS, a cloud-based, AI-enabled decision-support system built from scratch at the University of East London and customised for the needs of London Borough of Newham to support statutory Temporary accommodation placement. DOMUS integrates household case records, policy-constrained affordability and suitability rules, and live private-rental listings within a single governance-aligned workflow. The system combines transparent, rule-based filtering with large language model-assisted search to standardise the application of bedroom need, affordability thresholds, geographic preferences, and accessibility requirements, while preserving officer discretion and audibility. Household and property attributes are encoded into policy-consistent representations prior to AI-assisted ranking and explanation. A pilot deployment in Newham's secure environment evaluated operational performance relative to manual workflows. Results indicate substantial reductions in search time, improved adherence to key placement constraints, and high staff satisfaction, while maintaining statutory compliance and role-based accountability. Beyond TA, the paper frames DOMUS as replicable digital public infrastructure: a modular, cloud-native Software-as-a-Service architecture that can be deployed across other UK boroughs and adapted to other public administration tasks characterised by scarcity, rule-bound eligibility, and high stakes. The findings demonstrate the feasibility of scalable, ethically governed AI deployment in local government and contribute to debates on AI-enabled public value creation in e-governance.

cs.CY↗

arXiv2026

Diagnostic Tools for Extreme Value Regression Models

Visual and quantitative goodness-of-fit diagnostics are an important tool in the practitioner's toolbox. The need for convincing and reliable diagnostics is particularly clear when fitting extreme value regression models, which are used for extrapolation far beyond the observable range of the response variable, and often evaluated at unobserved covariate values. Despite this, few diagnostics have been developed for extreme value regression models, and those available often suffer in terms of interpretability or scalability on low-dimensional or non-Euclidean covariate domains, often encountered in modern applications. Moreover, existing methods tend to offer a global perspective on model fit; that is, they quantify goodness-of-fit across the entire dataset, without offering insight into regions of the covariate space where the model fit may be poor. We propose two novel visual diagnostics for extreme value regression models: the standardised tail plot and the normalised residual plot. By considering the asymptotic distribution of normalised exceedance probabilities, we show that uncertainty bounds for our plots are approximately independent of the sample size used in their construction. This allows us to propose visual diagnostics which can efficiently and consistently compare goodness-of-fit at both a global and regional level, despite varying sample sizes over regions of the covariate domain. Following a discussion of summary statistics for global and regional goodness-of-fit, we provide two applications of extreme value regression models that illustrate how our diagnostics can be used to perform model comparison (across thousands of candidate models) and provide actionable findings that support model design.

stat.ME↗

arXiv2026

Spatial Extremes at Scale: A Case Study of Surface Skin Temperature and Heat Risk in the United States

Understanding and mapping extreme heat is critical for risk management and public health planning, particularly in regions with complex terrain and heterogeneous climate. We present a case study of extreme heat in the Four Corners region of the United States, using high-resolution surface skin temperature data from the North American Land Data Assimilation System to characterize spatially heterogeneous and seasonally varying extremes across complex terrain, and to assess their implications for heat-related public health risks. Spatial extremes exhibit complex dependencies across geographic regions, which require sophisticated statistical models to capture. While recent advances in spatial extreme value modeling provide flexible representations of joint tail dependencies, statistical inference remains computationally demanding, especially for datasets with a large number of locations. To address this, we propose a random scale mixture process that facilitates Bayesian inference of spatial extremes, and develop scalable inference strategies that leverage advances in spatial modeling and amortized learning. We evaluate the proposed inference methods through large-scale simulation studies, representing the first such extensive study in spatial extremes, and a high-resolution surface skin temperature application in the Four Corners region. Surface skin temperature is particularly useful as a predictor for air temperature, for studying heatwaves and related environmental phenomena, and to calculate heat indices reflecting downstream health risks at any location. Our findings provide insights into efficient, data-driven approaches for modeling spatial extremes, and serve as guidelines for practitioners in the fields of climate science, environmental risk assessment, and beyond.

stat.AP↗

arXiv2025

Quantile-based causal inference for spatio-temporal processes: Assessing the impacts of wildfires on US air quality

Wildfires pose an increasingly severe threat to air quality, yet quantifying their causal impact remains challenging due to unmeasured meteorological and geographic confounders. Moreover, wildfire impacts on air quality may exhibit heterogeneous effects across pollution levels, which conventional mean-based causal methods fail to capture. To address these challenges, we develop a Quantile-based Latent Spatial Confounder Model (QLSCM) that substitutes conditional expectations with conditional quantiles, enabling causal analysis across the entire outcome distribution. We establish the causal interpretation of QLSCM theoretically, prove the identifiability of causal effects, and demonstrate estimator consistency under mild conditions. Simulations confirm the bias correction capability and the advantage of quantile-based inference over mean-based approaches. Applying our method to contiguous US wildfire and air quality data, we uncover important heterogeneous effects: fire radiative power exerts significant positive causal effects on aerosol optical depth at high quantiles in Western states like California and Oregon, while insignificant at lower quantiles. This indicates that wildfire impacts on air quality primarily manifest during extreme pollution events. Regional analyses reveal that Western and Northwestern regions experience the strongest causal effects during such extremes. These findings provide critical insights for environmental policy by identifying where and when mitigation efforts would be most effective.

stat.AP↗

arXiv2025

Canonical Tail Dependence for Soft Extremal Clustering of Multichannel Brain Signals

We develop a novel characterization of extremal dependence between two cortical regions of the brain when its signals display extremely large amplitudes. We show that connectivity in the tails of the distribution reveals unique features of extreme events (e.g., seizures) that can help to identify their occurrence. Numerous studies have established that connectivity-based features are effective for discriminating brain states. Here, we demonstrate the advantage of the proposed approach: that tail connectivity provides additional discriminatory power, enabling more accurate identification of extreme-related events and improved seizure risk management. Common approaches in tail dependence modeling use pairwise summary measures or parametric models. However, these approaches do not identify channels that drive the maximal tail dependence between two groups of signals -- an information that is useful when analyzing electroencephalography of epileptic patients where specific channels are responsible for seizure occurrences. A familiar approach in traditional signal processing is canonical correlation, which we extend to the tails to develop a visualization of extremal channel-contributions. Through the tail pairwise dependence matrix (TPDM), we develop a computationally-efficient estimator for our canonical tail dependence measure. Our method is then used for accurate frequency-based soft clustering of neonates, distinguishing those with seizures from those without.

stat.ML↗

arXiv2025

Clustering of multivariate tail dependence using conditional methods

The conditional extremes (CE) framework has proven useful for analysing the joint tail behaviour of random vectors. However, when applied across many locations or variables, it can be difficult to interpret or compare the resulting extremal dependence structures, particularly for high dimensional vectors. To address this, we propose a novel clustering method for multivariate extremes using the CE framework. Our approach introduces a closed-form, computationally efficient dissimilarity measure for multivariate tails, based on the skew-geometric Jensen-Shannon divergence, and is applicable in arbitrary dimensions. Applying standard clustering algorithms to a matrix of pairwise distances, we obtain interpretable groups of random vectors with homogeneous tail dependence. Simulation studies demonstrate that our method outperforms existing approaches for clustering bivariate extremes, and uniquely extends to the multivariate setting. In our application to Irish meteorological data, our clustering identifies spatially coherent regions with similar extremal dependence between precipitation and wind speeds.

stat.ME↗

arXiv2025

MOPED: A moving sum method for change point detection in pairwise extremal dependence

It is increasingly the case with modern time series that many data sets of practical interest contain abrupt changes in structure. These changes may occur in complex characteristics such as the extremal dependence structure, and identifying such structural breaks remains a challenging problem. Many existing change point detection algorithms focus on changes in dependence across the entire distribution, rather than the tails, and approaches that are tailored to extremes typically make strict parametric assumptions or they are only applicable to bivariate data. We propose a nonparametric MOving sum-based approach for detecting multiple changes in the Pairwise Extremal Dependence (MOPED) of multivariate regularly varying data. To avoid the classical problem of threshold selection in the study of multivariate extremes, we further propose a multiscale, multi-threshold variant of MOPED that pools change point estimates across choices of the threshold and the bandwidth used in local estimation. Good performance of MOPED is illustrated in a simulation study, and we showcase its ability to identify subtle changes in tail dependence class in the absence of correlation changes. We further demonstrate the usefulness of MOPED by identifying changes in the extremal connectivity of electroencephalogram (EEG) signals of seizure-prone neonates.

stat.ME↗

arXiv2025

Modeling Nonstationary Extremal Dependence via Deep Spatial Deformations

Modeling nonstationarity that often prevails in extremal dependence of spatial data can be challenging, and typically requires bespoke or complex spatial models that are difficult to estimate. Inference for stationary and isotropic models is considerably easier, but the assumptions that underpin these models are rarely met by data observed over large or topographically complex domains. A possible approach for accommodating nonstationarity in a spatial model is to warp the spatial domain to a latent space where stationarity and isotropy can be reasonably assumed. Although this approach is very flexible, estimating the warping function can be computationally expensive, and the transformation is not always guaranteed to be bijective, which may lead to physically unrealistic transformations when the domain folds onto itself. We overcome these challenges by developing deep compositional spatial models to capture nonstationarity in extremal dependence. Specifically, we focus on modeling high threshold exceedances of process functionals by leveraging efficient inference methods for limiting r-Pareto processes. A detailed high-dimensional simulation study demonstrates the superior performance of our model in estimating the warped space. We illustrate our method by modeling UK precipitation extremes and show that we can efficiently estimate the extremal dependence structure of data observed at thousands of locations.

stat.ME↗

arXiv2025

Separation-based causal discovery for extremes

Structural causal models (SCMs), with an underlying directed acyclic graph (DAG), provide a powerful analytical framework to describe the interaction mechanisms in large-scale complex systems. However, when the system exhibits extreme events, the governing mechanisms can change dramatically, and SCMs with a focus on rare events are needed. We propose a new class of SCMs, called XSCMs, which leverage transformed-linear algebra to model causal relationships among extreme values. Similar to traditional SCMs, we prove that XSCMs satisfy the causal Markov and causal faithfulness properties with respect to partial tail (un)correlatedness. This enables estimation of the underlying DAG for extremes using separation-based tests, and makes many state-of-the-art constraint-based causal discovery algorithms directly applicable. We further consider the problem of undirected graph estimation for relationships among tail-dependent (and potentially heavy-tailed) data. The effectiveness of our method, compared to alternative approaches, is validated through simulation studies on large-scale systems with up to 50 variables, and in a well-studied application to river discharge data from the Danube basin. Finally, we apply the framework to investigate complex market-wide relationships in China's derivatives market.

stat.ME↗

arXiv2025

Generative modelling of multivariate geometric extremes using normalising flows

Leveraging the recently emerging geometric approach to multivariate extremes and the flexibility of normalising flows on the hypersphere, we propose a principled deep-learning-based methodology that enables accurate joint tail extrapolation in all directions. We exploit theoretical links between intrinsic model parameters defined as functions on hyperspheres to construct models ranging from high flexibility to parsimony, thereby enabling the efficient modelling of multivariate extremes displaying complex dependence structures in higher dimensions with reasonable sample sizes. We use the generative feature of normalising flows to perform fast probability estimation for arbitrary Borel risk regions via an efficient Monte Carlo integration scheme. The good properties of our estimators are demonstrated via a simulation study in up to ten dimensions. We apply our methodology to the analysis of low and high extremes of wind speeds. In particular, we find that our methodology enables probability estimation for non-trivial extreme events in relation to electricity production via wind turbines and reveals interesting structure in the underlying data.

stat.ME↗

arXiv2025

Semi-parametric bulk and tail regression using spline-based neural networks

Semi-parametric quantile regression (SPQR) is a flexible approach to density regression that learns a spline-based representation of conditional density functions using neural networks. As it makes no parametric assumptions about the underlying density, SPQR performs well for in-sample testing and interpolation. However, it can perform poorly when modelling heavy-tailed data or when asked to extrapolate beyond the range of observations, as it fails to satisfy any of the asymptotic guarantees provided by extreme value theory (EVT). To build semi-parametric density regression models that can be used for reliable tail extrapolation, we create the blended generalised Pareto (GP) distribution, which i) provides a model for the entire range of data and, via a smooth and continuous transition, ii) benefits from exact GP upper-tails without the need for intermediate threshold selection. We combine SPQR with our blended GP to create semi-parametric quantile regression for extremes (SPQRx), which provides a flexible semi-parametric approach to density regression that is compliant with traditional EVT. We handle interpretability of SPQRx through the use of model-agnostic variable importance scores, which provide the relative importance of a covariate for separately determining the bulk and tail of the conditional density. The efficacy of SPQRx is illustrated on simulated data, and an application to U.S. wildfire burnt areas from 1990-2020.

stat.ME↗

arXiv2025

Spectral Extremal Connectivity of Two-State Seizure Brain Waves

Coherence analysis plays a vital role in the study of functional brain connectivity. However, coherence captures only linear spectral associations, and thus can produce misleading findings when ignoring variations of connectivity in the tails of the distribution. This limitation becomes important when investigating extreme neural events that are characterized by large signal amplitudes. The focus of this paper is to examine connectivity in the tails of the distribution, as this reveals salient information that may be overlooked by standard methods. We develop a novel notion of spectral tail association of periodograms to study connectivity in the network of electroencephalogram (EEG) signals of seizure-prone neonates. We further develop a novel non-stationary extremal dependence model for multivariate time series that captures differences in extremal dependence during different brain phases, namely burst-suppression and non-burst-suppression. One advantage of our proposed approach is its ability to identify tail connectivity at key frequency bands that could be associated with outbursts of energy which may lead to seizures. We discuss these novel scientific findings alongside a comparison of the extremal behavior of brain signals for epileptic and non-epileptic patients.

stat.AP↗

arXiv2024

Deep learning joint extremes of metocean variables using the SPAR model

This paper presents a novel deep learning framework for estimating multivariate joint extremes of metocean variables, based on the Semi-Parametric Angular-Radial (SPAR) model. When considered in polar coordinates, the problem of modelling multivariate extremes is transformed to one of modelling an angular density, and the tail of a univariate radial variable conditioned on angle. In the SPAR approach, the tail of the radial variable is modelled using a generalised Pareto (GP) distribution, providing a natural extension of univariate extreme value theory to the multivariate setting. In this work, we show how the method can be applied in higher dimensions, using a case study for five metocean variables: wind speed, wind direction, wave height, wave period, and wave direction. The angular variable is modelled using a kernel density method, while the parameters of the GP model are approximated using fully-connected deep neural networks. Our approach provides great flexibility in the dependence structures that can be represented, together with computationally efficient routines for training the model. Furthermore, the application of the method requires fewer assumptions about the underlying distribution(s) compared to existing approaches, and an asymptotically justified means for extrapolating outside the range of observations. Using various diagnostic plots, we show that the fitted models provide a good description of the joint extremes of the metocean variables considered.

stat.ML↗

arXiv2024

The Efficient Tail Hypothesis: An Extreme Value Perspective on Market Efficiency

In econometrics, the Efficient Market Hypothesis posits that asset prices reflect all available information in the market. Several empirical investigations show that market efficiency drops when it undergoes extreme events. Many models for multivariate extremes focus on positive dependence, making them unsuitable for studying extremal dependence in financial markets where data often exhibit both positive and negative extremal dependence. To this end, we construct regular variation models on the entirety of $\mathbb{R}^d$ and develop a bivariate measure for asymmetry in the strength of extremal dependence between adjacent orthants. Our directional tail dependence (DTD) measure allows us to define the Efficient Tail Hypothesis (ETH) -- an analogue of the Efficient Market Hypothesis -- for the extremal behaviour of the market. Asymptotic results for estimators of DTD are described, and we discuss testing of the ETH via permutation-based methods and present novel tools for visualization. Empirical study of China's futures market leads to a rejection of the ETH and we identify potential profitable investment opportunities. To promote the research of microstructure in China's derivatives market, we open-source our high-frequency data, which are being collected continuously from multiple derivative exchanges.

q-fin.ST↗

arXiv2024

Deep Learning of Multivariate Extremes via a Geometric Representation

The study of geometric extremes, where extremal dependence properties are inferred from the deterministic limiting shapes of scaled sample clouds, provides an exciting approach to modelling the extremes of multivariate data. These shapes, termed limit sets, link together several popular extremal dependence modelling frameworks. Although the geometric approach is becoming an increasingly popular modelling tool, current inference techniques are limited to a low dimensional setting (d < 5), and generally require rigid modelling assumptions. In this work, we propose a range of novel theoretical results to aid with the implementation of the geometric extremes framework and introduce the first approach to modelling limit sets using deep learning. By leveraging neural networks, we construct asymptotically-justified yet flexible semi-parametric models for extremal dependence of high-dimensional data. We showcase the efficacy of our deep approach by modelling the complex extremal dependencies between meteorological and oceanographic variables in the North Sea off the coast of the UK.

stat.ME↗

arXiv2024

Extreme quantile regression with deep learning

Estimation of extreme conditional quantiles is often required for risk assessment of natural hazards in climate and geo-environmental sciences and for quantitative risk management in statistical finance, econometrics, and actuarial sciences. Interest often lies in extrapolating to quantile levels that exceed any past observations. Therefore, it is crucial to use a statistical framework that is well-adapted and especially designed for this purpose, and here extreme-value theory plays a key role. This chapter reviews how extreme quantile regression may be performed using theoretically-justified models, and how modern deep learning approaches can be harnessed in this context to enhance the model's performance in complex high-dimensional settings. The power of deep learning combined with the rigor of theoretically-justified extreme-value methods opens the door to efficient extreme quantile regression, in cases where both the number of covariates and the quantile level of interest can be simultaneously ``extreme''.

stat.ME↗

arXiv2023

Modern extreme value statistics for Utopian extremes

Capturing the extremal behaviour of data often requires bespoke marginal and dependence models which are grounded in rigorous asymptotic theory, and hence provide reliable extrapolation into the upper tails of the data-generating distribution. We present a toolbox of four methodological frameworks, motivated by modern extreme value theory, that can be used to accurately estimate extreme exceedance probabilities or the corresponding level in either a univariate or multivariate setting. Our frameworks were used to facilitate the winning contribution of Team Yalla to the EVA (2023) Conference Data Challenge, which was organised for the 13$^\text{th}$ International Conference on Extreme Value Analysis. This competition comprised seven teams competing across four separate sub-challenges, with each requiring the modelling of data simulated from known, yet highly complex, statistical distributions, and extrapolation far beyond the range of the available samples in order to predict probabilities of extreme events. Data were constructed to be representative of real environmental data, sampled from the fantasy country of "Utopia"

stat.ME↗

arXiv2023

Neural Bayes Estimators for Irregular Spatial Data using Graph Neural Networks

Neural Bayes estimators are neural networks that approximate Bayes estimators in a fast and likelihood-free manner. Although they are appealing to use with spatial models, where estimation is often a computational bottleneck, neural Bayes estimators in spatial applications have, to date, been restricted to data collected over a regular grid. These estimators are also currently dependent on a prescribed set of spatial locations, which means that the neural network needs to be re-trained for new data sets; this renders them impractical in many applications and impedes their widespread adoption. In this work, we employ graph neural networks to tackle the important problem of parameter point estimation from data collected over arbitrary spatial locations. In addition to extending neural Bayes estimation to irregular spatial data, our architecture leads to substantial computational benefits, since the estimator can be used with any configuration or number of locations and independent replicates, thus amortising the cost of training for a given spatial model. We also facilitate fast uncertainty quantification by training an accompanying neural Bayes estimator that approximates a set of marginal posterior quantiles. We illustrate our methodology on Gaussian and max-stable processes. Finally, we showcase our methodology on a data set of global sea-surface temperature, where we estimate the parameters of a Gaussian process model in 2161 spatial regions, each containing thousands of irregularly-spaced data points, in just a few minutes with a single graphics processing unit.

stat.ME↗