arXiv subjects

Sebastian Engelke

Publications and source records attributed to Sebastian Engelke.

At least 37 records · Page 2Linked to original sources

arXiv2024

Progression: an extrapolation principle for regression

The problem of regression extrapolation, or out-of-distribution generalization, arises when predictions are required at test points outside the range of the training data. In such cases, the non-parametric guarantees for regression methods from both statistics and machine learning typically fail. Based on the theory of tail dependence, we propose a novel statistical extrapolation principle. After a suitable, data-adaptive marginal transformation, it assumes a simple relationship between predictors and the response at the boundary of the training predictor samples. This assumption holds for a wide range of models, including non-parametric regression functions with additive noise. Our semi-parametric method, progression, leverages this extrapolation principle and offers guarantees on the approximation error beyond the training data range. We demonstrate how this principle can be effectively integrated with existing approaches, such as random forests and additive models, to improve extrapolation performance on out-of-distribution samples.

stat.ME↗

arXiv2024

Graphical models for infinite measures with applications to extremes

Conditional independence and graphical models are well studied for probability distributions on product spaces. We propose a new notion of conditional independence for any measure $Λ$ on the punctured Euclidean space $\mathbb R^d\setminus \{0\}$ that explodes at the origin. The importance of such measures stems from their connection to infinitely divisible and max-infinitely divisible distributions, where they appear as Lévy measures and exponent measures, respectively. We characterize independence and conditional independence for $Λ$ in various ways through kernels and factorization of a modified density, including a Hammersley-Clifford type theorem for undirected graphical models. As opposed to the classical conditional independence, our notion is intimately connected to the support of the measure $Λ$. Our general theory unifies and extends recent approaches to graphical modeling in the fields of extreme value analysis and Lévy processes. Our results for the corresponding undirected and directed graphical models lay the foundation for new statistical methodology in these areas.

math.ST↗

arXiv2024

Graphical models for multivariate extremes

Graphical models in extremes have emerged as a diverse and quickly expanding research area in extremal dependence modeling. They allow for parsimonious statistical methodology and are particularly suited for enforcing sparsity in high-dimensional problems. In this work, we provide the fundamental concepts of extremal graphical models and discuss recent advances in the field. Different existing perspectives on graphical extremes are presented in a unified way through graphical models for exponent measures. We discuss the important cases of nonparametric extremal graphical models on simple graph structures, and the parametric class of Hüsler--Reiss models on arbitrary undirected graphs. In both cases, we describe model properties, methods for statistical inference on known graph structures, and structure learning algorithms when the graph is unknown. We illustrate different methods in an application to flight delay data at US airports.

stat.ME↗

arXiv2024

Extremal Random Forests

Classical methods for quantile regression fail in cases where the quantile of interest is extreme and only few or no training data points exceed it. Asymptotic results from extreme value theory can be used to extrapolate beyond the range of the data, and several approaches exist that use linear regression, kernel methods or generalized additive models. Most of these methods break down if the predictor space has more than a few dimensions or if the regression function of extreme quantiles is complex. We propose a method for extreme quantile regression that combines the flexibility of random forests with the theory of extrapolation. Our extremal random forest (ERF) estimates the parameters of a generalized Pareto distribution, conditional on the predictor vector, by maximizing a local likelihood with weights extracted from a quantile random forest. We penalize the shape parameter in this likelihood to regularize its variability in the predictor space. Under general domain of attraction conditions, we show consistency of the estimated parameters in both the unpenalized and penalized case. Simulation studies show that our ERF outperforms both classical quantile regression methods and existing regression approaches from extreme value theory. We apply our methodology to extreme quantile prediction for U.S. wage data.

stat.ME↗

arXiv2023

Statistical Inference for Hüsler-Reiss Graphical Models Through Matrix Completions

The severity of multivariate extreme events is driven by the dependence between the largest marginal observations. The Hüsler-Reiss distribution is a versatile model for this extremal dependence, and it is usually parameterized by a variogram matrix. In order to represent conditional independence relations and obtain sparse parameterizations, we introduce the novel Hüsler-Reiss precision matrix. Similarly to the Gaussian case, this matrix appears naturally in density representations of the Hüsler-Reiss Pareto distribution and encodes the extremal graphical structure through its zero pattern. For a given, arbitrary graph we prove the existence and uniqueness of the completion of a partially specified Hüsler-Reiss variogram matrix so that its precision matrix has zeros on non-edges in the graph. Using suitable estimators for the parameters on the edges, our theory provides the first consistent estimator of graph structured Hüsler-Reiss distributions. If the graph is unknown, our method can be combined with recent structure learning algorithms to jointly infer the graph and the corresponding parameter matrix. Based on our methodology, we propose new tools for statistical inference of sparse Hüsler-Reiss models and illustrate them on large flight delay data in the U.S., as well as Danube river flow data.

stat.ME↗

arXiv2023

Extremal Dependence of Moving Average Processes Driven by Exponential-Tailed Lévy Noise

Moving average processes driven by exponential-tailed Lévy noise are important extensions of their Gaussian counterparts in order to capture deviations from Gaussianity, more flexible dependence structures, and sample paths with jumps. Popular examples include non-Gaussian Ornstein--Uhlenbeck processes and type G Matérn stochastic partial differential equation random fields. This paper is concerned with the open problem of determining their extremal dependence structure. We leverage the fact that such processes admit approximations on grids or triangulations that are used in practice for efficient simulations and inference. These approximations can be expressed as special cases of a class of linear transformations of independent, exponential-tailed random variables, that bridge asymptotic dependence and independence in a novel, tractable way. This result is of independent interest since models that can capture both extremal dependence regimes are scarce and the construction of such flexible models is an active area of research. This new fundamental result allows us to show that the integral approximation of general moving average processes with exponential-tailed Lévy noise is asymptotically independent when the mesh is fine enough. Under mild assumptions on the kernel function we also derive the limiting residual tail dependence function. For the popular exponential-tailed Ornstein--Uhlenbeck process we prove that it is asymptotically independent, but with a different residual tail dependence function than its Gaussian counterpart. Our results are illustrated through simulation studies.

math.ST↗

arXiv2023

Estimation and Inference of Extremal Quantile Treatment Effects for Heavy-Tailed Distributions

Causal inference for extreme events has many potential applications in fields such as climate science, medicine and economics. We study the extremal quantile treatment effect of a binary treatment on a continuous, heavy-tailed outcome. Existing methods are limited to the case where the quantile of interest is within the range of the observations. For applications in risk assessment, however, the most relevant cases relate to extremal quantiles that go beyond the data range. We introduce an estimator of the extremal quantile treatment effect that relies on asymptotic tail approximation, and use a new causal Hill estimator for the extreme value indices of potential outcome distributions. We establish asymptotic normality of the estimators and propose a consistent variance estimator to achieve valid statistical inference. We illustrate the performance of our method in simulation studies, and apply it to a real data set to estimate the extremal quantile treatment effect of college education on wage.

stat.ME↗

arXiv2023

Total positivity in multivariate extremes

Positive dependence is present in many real world data sets and has appealing stochastic properties that can be exploited in statistical modeling and in estimation. In particular, the notion of multivariate total positivity of order 2 ($ \mathrm{MTP}_{2} $) is a convex constraint and acts as an implicit regularizer in the Gaussian case. We study positive dependence in multivariate extremes and introduce $ \mathrm{EMTP}_{2} $, an extremal version of $ \mathrm{MTP}_{2} $. This notion turns out to appear prominently in extremes, and in fact, it is satisfied by many classical models. For a Hüsler--Reiss distribution, the analogue of a Gaussian distribution in extremes, we show that it is $ \mathrm{EMTP}_{2} $ if and only if its precision matrix is a Laplacian of a connected graph. We propose an estimator for the parameters of the Hüsler--Reiss distribution under $ \mathrm{EMTP}_{2} $ as the solution of a convex optimization problem with Laplacian constraint. We prove that this estimator is consistent and typically yields a sparse model with possibly nondecomposable extremal graphical structure. Applying our methods to a data set of Danube River flows, we illustrate this regularization and the superior performance compared to existing methods.

math.ST↗

arXiv2023

Asymmetric dependence in hydrological extremes

Extremal dependence describes the strength of correlation between the largest observations of two variables. It is usually measured with symmetric dependence coefficients that do not depend on the order of the variables. In many cases, there is a natural asymmetry between extreme observations that can not be captured by such coefficients. An example for such asymmetry are large discharges at an upstream and a downstream stations on a river network: an extreme discharge at the upstream station will directly influence the discharge at the downstream station, but not vice versa. Simple measures for asymmetric dependence in extreme events have not yet been investigated. We propose the asymmetric tail Kendall's $τ$ as a measure for extremal dependence that is sensitive to asymmetric behaviour in the largest observations. It essentially computes the classical Kendall's $τ$ but conditioned on the extreme observations of one of the two variables. We show theoretical properties of this new coefficient and derive a formula to compute it for existing copula models. We further study its effectiveness and connections to causality in simulation experiments. We apply our methodology to a case study on river networks in the United Kingdom to illustrate the importance of measuring asymmetric extremal dependence in hydrology. Our results show that there is important structural information in the asymmetry that would have been missed by a symmetric measure. Our methodology is an easy but effective tool that can be applied in exploratory analysis for understanding the connections among variables and to detect possible asymmetric dependencies.

stat.ME↗

arXiv2022

Gradient boosting for extreme quantile regression

Extreme quantile regression provides estimates of conditional quantiles outside the range of the data. Classical quantile regression performs poorly in such cases since data in the tail region are too scarce. Extreme value theory is used for extrapolation beyond the range of observed values and estimation of conditional extreme quantiles. Based on the peaks-over-threshold approach, the conditional distribution above a high threshold is approximated by a generalized Pareto distribution with covariate dependent parameters. We propose a gradient boosting procedure to estimate a conditional generalized Pareto distribution by minimizing its deviance. Cross-validation is used for the choice of tuning parameters such as the number of trees and the tree depths. We discuss diagnostic plots such as variable importance and partial dependence plots, which help to interpret the fitted models. In simulation studies we show that our gradient boosting procedure outperforms classical methods from quantile regression and extreme value theory, especially for high-dimensional predictor spaces and complex parameter response surfaces. An application to statistical post-processing of weather forecasts with precipitation data in the Netherlands is proposed.

stat.ME↗

arXiv2022

Structure learning for extremal tree models

Extremal graphical models are sparse statistical models for multivariate extreme events. The underlying graph encodes conditional independencies and enables a visual interpretation of the complex extremal dependence structure. For the important case of tree models, we develop a data-driven methodology for learning the graphical structure. We show that sample versions of the extremal correlation and a new summary statistic, which we call the extremal variogram, can be used as weights for a minimum spanning tree to consistently recover the true underlying tree. Remarkably, this implies that extremal tree models can be learned in a completely non-parametric fashion by using simple summary statistics and without the need to assume discrete distributions, existence of densities, or parametric models for bivariate distributions.

stat.ME↗

arXiv2022

Modeling panels of extremes

Extreme value applications commonly employ regression techniques to capture cross-sectional heterogeneity or time-variation in the data. Estimation of the parameters of an extreme value regression model is notoriously challenging due to the small number of observations that are usually available in applications. When repeated extreme measurements are collected on the same individuals, i.e., a panel of extremes is available, pooling the observations in groups can improve the statistical inference. We study three data sets related to risk assessment in finance, climate science, and hydrology. In all three cases, the problem can be formulated as an extreme value panel regression model with a latent group structure and group-specific parameters. We propose a new algorithm that jointly assigns the individuals to the latent groups and estimates the parameters of the regression model inside each group. Our method efficiently recovers the underlying group structure without prior information, and for the three data sets it provides improved return level estimates and helps answer important domain-specific questions.

stat.ME↗

arXiv2022

Modelling and simulating spatial extremes by combining extreme value theory with generative adversarial networks

Modelling dependencies between climate extremes is important for climate risk assessment, for instance when allocating emergency management funds. In statistics, multivariate extreme value theory is often used to model spatial extremes. However, most commonly used approaches require strong assumptions and are either too simplistic or over-parameterized. From a machine learning perspective, Generative Adversarial Networks (GANs) are a powerful tool to model dependencies in high-dimensional spaces. Yet in the standard setting, GANs do not well represent dependencies in the extremes. Here we combine GANs with extreme value theory (evtGAN) to model spatial dependencies in summer maxima of temperature and winter maxima in precipitation over a large part of western Europe. We use data from a stationary 2000-year climate model simulation to validate the approach and explore its sensitivity to small sample sizes. Our results show that evtGAN outperforms classical GANs and standard statistical approaches to model spatial extremes. Already with about 50 years of data, which corresponds to commonly available climate records, we obtain reasonably good performance. In general, dependencies between temperature extremes are better captured than dependencies between precipitation extremes due to the high spatial coherence in temperature fields. Our approach can be applied to other climate variables and can be used to emulate climate models when running very long simulations to determine dependencies in the extremes is deemed infeasible.

stat.AP↗

arXiv2021

Distributionally robust tail bounds based on Wasserstein distance and $f$-divergence

In this work, we provide robust bounds on the tail probabilities and the tail index of heavy-tailed distributions in the context of model misspecification. They are defined as the optimal value when computing the worst-case tail behavior over all models within some neighborhood of the reference model. The choice of the discrepancy between the models used to build this neighborhood plays a crucial role in assessing the size of the asymptotic bounds. We evaluate the robust tail behavior in ambiguity sets based on the Wasserstein distance and Csiszár $f$-divergence and obtain explicit expressions for the corresponding asymptotic bounds. In an application to Danish fire insurance claims we compare the difference between these bounds and show the importance of the choice of discrepancy measure.

math.ST↗

arXiv2021

Rank-based Estimation under Asymptotic Dependence and Independence, with Applications to Spatial Extremes

Multivariate extreme value theory is concerned with modeling the joint tail behavior of several random variables. Existing work mostly focuses on asymptotic dependence, where the probability of observing a large value in one of the variables is of the same order as observing a large value in all variables simultaneously. However, there is growing evidence that asymptotic independence is equally important in real world applications. Available statistical methodology in the latter setting is scarce and not well understood theoretically. We revisit non-parametric estimation and introduce rank-based M-estimators for parametric models that simultaneously work under asymptotic dependence and asymptotic independence, without requiring prior knowledge on which of the two regimes applies. Asymptotic normality of the proposed estimators is established under weak regularity conditions. We further show how bivariate estimators can be leveraged to obtain parametric estimators in spatial tail models, and again provide a thorough theoretical justification for our approach.

math.ST↗

arXiv2020

Causal discovery in heavy-tailed models

Causal questions are omnipresent in many scientific problems. While much progress has been made in the analysis of causal relationships between random variables, these methods are not well suited if the causal mechanisms only manifest themselves in extremes. This work aims to connect the two fields of causal inference and extreme value theory. We define the causal tail coefficient that captures asymmetries in the extremal dependence of two random variables. In the population case, the causal tail coefficient is shown to reveal the causal structure if the distribution follows a linear structural causal model. This holds even in the presence of latent common causes that have the same tail index as the observed variables. Based on a consistent estimator of the causal tail coefficient, we propose a computationally highly efficient algorithm that estimates the causal structure. We prove that our method consistently recovers the causal order and we compare it to other well-established and non-extremal approaches in causal discovery on synthetic and real data. The code is available as an open-access R package.

stat.ME↗

arXiv2020

Sparse Structures for Multivariate Extremes

Extreme value statistics provides accurate estimates for the small occurrence probabilities of rare events. While theory and statistical tools for univariate extremes are well-developed, methods for high-dimensional and complex data sets are still scarce. Appropriate notions of sparsity and connections to other fields such as machine learning, graphical models and high-dimensional statistics have only recently been established. This article reviews the new domain of research concerned with the detection and modeling of sparse patterns in rare events. We first describe the different forms of extremal dependence that can arise between the largest observations of a multivariate random vector. We then discuss the current research topics including clustering, principal component analysis and graphical modeling for extremes. Identification of groups of variables which can be concomitantly extreme is also addressed. The methods are illustrated with an application to flood risk assessment.

stat.ME↗

arXiv2019

Graphical Models for Extremes

Conditional independence, graphical models and sparsity are key notions for parsimonious statistical models and for understanding the structural relationships in the data. The theory of multivariate and spatial extremes describes the risk of rare events through asymptotically justified limit models such as max-stable and multivariate Pareto distributions. Statistical modelling in this field has been limited to moderate dimensions so far, partly owing to complicated likelihoods and a lack of understanding of the underlying probabilistic structures. We introduce a general theory of conditional independence for multivariate Pareto distributions that allows the definition of graphical models and sparsity for extremes. A Hammersley-Clifford theorem links this new notion to the factorization of densities of extreme value models on graphs. For the popular class of Hüsler-Reiss distributions we show that, similarly to the Gaussian case, the sparsity pattern of a general extremal graphical model can be read off from suitable inverse covariance matrices. New parametric models can be built in a modular way and statistical inference can be simplified to lower-dimensional marginals. We discuss learning of minimum spanning trees and model selection for extremal graph structures, and illustrate their use with an application to flood risk assessment on the Danube river.

math.ST↗