arXiv subjects

Matthew Chantry

Publications and source records attributed to Matthew Chantry.

At least 19 recordsLinked to original sources

arXiv2026

AIFS-TC: A simple correction competitive with the operational frontier for tropical cyclone intensity forecasting

AI weather models are in the process of revolutionising weather forecasting. While these models have been shown to achieve superior performance to physics-based NWP in forecasting tropical cyclone (TC) tracks, they tend to dramatically underestimate intensity. Here we present AIFS-TC, a simple correction to the AIFS-Single model that is competitive with the operational state-of-the-art for forecasting maximum wind speed and minimum central pressure at lead times of 12 h to seven days. This performance also holds for rapid intensification events. Notably, the entire system was autonomously designed and built by a large language model (Claude Fable 5) in a few hours, directed through a small number of natural-language prompts by a single domain scientist. That the operational frontier can be reached with an open-source AI forecast model (AIFS-Single) and relatively simple, cheap post-processing is significant for TC science, and points to agentic coding as a route to rapid exploration and progress in life-saving early-warning systems in other domains.

physics.ao-ph↗

arXiv2026

HourGlass: A probabilistic data-driven temporal downscaler for global and regional weather forecasting

Many forecast applications require high frequency temporal resolution, yet most state-of-the-art data-driven weather forecasting systems operate at 6-hourly resolution. Although direct hourly forecasting is possible, it suffers from error accumulation and temporal inconsistency. We introduce HourGlass, a probabilistic data-driven temporal downscaling method that reconstructs the evolution between forecast states. HourGlass is trained using variants of the continuous ranked probability score (CRPS) preserving small-scale spatial variability while encouraging temporal consistency. Unlike existing deterministic temporal downscaling approaches, which tend to produce overly smooth fields, HourGlass generates realistic probabilistic forecasts. Training on forecast trajectories rather than reanalysis or analysis data also avoids the temporal inconsistencies present in datasets used by previous methods. We evaluate HourGlass in two settings: AIFS-HourGlass, applied globally to ECMWF's AIFS-Single and AIFS-ENS forecast systems, and Bris-HourGlass, applied regionally to MET Norway's high-resolution stretched-grid ensemble model, Bris. Verification against observations shows that both models retain the skill of their underlying forecasting systems while producing temporally coherent hourly forecasts with realistic small-scale variability. Case studies demonstrate physically consistent evolution during rapidly developing weather events, including extratropical cyclones and organised convection. Hourly precipitation remains challenging: HourGlass improves the spatial realism of precipitation fields but still underestimates the most intense extremes, a common limitation of data-driven weather forecasting models. These results demonstrate that HourGlass effectively bridges the gap between 6-hourly data-driven forecasts and the hourly products required for operational regional and global forecasting.

physics.ao-ph↗

arXiv2026

Global reanalysis from observations alone with machine learning

Earth system reanalysis datasets are foundational for weather and climate research and provide the gridded training data used by most machine learning weather prediction systems. Here we show results from a prototype system that suggest that machine learning models trained only on Earth system observations can potentially be used to generate multi-decade global reanalyses without using physics-based numerical models. The resulting gridded fields capture large-scale atmospheric structure and variability across multiple timescales, while exhibiting signs of physical coherence in several key dynamical diagnostics. Evaluations of the prototype against held-out independent atmospheric observations indicate that the root mean square vector error of upper-level winds is close to that of ERA5 when compared at a consistent resolution, and that the standard deviation of the error at the surface is between that of 4th- and 5th-generation ECMWF reanalyses (ERA-Interim and ERA5). Furthermore, while traditional reanalysis production is computationally expensive, typically taking several years to produce, the reanalysis presented here was generated during the course of a single working day. These results suggest that observation-trained machine learning models offer a promising new approach for reanalysis production from observations alone.

physics.ao-ph↗

arXiv2026

AIFS-SUBS: Extending Data-Driven Forecasting to Sub-Seasonal Timescales

Data-driven models now rival numerical weather prediction in the medium range, but extending them to sub-seasonal lead times raises challenges absent at shorter horizons. Errors accumulate over long autoregressive rollouts, systematic biases grow with lead time, and several years of data must be held out for independent verification, even though machine-learning models otherwise benefit from longer training records. To address these challenges, we adapt ECMWF's AIFS-CRPS medium-range model. AIFS-SUBS adopts a 24h autoregressive time step to reduce error accumulation, adds stratospheric levels and top-of-atmosphere thermal radiation as predictors, and reserves 2007--2011 as an independent verification window. We evaluate two config-durations: AIFS-SUBS, fine-tuned on operational analyses, and AIFS-SUBS-ERA5, trained on ERA5 alone. Across weeks 2--6, AIFS-SUBS matches the operational Integrated Forecasting System (IFS) in probabilistic skill while reducing systematic biases. For the convective (OLR) component of the Madden--Julian Oscillation (MJO), AIFS-SUBS extends skilful forecasts (correlation > 0.5) by eight days relative to the IFS, while matching or exceeding the IFS for the full multivariate RMM index. AIFS-SUBS also reproduces the observed MJO modulation of tropical cyclone activity comparably. Stratospheric skill is particularly strong with AIFS-SUBS reproducing sudden stratospheric warming (SSW) frequency and surface impact. In the AI Weather Quest, AIFS-SUBS-ERA5 attains a variable-averaged ranked probability skill score slightly ahead of the IFS at weeks 3 and 4. At inference, AIFS-SUBS uses about 200 times less energy than the IFS, opening the door to much larger real-time ensembles. AIFS-SUBS is ECMWF's first machine-learning model targeted at sub-seasonal time-scales.

physics.ao-ph↗

arXiv2026

AIFS-DOP: End-to-End Medium-Range Weather Prediction from Observations Alone with Machine Learning

We introduce the Artificial Intelligence Forecasting System for Direct Observation Prediction (AIFS-DOP). AIFS-DOP is trained on a 40-year harmonized dataset of gridded observations, without using numerical weather prediction (NWP) reanalysis or model data. The resulting model is competitive with ECMWF's Integrated Forecasting System (IFS) when scored on a one year period of forecasts across 2021/2022. This progress on Direct Observation Prediction represents the first time that a data-driven model, trained solely on observations, is competitive with the IFS at medium ranges for several key upper-air and surface headline scores, when verified against observation data.

physics.ao-ph↗

arXiv2026

Representing the Surface Ocean in ECMWF's data-driven forecasting system AIFS

Machine-learning (ML) models, such as the AIFS at the ECMWF, have revolutionised weather forecasting in recent years. We present an extension of the AIFS that jointly models the atmosphere and surface ocean, including ocean waves and sea ice. The primary objective of this extension is to enhance machine-learning medium-range forecasting and enable new use cases by expanding the weather state to better capture coupled surface processes. Our approach departs from traditional numerical models by not having two separate models for the atmosphere and marine components. The joint model instead learns correlations across the entire atmosphere-ocean interface in a component-agnostic way, and can exploit the expressive capacity of ML architectures to learn cross-component relationships directly from the data. We leverage tailored and targeted datasets and solve model design challenges such as missing values over land, multi-scale temporal dynamics, and physical realism of forecast fields and demonstrate the utility of loss scaling in guiding the learning process. We evaluate how representing the surface ocean affects medium-range weather forecasts. We also assess the model's ability to predict surface-ocean fields, including wave swell and tropical-cyclone cold wakes. For nearly all evaluated marine variables, we observe an improvement of approximately one day in forecast skill at medium-range lead times compared to physics-based models. Furthermore, we demonstrate that the model is robust to idealised initial conditions outside the training distribution and responds to them in a physically consistent way. Overall, our findings suggest that the joint AIFS modelling approach offers significant potential for combined atmosphere-ocean forecasting. Our work provides a solid foundation for future development of data-driven coupled Earth system models with greater flexibility and physical fidelity.

physics.ao-ph↗

arXiv2026

Downscaling weather forecasts from Low- to High-Resolution with Diffusion Models

We introduce a probabilistic diffusion-based method for global atmospheric downscaling implemented within the Anemoi framework. The approach transforms low-resolution ensemble forecasts into high-resolution ensembles by learning the conditional distribution of finer-scale residuals, defined as the difference between the high-resolution fields and the interpolated low-resolution inputs. The system is trained on reforecast pairs from ECMWF IFS, using coarse fields at 100 km to reconstruct fine-scale variability at 30 km resolution. The bulk of the training focuses on recovering small-scale structures, while fine-tuning in high-noise regimes enables the generation of extremes. Evaluation against the medium-range IFS ensemble target shows that the model increases probabilistic skill (FCRPS) for surface variables, reproduces target power spectra at small scales, captures physically consistent multivariate relationships such as wind-pressure coupling, and generates extreme values consistent with those of the target ensemble in tropical cyclones.

physics.ao-ph↗

arXiv2026

AIFS-COMPO: A Global Data-Driven Atmospheric Composition Forecasting System

We introduce AIFS-COMPO, a skilful medium-range data-driven global forecasting system for aerosols and reactive gases. Building on the ECMWF Artificial Intelligence Forecast System (AIFS), AIFS-COMPO employs a transformer-based encoder-processor-decoder architecture to jointly model meteorological and atmospheric composition variables. The model is trained on Copernicus Atmosphere Monitoring Service (CAMS) reanalysis, analysis, and forecast data to learn the coupled dynamics of weather, emissions, transport, and atmospheric chemistry. We evaluate AIFS-COMPO against a range of atmospheric composition observations and compare its performance with the operational CAMS global forecasting system IFS-COMPO. The results show that AIFS-COMPO achieves comparable or improved forecast skill for several key species while requiring only a fraction of the computational resources. Furthermore, the efficiency of the approach enables forecasts beyond the current operational horizon, demonstrating the potential of AI-based systems for fast and accurate global atmospheric composition prediction.

physics.ao-ph↗

arXiv2026

AIFL: A Global Daily Streamflow Forecasting Model Using a Deterministic LSTM Pre-trained on ERA5-Land and Fine-tuned on IFS

Reliable global streamflow forecasting is essential for flood preparedness and water resource management, yet data-driven models often suffer from a performance gap when transitioning from historical reanalysis to operational forecast products. This paper introduces AIFL (Artificial Intelligence for Floods), a deterministic LSTM-based model designed for global daily streamflow forecasting. Trained on 18,588 basins curated from the Caravan dataset, AIFL utilises a two-stage transfer-learning strategy to bridge the reanalysis-to-forecast domain shift. The model is first pre-trained on 40 years of ERA5-Land reanalysis (1980-2019) to capture robust hydrological processes, then fine-tuned on operational Integrated Forecasting System (IFS) forecasts (2016-2019) to adapt to the specific error structures and biases of operational numerical weather prediction. Ablation experiments confirm that this two-stage approach outperforms both a naive IFS-only baseline and a mixed-forcing single-stage alternative. To our knowledge, this is the first global model trained end-to-end within the Caravan ecosystem. On an independent temporal test set (2021-2024), AIFL achieves high predictive skill with a median modified Kling-Gupta Efficiency (KGE') of 0.66 and a median Nash-Sutcliffe Efficiency (NSE) of 0.53. Benchmarking results show that AIFL achieves comparable accuracy to current state-of-the-art global systems. The model provides a streamlined and operationally robust baseline for the global hydrological community.

cs.LG↗

arXiv2025

Improved rainfall forecasts in daily use over East Africa

Ensemble forecasting has proven to be a vital tool for predicting extreme, life-threatening or only partially predictable weather events. As well as providing probabilistic products, individual ensemble members provide context and realisations of possible extreme weather. However, many National Meteorological Services in East Africa do not have the computing resources to enable them to run their local area models in ensemble mode over the full period of the two-week medium range. In this paper we test the performance of a forecast system, comprising the global ECMWF ensemble forecast, post-processed using cGAN, a neural network model, and forecasts calibrated using IDR, a recent statistical method, against the probabilistic climatology. cGAN provides comparable levels of probabilistic skill as IDR applied to the ECMWF ensemble or IDR applied to the machine-learning models FuXi and GraphCast. These methods improve the raw ECMWF ensemble forecast substantially, which is itself an improvement on deterministic forecasts. The ability of cGAN to produce individual realisations of future rainfall was important when assessed in an operational context by the National Meteorological and Hydrological Services of Kenya and Ethiopia. Moreover, in common with IDR, cGAN is cheap to train/run and requires no additional post-processing. It is run on laptops to generate many thousands of ensemble members, making it suitable for Meteorological Services with limited computational facilities.

physics.ao-ph↗

arXiv2025

On Global Applicability and Location Transferability of Generative Deep Learning Models for Precipitation Downscaling

Deep learning offers promising capabilities for the statistical downscaling of climate and weather forecasts, with generative approaches showing particular success in capturing fine-scale precipitation patterns. However, most existing models are region-specific, and their ability to generalize to unseen geographic areas remains largely unexplored. In this study, we evaluate the generalization performance of generative downscaling models across diverse regions. Using a global framework, we employ ERA5 reanalysis data as predictors and IMERG precipitation estimates at $0.1^\circ$ resolution as targets. A hierarchical location-based data split enables a systematic assessment of model performance across 15 regions around the world.

cs.LG↗

arXiv2025

High-Resolution Probabilistic Data-Driven Weather Modeling with a Stretched-Grid

We present a probabilistic data-driven weather model providing ensembles of high spatial resolution realizations of 87 variables at arbitrary ensemble size and forecast length. The model uses a global stretched grid, dedicating 2.5 km resolution to our Nordic region of interest and 31 km resolution elsewhere, with 6-hour temporal resolution. Unique ensemble members are generated by a stochastic model architecture, and we train it using a loss function based on the Continuous Ranked Probability Score (CRPS) evaluated in grid-point and spectral space. The spectral loss component is shown to be necessary to create fields that are spatially coherent, which is not the case when training with mean-squared error loss, nor CRPS in grid-point space only. We evaluate the forecasts against observations from surface weather stations and compare them to high-resolution operational numerical weather prediction forecasts from the MetCoOp Ensemble Prediction System (MEPS). The model shows lower CRPS than MEPS for 2 m temperature and mean sea-level pressure, with average improvements of 13\% and 10\%, respectively, while differences for wind speed and precipitation are smaller. For Storm Dave, the model captures the location and structure of strong-wind systems, but underestimates the peak winds.

physics.ao-ph↗

arXiv2025

An update to ECMWF's machine-learned weather forecast model AIFS

We present an update to ECMWF's machine-learned weather forecasting model AIFS Single with several key improvements. The model now incorporates physical consistency constraints through bounding layers, an updated training schedule, and an expanded set of variables. The physical constraints substantially improve precipitation forecasts and the new variables show a high level of skill. Upper-air headline scores also show improvement over the previous AIFS version. The AIFS has been fully operational at ECMWF since the 25th of February 2025.

physics.ao-ph↗

arXiv2025

RainShift: A Benchmark for Precipitation Downscaling Across Geographies

Earth System Models (ESM) are our main tool for projecting the impacts of climate change. However, running these models at sufficient resolution for local-scale risk-assessments is not computationally feasible. Deep learning-based super-resolution models offer a promising solution to downscale ESM outputs to higher resolutions by learning from data. Yet, due to regional variations in climatic processes, these models typically require retraining for each geographical area-demanding high-resolution observational data, which is unevenly available across the globe. This highlights the need to assess how well these models generalize across geographic regions. To address this, we introduce RainShift, a dataset and benchmark for evaluating downscaling under geographic distribution shifts. We evaluate state-of-the-art downscaling approaches including GANs and diffusion models in generalizing across data gaps between the Global North and Global South. Our findings reveal substantial performance drops in out-of-distribution regions, depending on model and geographic area. While expanding the training domain generally improves generalization, it is insufficient to overcome shifts between geographically distinct regions. We show that addressing these shifts through, for example, data alignment can improve spatial generalization. Our work advances the global applicability of downscaling methods and represents a step toward reducing inequities in access to high-resolution climate information.

cs.CV↗

arXiv2024

GraphDOP: Towards skilful data-driven medium-range weather forecasts learnt and initialised directly from observations

We introduce GraphDOP, a new data-driven, end-to-end forecast system developed at the European Centre for Medium-Range Weather Forecasts (ECMWF) that is trained and initialised exclusively from Earth System observations, with no physics-based (re)analysis inputs or feedbacks. GraphDOP learns the correlations between observed quantities - such as brightness temperatures from polar orbiters and geostationary satellites - and geophysical quantities of interest (that are measured by conventional observations), to form a coherent latent representation of Earth System state dynamics and physical processes, and is capable of producing skilful predictions of relevant weather parameters up to five days into the future.

physics.ao-ph↗

arXiv2024

AIFS-CRPS: Ensemble forecasting using a model trained with a loss function based on the Continuous Ranked Probability Score

Over the last three decades, ensemble forecasts have become an integral part of forecasting the weather. They provide users with more complete information than single forecasts as they permit to estimate the probability of weather events by representing the sources of uncertainties and accounting for the day-to-day variability of error growth in the atmosphere. This paper presents a novel approach to obtain a weather forecast model for ensemble forecasting with machine-learning. AIFS-CRPS is a variant of the Artificial Intelligence Forecasting System (AIFS) developed at ECMWF. Its loss function is based on a proper score, the Continuous Ranked Probability Score (CRPS). For the loss, the almost fair CRPS is introduced because it approximately removes the bias in the score due to finite ensemble size yet avoids a degeneracy of the fair CRPS. The trained model is stochastic and can generate as many exchangeable members as desired and computationally feasible in inference. For medium-range forecasts AIFS-CRPS outperforms the physics-based Integrated Forecasting System (IFS) ensemble for the majority of variables and lead times. For subseasonal forecasts, AIFS-CRPS outperforms the IFS ensemble before calibration and is competitive with the IFS ensemble when forecasts are evaluated as anomalies to remove the influence of model biases.

physics.ao-ph↗

arXiv2024

Hydra-LSTM: A semi-shared Machine Learning architecture for prediction across Watersheds

Long Short Term Memory networks (LSTMs) are used to build single models that predict river discharge across many catchments. These models offer greater accuracy than models trained on each catchment independently if using the same data. However, the same data is rarely available for all catchments. This prevents the use of variables available only in some catchments, such as historic river discharge or upstream discharge. The only existing method that allows for optional variables requires all variables to be considered in the initial training of the model, limiting its transferability to new catchments. To address this limitation, we develop the Hydra-LSTM. The Hydra-LSTM processes variables used across all catchments and variables used in only some catchments separately to allow general training and use of catchment-specific data in individual catchments. The bulk of the model can be shared across catchments, maintaining the benefits of multi-catchment models to generalise, while also benefitting from the advantages of using bespoke data. We apply this methodology to 1 day-ahead river discharge prediction in the Western US, as next-day river discharge prediction is the first step towards prediction across longer time scales. We obtain state-of-the-art performance, generating more accurate median and quantile predictions than Multi-Catchment and Single-Catchment LSTMs while allowing local forecasters to easily introduce and remove variables from their prediction set. We test the ability of the Hydra-LSTM to incorporate catchment-specific data by introducing historical river discharge as a catchment-specific input, outperforming state-of-the-art models without needing to train an entirely new model.

cs.LG↗

arXiv2024

Robustness of AI-based weather forecasts in a changing climate

Data-driven machine learning models for weather forecasting have made transformational progress in the last 1-2 years, with state-of-the-art ones now outperforming the best physics-based models for a wide range of skill scores. Given the strong links between weather and climate modelling, this raises the question whether machine learning models could also revolutionize climate science, for example by informing mitigation and adaptation to climate change or to generate larger ensembles for more robust uncertainty estimates. Here, we show that current state-of-the-art machine learning models trained for weather forecasting in present-day climate produce skillful forecasts across different climate states corresponding to pre-industrial, present-day, and future 2.9K warmer climates. This indicates that the dynamics shaping the weather on short timescales may not differ fundamentally in a changing climate. It also demonstrates out-of-distribution generalization capabilities of the machine learning models that are a critical prerequisite for climate applications. Nonetheless, two of the models show a global-mean cold bias in the forecasts for the future warmer climate state, i.e. they drift towards the colder present-day climate they have been trained for. A similar result is obtained for the pre-industrial case where two out of three models show a warming. We discuss possible remedies for these biases and analyze their spatial distribution, revealing complex warming and cooling patterns that are partly related to missing ocean-sea ice and land surface information in the training data. Despite these current limitations, our results suggest that data-driven machine learning models will provide powerful tools for climate science and transform established approaches by complementing conventional physics-based models.

physics.ao-ph↗