arXiv subjects

Massimo Bonavita

Publications and source records attributed to Massimo Bonavita.

9 recordsLinked to original sources

arXiv2026

WP-MIP: An Artificial Intelligence, Hybrid, and Physically Based Model Intercomparison Project for Weather Prediction

Rapid progress in the field of machine-learning for weather prediction has led to the emergence of algorithms whose forecasting skill can exceed that of traditional physically based models. This development represents an opportunity to improve the quality of forecasting services provided by operational centers, particularly given the speed at which machine-learning based models generate predictions. Despite the clear promise of these systems, questions remain about the ability of the current generation of machine-learning models to generate physically consistent predictions of the full suite of required forecast fields under all conditions. Answering these questions will require careful comparisons between the well-understood physically based models, current state-of-the-art machine-learning models, and the hybrid models that combine elements of these two archetypes. The Weather Prediction Model Intercomparison Project (WP-MIP) is a World Meteorological Organization-supported initiative whose initial goal is to create a centralized database of physically based, machine-learning, and hybrid model forecasts to enable a distributed assessment and evaluation effort. The first instance of WP-MIP focuses on global deterministic predictions using both center-specific and common initializations to facilitate sensitivity studies. Forecasts contributed by institutions across six continents will be used to develop AI-ready verification techniques that highlight the strengths and weaknesses of each class of prediction system, with the goal of establishing best-practice guidance to model developers and national weather centers. The broad engagement of the operational and forecast-evaluation communities in WP-MIP will ensure that the project results are highly relevant to the development and deployment of next-generation weather prediction systems.

physics.ao-ph↗

arXiv2026

Error in ERA5 2m Temperature identified using GraphCast

Reanalyses such as ERA5 have long been foundational for weather and climate science. They have also found a new use case, as training and verification data for machine-learnt weather prediction (MLWP) models. Here we compare short-lead time (6h) forecasts from the MLWP model GraphCast against ERA5. In doing so, we identify a recurrent, spatially coherent error in 2m Temperature centred on the Ethiopian Highlands, that occurs predominantly at 0600 UTC. We show that these error events are not an error in the forecast from GraphCast, but are in fact an error in ERA5, and are also present in the ECMWF operational analysis. They arise from the 2D optimal interpolation procedure, when surface reports are assimilated that are temporally displaced compared to the background forecast. This produces spuriously warm analysis increments over Ethiopia on approximately 7\% of dates at 0600 UTC across the reanalysis record. The spread from the ensemble of data assimilation partially flags these cases but is underdispersive. We assess the impact on GraphCast, which was trained on ERA5. While GraphCast can largely ignore these unphysical error events, a small systematic degradation in forecast skill over the region is observed. We discuss implications for using reanalysis as truth in machine learning training and verification, and recommend simple changes to reduce such artefacts in future analyses.

physics.ao-ph↗

arXiv2025

Development of an offline and online hybrid model for the Integrated Forecasting System

In recent years, there has been significant progress in the development of fully data-driven global numerical weather prediction models. These machine learning weather prediction models have their strength, notably accuracy and low computational requirements, but also their weakness: they struggle to represent fundamental dynamical balances, and they are far from being suitable for data assimilation experiments. Hybrid modelling emerges as a promising approach to address these limitations. Hybrid models integrate a physics-based core component with a statistical component, typically a neural network, to enhance prediction capabilities. In this article, we propose to develop a model error correction for the operational Integrated Forecasting System (IFS) of the European Centre for Medium-Range Weather Forecasts using a neural network. The neural network is initially pre-trained offline using a large dataset of operational analyses and analysis increments. Subsequently, the trained network is integrated into the IFS within the Object-Oriented Prediction System (OOPS) so as to be used in data assimilation and forecast experiments. It is then further trained online using a recently developed variant of weak-constraint 4D-Var. The results show that the pre-trained neural network already provides a reliable model error correction, which translates into reduced forecast errors in many conditions and that the online training further improves the accuracy of the hybrid model in many conditions.

stat.ML↗

arXiv2023

On some limitations of data-driven weather forecasting models

As in many other areas of engineering and applied science, Machine Learning (ML) is having a profound impact in the domain of Weather and Climate Prediction. A very recent development in this area has been the emergence of fully data-driven ML prediction models which routinely claim superior performance to that of traditional physics-based models. In this work, we examine some aspects of the forecasts produced by an exemplar of the current generation of ML models, Pangu-Weather, with a focus on the fidelity and physical consistency of those forecasts and how these characteristics relate to perceived forecast performance. The main conclusion is that Pangu-Weather forecasts, and possibly those of similar ML models, do not have the fidelity and physical consistency of physics-based models and their advantage in accuracy on traditional deterministic metrics of forecast skill can be at least partly attributed to these peculiarities. Balancing forecast skill and physical consistency of ML-driven predictions will be an important consideration for future ML models. However, and similarly to other modern post-processing technologies, the current ML models appear to be already able to add value to standard NWP output for specific forecast applications and combined with their extremely low computational cost during deployment, are set to provide an additional, useful source of forecast information. .

stat.ML↗

arXiv2022

Online model error correction with neural networks in the incremental 4D-Var framework

Recent studies have demonstrated that it is possible to combine machine learning with data assimilation to reconstruct the dynamics of a physical model partially and imperfectly observed. Data assimilation is used to estimate the system state from the observations, while machine learning computes a surrogate model of the dynamical system based on those estimated states. The surrogate model can be defined as an hybrid combination where a physical model based on prior knowledge is enhanced with a statistical model estimated by a neural network. The training of the neural network is typically done offline, once a large enough dataset of model state estimates is available. By contrast, with online approaches the surrogate model is improved each time a new system state estimate is computed. Online approaches naturally fit the sequential framework encountered in geosciences where new observations become available with time. In a recent methodology paper, we have developed a new weak-constraint 4D-Var formulation which can be used to train a neural network for online model error correction. In the present article, we develop a simplified version of that method, in the incremental 4D-Var framework adopted by most operational weather centres. The simplified method is implemented in the ECMWF Object-Oriented Prediction System, with the help of a newly developed Fortran neural network library, and tested with a two-layer two-dimensional quasi geostrophic model. The results confirm that online learning is effective and yields a more accurate model error correction than offline learning. Finally, the simplified method is compatible with future applications to state-of-the-art models such as the ECMWF Integrated Forecasting System.

stat.ML↗

arXiv2022

Estimating Model Error Covariances with Artificial Neural Networks

Methods to deal with systematic model errors are an increasingly important component of modern data assimilation systems and their effectiveness has increased in recent years thanks to advances in methodology and the quality and density of the global observing system. The weak constraint 4D-Var assimilation algorithm employed at ECMWF is well suited to the estimation and correction of model errors as they are explicitly accounted for in the cost function. This has led to significant improvements in recent years to the accuracy of stratospheric analyses. One question that remains open is about the estimation of the model error covariance matrix to use in weak constraint 4D-Var. Encouraged by the promising results we have obtained in the recent past through the use of Artificial Neural Networks (ANNs) to estimate slowly-varying model errors in the ECMWF assimilation cycle, we explore in this work the use of ANNs to sample the model error distribution and provide an alternative way to construct a model error covariance matrix. Results from the application of the new model error covariance in cycling assimilation experiments are described and implications for further developments of the ECMWF data assimilation system are discussed.

stat.AP↗

arXiv2021

A comparison of combined data assimilation and machine learning methods for offline and online model error correction

Recent studies have shown that it is possible to combine machine learning methods with data assimilation to reconstruct a dynamical system using only sparse and noisy observations of that system. The same approach can be used to correct the error of a knowledge-based model. The resulting surrogate model is hybrid, with a statistical part supplementing a physical part. In practice, the correction can be added as an integrated term (i.e. in the model resolvent) or directly inside the tendencies of the physical model. The resolvent correction is easy to implement. The tendency correction is more technical, in particular it requires the adjoint of the physical model, but also more flexible. We use the two-scale Lorenz model to compare the two methods. The accuracy in long-range forecast experiments is somewhat similar between the surrogate models using the resolvent correction and the tendency correction. By contrast, the surrogate models using the tendency correction significantly outperform the surrogate models using the resolvent correction in data assimilation experiments. Finally, we show that the tendency correction opens the possibility to make online model error correction, i.e. improving the model progressively as new observations become available. The resulting algorithm can be seen as a new formulation of weak-constraint 4D-Var. We compare online and offline learning using the same framework with the two-scale Lorenz system, and show that with online learning, it is possible to extract all the information from sparse and noisy observations.

stat.ML↗

arXiv2021

Exploring the structure of time-correlated model errors in the ECMWF Data Assimilation System

Model errors are increasingly seen as a fundamental performance limiter in both Numerical Weather Prediction and Climate Prediction simulations run with state of the art Earth system digital twins.This has motivated recent efforts aimed at estimating and correcting the systematic, predictable components of model error in a consistent data assimilation framework. While encouraging results have been obtained with a careful examination of the spatial aspects of the model error estimates, less attention has been devoted to the time correlation aspects of model errors and their impact on the assimilation cycle. In this work we employ a Lagged Analysis Increment Covariance (LAIG) diagnostic to gain insight in the temporal evolution of systematic model errors in the ECMWF operational data assimilation system, evaluate the effectiveness of the current weak constraint 4DVar algorithm in reducing these types of errors and, based on these findings,start exploring new ideas for the development of model error estimation and correction strategies in data assimilation.

stat.AP↗

arXiv2021

Using machine learning to correct model error in data assimilation and forecast applications

The idea of using machine learning (ML) methods to reconstruct the dynamics of a system is the topic of recent studies in the geosciences, in which the key output is a surrogate model meant to emulate the dynamical model. In order to treat sparse and noisy observations in a rigorous way, ML can be combined to data assimilation (DA). This yields a class of iterative methods in which, at each iteration a DA step assimilates the observations, and alternates with a ML step to learn the underlying dynamics of the DA analysis. In this article, we propose to use this method to correct the error of an existent, knowledge-based model. In practice, the resulting surrogate model is an hybrid model between the original (knowledge-based) model and the ML model. We demonstrate numerically the feasibility of the method using a two-layer, two-dimensional quasi-geostrophic channel model. Model error is introduced by the means of perturbed parameters. The DA step is performed using the strong-constraint 4D-Var algorithm, while the ML step is performed using deep learning tools. The ML models are able to learn a substantial part of the model error and the resulting hybrid surrogate models produce better short- to mid-range forecasts. Furthermore, using the hybrid surrogate models for DA yields a significantly better analysis than using the original model.

stat.ML↗