Error in ERA5 2m Temperature identified using GraphCast

Abstract

Reanalyses such as ERA5 have long been foundational for weather and climate science. They have also found a new use case, as training and verification data for machine-learnt weather prediction (MLWP) models. Here we compare short-lead time (6h) forecasts from the MLWP model GraphCast against ERA5. In doing so, we identify a recurrent, spatially coherent error in 2m Temperature centred on the Ethiopian Highlands, that occurs predominantly at 0600 UTC. We show that these error events are not an error in the forecast from GraphCast, but are in fact an error in ERA5, and are also present in the ECMWF operational analysis. They arise from the 2D optimal interpolation procedure, when surface reports are assimilated that are temporally displaced compared to the background forecast. This produces spuriously warm analysis increments over Ethiopia on approximately 7\% of dates at 0600 UTC across the reanalysis record. The spread from the ensemble of data assimilation partially flags these cases but is underdispersive. We assess the impact on GraphCast, which was trained on ERA5. While GraphCast can largely ignore these unphysical error events, a small systematic degradation in forecast skill over the region is observed. We discuss implications for using reanalysis as truth in machine learning training and verification, and recommend simple changes to reduce such artefacts in future analyses.