Forecast error diagnostics in neural weather models

Abstract

Deep-learning (DL) weather prediction models offer some notable advantages over traditional physics-based models, including auto-differentiability and low computational cost, enabling detailed diagnostics of forecast errors. Using our convolutional encoder-decoder model, ConvCastNet, we systematically relax selected subdomains of the forecast fields towards "true" weather states (ERA5 reanalyses) and monitor the forecast skill gain in other regions. Our results show that a medium-range mid-latitude forecast improves substantially when the stratosphere and boundary layer are relaxed, while relaxation of the tropical atmosphere has a negligible effect. This underscores the need for a more accurate representation of the stratosphere and the planetary boundary layer to improve medium-range weather predictability. Additionally, we investigate the relationship between the forecast error sensitivity to initial conditions and relaxation experiments. By utilising auto-differentiability, we identify overlapping regions of large error sensitivity and strong forecast skill improvement from relaxation. Average mid-latitude error sensitivity to initial conditions shows negligible influence from the tropics, corroborating the results of the tropical relaxation experiments. The error sensitivity shows a physically consistent influence of upstream weather dynamics and sea surface temperatures on forecast accuracy. The latter also highlights the importance of accurately representing the atmosphere--ocean coupling in numerical weather prediction models. This combined approach could provide valuable heuristics for diagnosing neural model errors and guiding targeted model improvements.