arXiv subjects

Shiheng Duan

Publications and source records attributed to Shiheng Duan.

5 recordsLinked to original sources

arXiv2026

A PMP-inspired Evaluation Framework for Assessing Deep-Learning Earth System Models

In recent years, Deep-Learning Earth System Models (DL-ESMs) have emerged as promising, computationally efficient complements to traditional Earth system models. Here, we present an evaluation framework for testing DL-ESMs from an Earth system model-development perspective using standardized diagnostics from the PCMDI Metrics Package (PMP). This framework allows DL-ESMs, including Ai2's ACE2 and Google's NeuralGCM, to be assessed with metrics that quantify their ability to reproduce climatology, major modes of variability, monsoons, and precipitation variability relative to observational reference datasets and CMIP-class benchmarks. By evaluating DL-ESMs with tools commonly used for traditional models, we extend their assessment beyond short-range forecast skill and toward longer Earth System-relevant applications. The results identify encouraging strengths in several large-scale fields and modes of variability, while also highlighting persistent challenges in precipitation, tropical variability, and long-run stability for some model versions. This evaluation is a critical step toward building trust in DL-ESMs, guiding future model development, and clarifying their fit-for-purpose for Earth system science applications.

physics.ao-ph↗

arXiv2025

Testing NeuralGCM's capability to simulate future heatwaves based on the 2021 Pacific Northwest heatwave event

AI-based weather and climate models are emerging as accurate and computationally efficient tools. Beyond weather forecasting, they also show promise to accelerate storyline analyses. We evaluate NeuralGCM's ability to simulate an extreme heatwave against the Energy Exascale Earth System Model (E3SM), a physics-based climate model. NeuralGCM accurately replicates the targeted event, and generates stable and realistic mid-century projections. However, due to the absence of land feedbacks, NeuralGCM underestimates the projected warming amplitude compared to physics-based model references.

physics.ao-ph↗

arXiv2025

Recommendations for Comprehensive and Independent Evaluation of Machine Learning-Based Earth System Models

Machine learning (ML) is a revolutionary technology with demonstrable applications across multiple disciplines. Within the Earth science community, ML has been most visible for weather forecasting, producing forecasts that rival modern physics-based models. Given the importance of deepening our understanding and improving predictions of the Earth system on all time scales, efforts are now underway to develop forecasting models into Earth-system models (ESMs), capable of representing all components of the coupled Earth system (or their aggregated behavior) and their response to external changes. Modeling the Earth system is a much more difficult problem than weather forecasting, not least because the model must represent the alternate (e.g., future) coupled states of the system for which there are no historical observations. Given that the physical principles that enable predictions about the response of the Earth system are often not explicitly coded in these ML-based models, demonstrating the credibility of ML-based ESMs thus requires us to build evidence of their consistency with the physical system. To this end, this paper puts forward five recommendations to enhance comprehensive, standardized, and independent evaluation of ML-based ESMs to strengthen their credibility and promote their wider use.

cs.LG↗

arXiv2022

AutoML-based Almond Yield Prediction and Projection in California

Almonds are one of the most lucrative products of California, but are also among the most sensitive to climate change. In order to better understand the relationship between climatic factors and almond yield, an automated machine learning framework is used to build a collection of machine learning models. The prediction skill is assessed using historical records. Future projections are derived using 17 downscaled climate outputs. The ensemble mean projection displays almond yield changes under two different climate scenarios, along with two technology development scenarios, where the role of technology development is highlighted. The mean projections and distributions provide insightful results to stakeholders and can be utilized by policymakers for climate adaptation.

cs.LG↗

arXiv2022

AutoML-Based Drought Forecast with Meteorological Variables

A precise forecast for droughts is of considerable value to scientific research, agriculture, and water resource management. With emerging developments of data-driven approaches for hydro-climate modeling, this paper investigates an AutoML-based framework to forecast droughts in the U.S. Compared with commonly-used temporal deep learning models, the AutoML model can achieve comparable performance with less training data and time. As deep learning models are becoming popular for Earth system modeling, this paper aims to bring more efforts to AutoML-based methods, and the use of them as benchmark baselines for more complex deep learning models.

cs.LG↗