arXiv subjects

Baoxiang Pan

Publications and source records attributed to Baoxiang Pan.

15 recordsLinked to original sources

arXiv2026

Cast3: Translating numerical weather prediction principles into data-driven forecasting

Data-driven weather models have made rapid advances in recent years, reaching and in some metrics surpassing the large-scale forecast skill of operational numerical weather prediction. This progress, however, has been built almost entirely on the reanalysis data that NWP produced, while the methodological knowledge that the NWP community distilled over decades of multi-scale atmospheric modelling remains largely unused. Here we present Cast3, a generative forecasting framework that systematically absorbs NWP meta-knowledge to close this gap. Cast3 operates on variable-resolution cubed-sphere grids for scale-aware representation and constructs structurally diverse super-ensembles that sample the complementary biases of different grid discretizations, delivering state-of-the-art ensemble prediction. It further introduces generative nudging, a posterior-sampling strategy that distils the collective information of the full ensemble into a single forecast possessing both the large-scale accuracy of the ensemble mean and the mesoscale realism of a high-resolution member. Evaluated across synoptic-scale skill, spectral fidelity, station-level surface verification, and tropical cyclone prediction, Cast3 outperforms established deterministic and generative baselines across various dimensions. More broadly, these results demonstrate that the design principles embedded in computational atmospheric science offer a rich and largely untapped foundation for the next generation of data-driven Earth system modelling.

physics.ao-ph↗

arXiv2025

Efficient Generative AI Boosts Probabilistic Forecasting of Sudden Stratospheric Warmings

Sudden Stratospheric Warmings (SSWs) are key sources of subseasonal predictability and major drivers of extreme weather in winter. Accurate and efficient probabilistic forecasting of these events remains a persistent challenge for Numerical Weather Prediction (NWP) systems due to computational bottlenecks and limitations in physical representation. While data-driven forecasting is rapidly evolving, its application to the complex, three-dimensional dynamics of SSWs remains underexplored. Here, we bridge this gap by developing a Flow Matching-based generative AI model (FM-Cast) for efficient and skillful probabilistic forecasting of the spatiotemporal evolution of stratospheric circulation in winter. Evaluated across 18 major SSW events (1998-2024), FM-Cast successfully forecasts the onset, intensity, and 3D morphology of the polar vortex up to 15 days in advance for most cases. Notably, it achieves long-range probabilistic forecast skill comparable to or exceeding leading operational NWP systems (ECMWF and CMA) while generating a 30-day forecast with 50-member ensemble, in just two minutes on a consumer GPU. Furthermore, using idealized "perfect troposphere" experiments, we uncover distinct predictability regimes: events driven by continuous wave forcing versus those governed by an initial trigger and subsequent stratospheric dynamical memory. This work establishes a computationally efficient paradigm for probabilistic stratospheric forecasting that simultaneously deepens our physical understanding of atmosphere-climate dynamics.

cs.LG↗

arXiv2025

Lighting the Night with Generative Artificial Intelligence

The visible light reflectance data from geostationary satellites is crucial for meteorological observations and plays an important role in weather monitoring and forecasting. However, due to the lack of visible light at night, it is impossible to conduct continuous all-day weather observations using visible light reflectance data. This study pioneers the use of generative diffusion models to address this limitation. Based on the multi-band thermal infrared brightness temperature data from the Advanced Geostationary Radiation Imager (AGRI) onboard the Fengyun-4B (FY4B) geostationary satellite, we developed a high-precision visible light reflectance generative model, called Reflectance Diffusion (RefDiff), which enables 0.47~\mu\mathrm{m}, 0.65~\mu\mathrm{m}, and 0.825~\mu\mathrm{m} bands visible light reflectance generation at night. Compared to the classical models, RefDiff not only significantly improves accuracy through ensemble averaging but also provides uncertainty estimation. Specifically, the SSIM index of RefDiff can reach 0.90, with particularly significant improvements in areas with complex cloud structures and thick clouds. The model's nighttime generation capability was validated using VIIRS nighttime product, demonstrating comparable performance to its daytime counterpart. In summary, this research has made substantial progress in the ability to generate visible light reflectance at night, with the potential to expand the application of nighttime visible light data.

cs.CV↗

arXiv2025

Fusion of multi-source precipitation records via coordinate-based generative model

Precipitation remains one of the most challenging climate variables to observe and predict accurately. Existing datasets face intricate trade-offs: gauge observations are relatively trustworthy but sparse, satellites provide global coverage with retrieval uncertainties, and numerical models offer physical consistency but are biased and computationally intensive. Here we introduce PRIMER (Precipitation Record Infinite MERging), a deep generative framework that fuses these complementary sources to produce accurate, high-resolution, full-coverage precipitation estimates. PRIMER employs a coordinate-based diffusion model that learns from arbitrary spatial locations and associated precipitation values, enabling seamless integration of gridded data and irregular gauge observations. Through two-stage training--first learning large-scale patterns, then refining with accurate gauge measurements--PRIMER captures both large-scale climatology and local precision. Once trained, it can downscale forecasts, interpolate sparse observations, and correct systematic biases within a principled Bayesian framework. Using gauge observations as ground truth, PRIMER effectively corrects biases in existing datasets, yielding statistically significant error reductions at most stations and furthermore enhancing the spatial coherence of precipitation fields. Crucially, it generalizes without retraining, correcting biases in operational forecasts it has never seen. This demonstrates how generative AI can transform Earth system science by combining imperfect data, providing a scalable solution for global precipitation monitoring and prediction.

physics.ao-ph↗

arXiv2025

Supporting renewable energy planning and operation with data-driven high-resolution ensemble weather forecast

The planning and operation of renewable energy, especially wind power, depend crucially on accurate, timely, and high-resolution weather information. Coarse-grid global numerical weather forecasts are typically downscaled to meet these requirements, introducing challenges of scale inconsistency, process representation error, computation cost, and entanglement of distinct uncertainty sources from chaoticity, model bias, and large-scale forcing. We address these challenges by learning the climatological distribution of a target wind farm using its high-resolution numerical weather simulations. An optimal combination of this learned high-resolution climatological prior with coarse-grid large scale forecasts yields highly accurate, fine-grained, full-variable, large ensemble of weather pattern forecasts. Using observed meteorological records and wind turbine power outputs as references, the proposed methodology verifies advantageously compared to existing numerical/statistical forecasting-downscaling pipelines, regarding either deterministic/probabilistic skills or economic gains. Moreover, a 100-member, 10-day forecast with spatial resolution of 1 km and output frequency of 15 min takes < 1 hour on a moderate-end GPU, as contrast to $\mathcal{O}(10^3)$ CPU hours for conventional numerical simulation. By drastically reducing computational costs while maintaining accuracy, our method paves the way for more efficient and reliable renewable energy planning and operation.

cs.LG↗

arXiv2025

Generative assimilation and prediction for weather and climate

Machine learning models have shown great success in predicting weather up to two weeks ahead, outperforming process-based benchmarks. However, existing approaches mostly focus on the prediction task, and do not incorporate the necessary data assimilation. Moreover, these models suffer from error accumulation in long roll-outs, limiting their applicability to seasonal predictions or climate projections. Here, we introduce Generative Assimilation and Prediction (GAP), a unified deep generative framework for assimilation and prediction of both weather and climate. By learning to quantify the probabilistic distribution of atmospheric states under observational, predictive, and external forcing constraints, GAP excels in a broad range of weather-climate related tasks, including data assimilation, seamless prediction, and climate simulation. In particular, GAP is competitive with state-of-the-art ensemble assimilation, probabilistic weather forecast and seasonal prediction, yields stable millennial simulations, and reproduces climate variability from daily to decadal time scales.

cs.LG↗

arXiv2024

DRUM: Diffusion-based runoff model for probabilistic flood forecasting

Extreme floods pose escalating risks in a changing climate, yet forecasting remains challenging due to peak flow underestimation and high uncertainty. We introduce DRUM, a diffusion-based probabilistic deep learning approach that advances extreme flood forecasting across representative basins in the contiguous United States. DRUM outperforms state-of-the-art benchmarks, enhancing nowcasting skill for the top 0.1% of flows in 72.3% of studied basins. Under operational scenarios, DRUM extends reliable lead times by nearly a full day for 20- and 50-year floods. When evaluated with measured precipitation, an ideal condition, recall improves by 0.3-0.4 and the early warning window extends by 2.3 days for 50-year floods. The enhancement potential varies regionally, with precipitation-driven flood zones in the eastern and northwestern U.S. benefiting most, gaining 3-7 days in lead time. These findings highlight the transformative potential of diffusion models as a cutting-edge generative AI technique for advancing hydrology and broader Earth system sciences.

physics.geo-ph↗

arXiv2024

Boosting weather forecast via generative superensemble

Accurate weather forecasting is essential for socioeconomic activities. While data-driven forecasting demonstrates superior predictive capabilities over traditional Numerical Weather Prediction (NWP) with reduced computational demands, its deterministic nature and limited advantages over physics-based ensemble predictions restrict operational applications. We introduce the generative ensemble prediction system (GenEPS) framework to address these limitations by randomizing and mitigating both random errors and systematic biases. GenEPS provides a plug-and-play ensemble forecasting capability for deterministic models to eliminate random errors, while incorporating cross-model integration for cross-model ensembles to address systematic biases. The framework culminates in a super-ensemble approach utilizing all available data-driven models to further minimize systematic biases. GenEPS achieves an Anomaly Correlation Coefficient (ACC) of 0.679 for 500hPa geopotential (Z500), exceeding the ECMWF Ensemble Prediction System's (ENS) ACC of 0.646. Integration of the ECMWF ensemble mean further improves the ACC to 0.683. The framework also enhances extreme event representation and produces energy spectra more consistent with ERA5 reanalysis. GenEPS establishes a new paradigm in ensemble forecasting by enabling the integration of multiple data-driven models into a high-performing super-ensemble system.

physics.ao-ph↗

arXiv2024

HiHa: Introducing Hierarchical Harmonic Decomposition to Implicit Neural Compression for Atmospheric Data

The rapid development of large climate models has created the requirement of storing and transferring massive atmospheric data worldwide. Therefore, data compression is essential for meteorological research, but an efficient compression scheme capable of keeping high accuracy with high compressibility is still lacking. As an emerging technique, Implicit Neural Representation (INR) has recently acquired impressive momentum and demonstrates high promise for compressing diverse natural data. However, the INR-based compression encounters a bottleneck due to the sophisticated spatio-temporal properties and variability. To address this issue, we propose Hierarchical Harmonic decomposition implicit neural compression (HiHa) for atmospheric data. HiHa firstly segments the data into multi-frequency signals through decomposition of multiple complex harmonic, and then tackles each harmonic respectively with a frequency-based hierarchical compression module consisting of sparse storage, multi-scale INR and iterative decomposition sub-modules. We additionally design a temporal residual compression module to accelerate compression by utilizing temporal continuity. Experiments depict that HiHa outperforms both mainstream compressors and other INR-based methods in both compression fidelity and capabilities, and also demonstrate that using compressed data in existing data-driven models can achieve the same accuracy as raw data.

cs.LG↗

arXiv2024

High-resolution ensemble retrieval of cloud properties for all-day based on geostationary satellite

Clouds play a critical role in Earth's hydrological and energy cycles, and accurately representing their properties is essential for effective numerical modeling and weather forecasting. Machine learning methods have been widely used for cloud property retrieval; however, most existing techniques are deterministic and do not incorporate uncertainty quantification. Generative machine learning has made significant advances in various domains, including natural language processing, image generation, and notably weather forecasting, where it has enabled ensemble predictions and the quantification of forecast uncertainty. This ability to quantify uncertainty offers valuable opportunities for cloud remote sensing. In this study, we propose a novel cloud property retrieval method, CloudDiff, based on a generative diffusion model. By leveraging thermal infrared observations from the Himawari-8 Advanced Himawari Imager (AHI), CloudDiff generates high spatiotemporal resolution cloud properties for both daytime and nighttime conditions, increasing the resolution of Himawari-8/AHI cloud retrievals from 2 km to 1 km. Unlike deterministic retrieval methods, CloudDiff generates multiple samples from the underlying probability distribution, allowing for a diverse range of plausible retrievals and taking steps towards providing uncertainty assessment. Additionally, CloudDiff produces sharper samples and better captures fine local features, enhancing the precision of cloud property retrieval. By averaging over the ensemble of generated samples, we demonstrate that both the accuracy and reliability of the retrievals are significantly improved. These high-resolution cloud properties have been successfully applied to analyze extreme weather events, such as typhoons, providing potentially valuable insights into atmospheric processes.

physics.ao-ph↗

arXiv2024

Diffusion Model-based Probabilistic Downscaling for 180-year East Asian Climate Reconstruction

As our planet is entering into the "global boiling" era, understanding regional climate change becomes imperative. Effective downscaling methods that provide localized insights are crucial for this target. Traditional approaches, including computationally-demanding regional dynamical models or statistical downscaling frameworks, are often susceptible to the influence of downscaling uncertainty. Here, we address these limitations by introducing a diffusion probabilistic downscaling model (DPDM) into the meteorological field. This model can efficiently transform data from 1° to 0.1° resolution. Compared with deterministic downscaling schemes, it not only has more accurate local details, but also can generate a large number of ensemble members based on probability distribution sampling to evaluate the uncertainty of downscaling. Additionally, we apply the model to generate a 180-year dataset of monthly surface variables in East Asia, offering a more detailed perspective for understanding local scale climate change over the past centuries.

physics.ao-ph↗

arXiv2024

Conditional diffusion models for downscaling and bias correction of Earth system model precipitation

Climate change exacerbates extreme weather events like heavy rainfall and flooding. As these events cause severe socioeconomic damage, accurate high-resolution simulation of precipitation is imperative. However, existing Earth System Models (ESMs) struggle to resolve small-scale dynamics and suffer from biases. Traditional statistical bias correction and downscaling methods fall short in improving spatial structure, while recent deep learning methods lack controllability and suffer from unstable training. Here, we propose a machine learning framework for simultaneous bias correction and downscaling. We first map observational and ESM data to a shared embedding space, where both are unbiased towards each other, and then train a conditional diffusion model to reverse the mapping. Only observational data is used for the training, so that the diffusion model can be employed to correct and downscale any ESM field without need for retraining. Our approach ensures statistical fidelity and preserves spatial patterns larger than a chosen spatial correction scale. We demonstrate that our approach outperforms existing statistical and deep learning methods especially regarding extreme events.

physics.geo-ph↗

arXiv2024

Fast, Scale-Adaptive, and Uncertainty-Aware Downscaling of Earth System Model Fields with Generative Machine Learning

Accurate and high-resolution Earth system model (ESM) simulations are essential to assess the ecological and socio-economic impacts of anthropogenic climate change, but are computationally too expensive to be run at sufficiently high spatial resolution. Recent machine learning approaches have shown promising results in downscaling ESM simulations, outperforming state-of-the-art statistical approaches. However, existing methods require computationally costly retraining for each ESM and extrapolate poorly to climates unseen during training. We address these shortcomings by learning a consistency model (CM) that efficiently and accurately downscales arbitrary ESM simulations without retraining in a zero-shot manner. Our approach yields probabilistic downscaled fields at a resolution only limited by the observational reference data. We show that the CM outperforms state-of-the-art diffusion models at a fraction of computational cost while maintaining high controllability on the downscaling task. Further, our method generalizes to climate states unseen during training without explicitly formulated physical constraints.

physics.ao-ph↗

arXiv2023

Reconstructing Three-decade Global Fine-Grained Nighttime Light Observations by a New Super-Resolution Framework

Satellite-collected nighttime light provides a unique perspective on human activities, including urbanization, population growth, and epidemics. Yet, long-term and fine-grained nighttime light observations are lacking, leaving the analysis and applications of decades of light changes in urban facilities undeveloped. To fill this gap, we developed an innovative framework and used it to design a new super-resolution model that reconstructs low-resolution nighttime light data into high resolution. The validation of one billion data points shows that the correlation coefficient of our model at the global scale reaches 0.873, which is significantly higher than that of other existing models (maximum = 0.713). Our model also outperforms existing models at the national and urban scales. Furthermore, through an inspection of airports and roads, only our model's image details can reveal the historical development of these facilities. We provide the long-term and fine-grained nighttime light observations to promote research on human activities. The dataset is available at \url{https://doi.org/10.5281/zenodo.7859205}.

eess.IV↗

arXiv2020

Improving seasonal forecast using probabilistic deep learning

The path toward realizing the potential of seasonal forecasting and its socioeconomic benefits depends heavily on improving general circulation model based dynamical forecasting systems. To improve dynamical seasonal forecast, it is crucial to set up forecast benchmarks, and clarify forecast limitations posed by model initialization errors, formulation deficiencies, and internal climate variability. With huge cost in generating large forecast ensembles, and limited observations for forecast verification, the seasonal forecast benchmarking and diagnosing task proves challenging. In this study, we develop a probabilistic deep neural network model, drawing on a wealth of existing climate simulations to enhance seasonal forecast capability and forecast diagnosis. By leveraging complex physical relationships encoded in climate simulations, our probabilistic forecast model demonstrates favorable deterministic and probabilistic skill compared to state-of-the-art dynamical forecast systems in quasi-global seasonal forecast of precipitation and near-surface temperature. We apply this probabilistic forecast methodology to quantify the impacts of initialization errors and model formulation deficiencies in a dynamical seasonal forecasting system. We introduce the saliency analysis approach to efficiently identify the key predictors that influence seasonal variability. Furthermore, by explicitly modeling uncertainty using variational Bayes, we give a more definitive answer to how the El Nino/Southern Oscillation, the dominant mode of seasonal variability, modulates global seasonal predictability.

physics.geo-ph↗