arXiv subjects

Xiaoye Zhang

Publications and source records attributed to Xiaoye Zhang.

5 recordsLinked to original sources

arXiv2026

RACE: Scalable Statistical Estimation of Functional Consistency in LLM Neurons

Discovering stable neuron behavior across entire domains remains a challenge in mechanistic interpretability. Existing methods often rely on instance-level point estimates or computationally expensive procedures, which either obscure population-level variability or limit scalable domain-wide analysis. We present RACE (Residual Alignment for Consistency Estimation), a forward-pass statistical framework that evaluates the domain-wide functional consistency of Transformer neurons. Compared with gradient-based point estimates, RACE produces neuron rankings that yield more domain-specific effects under perturbation. Token-distribution shifts support the connection between the selected neurons and the target domain, while scoring requires roughly one-hundredth of the computational overhead of the gradient-based methods. Code is available at https://github.com/Nexround/RACE.

cs.AI↗

arXiv2026

Nowcast3D: Reliable precipitation nowcasting via gray-box learning

Reliable nowcasting of extreme precipitation remains difficult because convective systems are strongly nonlinear, multiscale, and nonstationary in 3D. Radar is the backbone of nowcasting, yet existing methods struggle to predict extremes: physics-based extrapolation cannot capture growth and decay, deterministic learning tends to oversmooth and underestimate peaks, and purely generative models often lack physical consistency. Hybrid schemes help but are mostly limited to 2D composite reflectivity, collapsing the atmosphere into one layer and discarding vertical structure critical for height-dependent dynamics. We introduce Nowcast3D, a gray-box, fully 3D framework that works directly on volumetric radar reflectivity. The end-to-end model couples physically constrained neural operators (advection, local diffusion, and microphysics) with a conditional diffusion model to generate ensemble forecasts with quantified uncertainty. Trained on provincial-scale 3D volumes over a $10.24^\circ \times 10.24^\circ$ region and fine-tuned on a $2.56^\circ \times 2.56^\circ$ city region ($0.01^\circ \approx 1$ km), Nowcast3D provides near-real-time forecasts up to 3 h and outperforms competitive baselines in cross-region and temporal out-of-sample tests. It can also infer wind fields without labeled supervision, supporting physically plausible transport. In a nationwide blind evaluation by 160 meteorologists, Nowcast3D ranked first and was preferred in 57% of post-hoc assessments, surpassing the leading baseline (27%). These results highlight its reliability and operational value for extreme precipitation nowcasting.

cs.LG↗

arXiv2025

LoKI: Low-damage Knowledge Implanting of Large Language Models

Fine-tuning adapts pretrained models for specific tasks but poses the risk of catastrophic forgetting (CF), where critical knowledge from pretraining is overwritten. To address the issue of CF in a general-purpose framework, we propose Low-damage Knowledge Implanting (LoKI), a parameter-efficient fine-tuning (PEFT) technique that utilizes recent mechanistic understanding of how knowledge is stored in transformer architectures. We compare LoKI against state-of-the-art PEFT methods in two real-world fine-tuning scenarios. The results show that LoKI demonstrates significantly better preservation of general capabilities. At the same time, its task-specific performance is comparable to or even surpasses that of full parameter fine-tuning and these PEFT methods across various model architectures. Our work bridges the mechanistic insights of LLMs' knowledge storage with practical fine-tuning objectives, enabling an effective balance between task-specific adaptation and the retention of general-purpose capabilities.

cs.CL↗

arXiv2025

A machine learning model for skillful climate system prediction

Climate system models (CSMs), through integrating cross-sphere interactions among the atmosphere, ocean, land, and cryosphere, have emerged as pivotal tools for deciphering climate dynamics and improving forecasting capabilities. Recent breakthroughs in artificial intelligence (AI)-driven meteorological modeling have demonstrated remarkable success in single-sphere systems and partially spheres coupled systems. However, the development of a fully coupled AI-based climate system model encompassing atmosphere-ocean-land-sea ice interactions has remained an unresolved challenge. This paper introduces FengShun-CSM, an AI-based CSM model that provides 60-day global daily forecasts for 29 critical variables across atmospheric, oceanic, terrestrial, and cryospheric domains. The model significantly outperforms the European Centre for Medium-Range Weather Forecasts (ECMWF) subseasonal-to-seasonal (S2S) model in predicting most variables, particularly precipitation, land surface, and oceanic components. This enhanced capability is primarily attributed to its improved representation of intra-seasonal variability modes, most notably the Madden-Julian Oscillation (MJO). Remarkably, FengShun-CSM exhibits substantial potential in predicting subseasonal extreme events. Such breakthroughs will advance its applications in meteorological disaster mitigation, marine ecosystem conservation, and agricultural productivity enhancement. Furthermore, it validates the feasibility of developing AI-powered CSMs through machine learning technologies, establishing a transformative paradigm for next-generation Earth system modeling.

cs.LG↗

arXiv2024

Advancing global aerosol forecasting with artificial intelligence

Aerosol forecasting is essential for air quality warnings, health risk assessment, and climate change mitigation. However, it is more complex than weather forecasting due to the intricate interactions between aerosol physicochemical processes and atmospheric dynamics, resulting in significant uncertainty and high computational costs. Here, we develop an artificial intelligence-driven global aerosol-meteorology forecasting system (AI-GAMFS), which provides reliable 5-day, 3-hourly forecasts of aerosol optical components and surface concentrations at a 0.5° x 0.625° resolution. AI-GAMFS combines Vision Transformer and U-Net in a backbone network, robustly capturing the complex aerosol-meteorology interactions via global attention and spatiotemporal encoding. Trained on 42 years of advanced aerosol reanalysis data and initialized with GEOS Forward Processing (GEOS-FP) analyses, AI-GAMFS delivers operational 5-day forecasts in one minute. It outperforms the Copernicus Atmosphere Monitoring Service (CAMS) global forecasting system, GEOS-FP forecasts, and several regional dust forecasting systems in forecasting most aerosol variables including aerosol optical depth and dust components. Our results mark a significant step forward in leveraging AI to refine physics-based aerosol forecasting, facilitating more accurate global warnings for aerosol pollution events, such as dust storms and wildfires.

physics.ao-ph↗