arXiv subjects

Pedram Hassanzadeh

Publications and source records attributed to Pedram Hassanzadeh.

At least 55 records · Page 3Linked to original sources

arXiv2022

FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators

FourCastNet, short for Fourier Forecasting Neural Network, is a global data-driven weather forecasting model that provides accurate short to medium-range global predictions at $0.25^{\circ}$ resolution. FourCastNet accurately forecasts high-resolution, fast-timescale variables such as the surface wind speed, precipitation, and atmospheric water vapor. It has important implications for planning wind energy resources, predicting extreme weather events such as tropical cyclones, extra-tropical cyclones, and atmospheric rivers. FourCastNet matches the forecasting accuracy of the ECMWF Integrated Forecasting System (IFS), a state-of-the-art Numerical Weather Prediction (NWP) model, at short lead times for large-scale variables, while outperforming IFS for variables with complex fine-scale structure, including precipitation. FourCastNet generates a week-long forecast in less than 2 seconds, orders of magnitude faster than IFS. The speed of FourCastNet enables the creation of rapid and inexpensive large-ensemble forecasts with thousands of ensemble-members for improving probabilistic forecasting. We discuss how data-driven deep learning models such as FourCastNet are a valuable addition to the meteorology toolkit to aid and augment NWP models.

physics.ao-ph↗

arXiv2022

Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES

We demonstrate how incorporating physics constraints into convolutional neural networks (CNNs) enables learning subgrid-scale (SGS) closures for stable and accurate large-eddy simulations (LES) in the small-data regime (i.e., when the availability of high-quality training data is limited). Using several setups of forced 2D turbulence as the testbeds, we examine the {\it a priori} and {\it a posteriori} performance of three methods for incorporating physics: 1) data augmentation (DA), 2) CNN with group convolutions (GCNN), and 3) loss functions that enforce a global enstrophy-transfer conservation (EnsCon). While the data-driven closures from physics-agnostic CNNs trained in the big-data regime are accurate and stable, and outperform dynamic Smagorinsky (DSMAG) closures, their performance substantially deteriorate when these CNNs are trained with 40x fewer samples (the small-data regime). We show that CNN with DA and GCNN address this issue and each produce accurate and stable data-driven closures in the small-data regime. Despite its simplicity, DA, which adds appropriately rotated samples to the training set, performs as well or in some cases even better than GCNN, which uses a sophisticated equivariance-preserving architecture. EnsCon, which combines structural modeling with aspect of functional modeling, also produces accurate and stable closures in the small-data regime. Overall, GCNN+EnCon, which combines these two physics constraints, shows the best {\it a posteriori} performance in this regime. These results illustrate the power of physics-constrained learning in the small-data regime for accurate and stable LES.

physics.flu-dyn↗

arXiv2021

Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformers

There is growing interest in data-driven weather prediction (DDWP), for example using convolutional neural networks such as U-NETs that are trained on data from models or reanalysis. Here, we propose 3 components to integrate with commonly used DDWP models in order to improve their physical consistency and forecast accuracy. These components are 1) a deep spatial transformer added to the latent space of the U-NETs to preserve a property called equivariance, which is related to correctly capturing rotations and scalings of features in spatio-temporal data, 2) a data-assimilation (DA) algorithm to ingest noisy observations and improve the initial conditions for next forecasts, and 3) a multi-time-step algorithm, which combines forecasts from DDWP models with different time steps through DA, improving the accuracy of forecasts at short intervals. To show the benefit/feasibility of each component, we use geopotential height at 500~hPa (Z500) from ERA5 reanalysis and examine the short-term forecast accuracy of specific setups of the DDWP framework. Results show that the equivariance-preserving networks (U-STNs) clearly outperform the U-NETs, for example improving the forecast skill by $45\%$. Using a sigma-point ensemble Kalman (SPEnKF) algorithm for DA and U-STN as the forward model, we show that stable, accurate DA cycles are achieved even with high observation noise. The DDWP+DA framework substantially benefits from large ($O(1000)$) ensembles that are inexpensively generated with the data-driven forward model in each DA cycle. The multi-time-step DDWP+DA framework also shows promises, e.g., it reduces the average error by factors of 2-3.

physics.ao-ph↗

arXiv2021

Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learning

There is a growing interest in developing data-driven subgrid-scale (SGS) models for large-eddy simulation (LES) using machine learning (ML). In a priori (offline) tests, some recent studies have found ML-based data-driven SGS models that are trained on high-fidelity data (e.g., from direct numerical simulation, DNS) to outperform baseline physics-based models and accurately capture the inter-scale transfers, both forward (diffusion) and backscatter. While promising, instabilities in a posteriori (online) tests and inabilities to generalize to a different flow (e.g., with a higher Reynolds number, Re) remain as major obstacles in broadening the applications of such data-driven SGS models. For example, many of the same aforementioned studies have found instabilities that required often ad-hoc remedies to stabilize the LES at the expense of reducing accuracy. Here, using 2D decaying turbulence as the testbed, we show that deep fully convolutional neural networks (CNNs) can accurately predict the SGS forcing terms and the inter-scale transfers in a priori tests, and if trained with enough samples, lead to stable and accurate a posteriori LES-CNN. Further analysis attributes these instabilities to the disproportionally lower accuracy of the CNNs in capturing backscattering when the training set is small. We also show that transfer learning, which involves re-training the CNN with a small amount of data (e.g., 1%) from the new flow, enables accurate and stable a posteriori LES-CNN for flows with 16x higher Re (as well as higher grid resolution if needed). These results show the promise of CNNs with transfer learning to provide stable, accurate, and generalizable LES for practical use.

physics.flu-dyn↗

arXiv2020

Data-driven subgrid-scale modeling of forced Burgers turbulence using deep learning with generalization to higher Reynolds numbers via transfer learning

Developing data-driven subgrid-scale (SGS) models for large eddy simulations (LES) has received substantial attention recently. Despite some success, particularly in a priori (offline) tests, challenges have been identified that include numerical instabilities in a posteriori (online) tests and generalization (i.e., extrapolation) of trained data-driven SGS models, for example to higher Reynolds numbers. Here, using the stochastically forced Burgers turbulence as the test-bed, we show that deep neural networks trained using properly pre-conditioned (augmented) data yield stable and accurate a posteriori LES models. Furthermore, we show that transfer learning enables accurate/stable generalization to a flow with 10x higher Reynolds number.

physics.flu-dyn↗

arXiv2020

An Eddy-Zonal Flow Feedback Model for Propagating Annular Modes

The variability of the zonal-mean large-scale extratropical circulation is often studied using individual modes obtained from empirical orthogonal function (EOF) analyses. The prevailing reduced-order model of the leading EOF (EOF1) of zonal-mean zonal wind, called the annular mode, consists of an eddy-mean flow interaction mechanism that results in a positive feedback of EOF1 onto itself. However, a few studies have pointed out that under some circumstances in observations and GCMs, strong couplings exist between EOF1 and EOF2 at some lag times, resulting in decaying-oscillatory, or propagating, annular modes. Here, we introduce a reduced-order model for coupled EOF1 and EOF2 that accounts for potential cross-EOF eddy-zonal flow feedbacks. Using the analytical solution of this model, we derive conditions for the existence of the propagating regime based on the feedback strengths. Using this model, and idealized GCMs and stochastic prototypes, we show that cross-EOF feedbacks play an important role in controlling the persistence of the annular modes by setting the frequency of the oscillation. We find that stronger cross-EOF feedbacks lead to less persistent annular modes. Applying the coupled-EOF model to the Southern Hemisphere reanalysis data shows the existence of strong cross-EOF feedbacks. The results highlight the importance of considering the coupling of EOFs and cross-EOF feedbacks to fully understand the natural and forced variability of the zonal-mean large-scale circulation.

physics.ao-ph↗

arXiv2020

Data-driven super-parameterization using deep learning: Experimentation with multi-scale Lorenz 96 systems and transfer-learning

To make weather/climate modeling computationally affordable, small-scale processes are usually represented in terms of the large-scale, explicitly-resolved processes using physics-based or semi-empirical parameterization schemes. Another approach, computationally more demanding but often more accurate, is super-parameterization (SP), which involves integrating the equations of small-scale processes on high-resolution grids embedded within the low-resolution grids of large-scale processes. Recently, studies have used machine learning (ML) to develop data-driven parameterization (DD-P) schemes. Here, we propose a new approach, data-driven SP (DD-SP), in which the equations of the small-scale processes are integrated data-drivenly using ML methods such as recurrent neural networks. Employing multi-scale Lorenz 96 systems as testbed, we compare the cost and accuracy (in terms of both short-term prediction and long-term statistics) of parameterized low-resolution (LR), SP, DD-P, and DD-SP models. We show that with the same computational cost, DD-SP substantially outperforms LR, and is better than DD-P, particularly when scale separation is lacking. DD-SP is much cheaper than SP, yet its accuracy is the same in reproducing long-term statistics and often comparable in short-term forecasting. We also investigate generalization, finding that when models trained on data from one system are applied to a system with different forcing (e.g., more chaotic), the models often do not generalize, particularly when the short-term prediction accuracy is examined. But we show that transfer-learning, which involves re-training the data-driven model with a small amount of data from the new system, significantly improves generalization. Potential applications of DD-SP and transfer-learning in climate/weather modeling and the expected challenges are discussed.

physics.ao-ph↗

arXiv2020

Analog forecasting of extreme-causing weather patterns using deep learning

Numerical weather prediction (NWP) models require ever-growing computing time/resources, but still, have difficulties with predicting weather extremes. Here we introduce a data-driven framework that is based on analog forecasting (prediction using past similar patterns) and employs a novel deep learning pattern-recognition technique (capsule neural networks, CapsNets) and impact-based auto-labeling strategy. CapsNets are trained on mid-tropospheric large-scale circulation patterns (Z500) labeled $0-4$ depending on the existence and geographical region of surface temperature extremes over North America several days ahead. The trained networks predict the occurrence/region of cold or heat waves, only using Z500, with accuracies (recalls) of $69\%-45\%$ $(77\%-48\%)$ or $62\%-41\%$ $(73\%-47\%)$ $1-5$ days ahead. CapsNets outperform simpler techniques such as convolutional neural networks and logistic regression. Using both temperature and Z500, accuracies (recalls) with CapsNets increase to $\sim 80\%$ $(88\%)$, showing the promises of multi-modal data-driven frameworks for accurate/fast extreme weather predictions, which can augment NWP efforts in providing early warnings.

physics.ao-ph↗

arXiv2019

Data-driven prediction of a multi-scale Lorenz 96 chaotic system using deep learning methods: Reservoir computing, ANN, and RNN-LSTM

In this paper, the performance of three deep learning methods for predicting short-term evolution and for reproducing the long-term statistics of a multi-scale spatio-temporal Lorenz 96 system is examined. The methods are: echo state network (a type of reservoir computing, RC-ESN), deep feed-forward artificial neural network (ANN), and recurrent neural network with long short-term memory (RNN-LSTM). This Lorenz 96 system has three tiers of nonlinearly interacting variables representing slow/large-scale ($X$), intermediate ($Y$), and fast/small-scale ($Z$) processes. For training or testing, only $X$ is available; $Y$ and $Z$ are never known or used. We show that RC-ESN substantially outperforms ANN and RNN-LSTM for short-term prediction, e.g., accurately forecasting the chaotic trajectories for hundreds of numerical solver's time steps, equivalent to several Lyapunov timescales. The RNN-LSTM and ANN show some prediction skills as well; RNN-LSTM bests ANN. Furthermore, even after losing the trajectory, data predicted by RC-ESN and RNN-LSTM have probability density functions (PDFs) that closely match the true PDF, even at the tails. The PDF of the data predicted using ANN, however, deviates from the true PDF. Implications, caveats, and applications to data-driven and data-assisted surrogate modeling of complex nonlinear dynamical systems such as weather/climate are discussed.

cs.LG↗

arXiv2019

A Koopman-based framework for forecasting the spatiotemporal evolution of chaotic dynamics with nonlinearities modeled as exogenous forcings

We introduce a data-driven method and shows its skills for spatiotemporal prediction of high-dimensional chaotic dynamics and turbulence. The method is based on a finite-dimensional approximation of the Koopman operator where the observables are vector-valued and delay-embedded, and the nonlinearities are treated as external forcings. The predictive capabilities of the method are demonstrated for well-known prototypes of chaos such as the Kuramoto-Sivashinsky equation and Lorenz-96 system, for which the data-driven predictions are accurate for several Lyapunov timescales. Similar performance is seen for two-dimensional lid-driven cavity flows at high Reynolds numbers.

physics.flu-dyn↗

arXiv2019

Data-driven Spatio-temporal Prediction of High-dimensional Geophysical Turbulence using Koopman Operator Approximation

We show the skills of a data-driven low-dimensional linear model in predicting the spatio-temporal evolution of turbulent Rayleigh-Bénard convection. The model is based on dynamic mode decomposition with delay-embedding, which provides a data-driven finite-dimensional approximation to the system's Koopman operator. The model is built using vector-valued observables from direct numerical simulations, and can provide accurate predictions. Similar high prediction skills are found for the Kuramoto-Sivashinsky equation in the strongly-chaotic regimes.

physics.flu-dyn↗

arXiv2018

Reduced-order modeling of fully turbulent buoyancy-driven flows using the Green's function method

A One-Dimensional (1D) Reduced-Order Model (ROM) has been developed for a 3D Rayleigh-Bénard convection system in the turbulent regime with Rayleigh number $\mathrm{Ra}=10^6$. The state vector of the 1D ROM is horizontally averaged temperature. Using the Green's Function (GRF) method, which involves applying many localized, weak forcings to the system one at a time and calculating the responses using long-time averaged Direct Numerical Simulations (DNS), the system's Linear Response Function (LRF) has been computed. Another matrix, called the Eddy Flux Matrix (EFM), that relates changes in the divergence of vertical eddy heat fluxes to changes in the state vector, has also been calculated. Using various tests, it is shown that the LRF and EFM can accurately predict the time-mean responses of temperature and eddy heat flux to external forcings, and that the LRF can well predict the forcing needed to change the mean flow in a specified way (inverse problem). The non-normality of the LRF is discussed and its eigen/singular vectors are compared with the leading Proper Orthogonal Decomposition (POD) modes of the DNS data. Furthermore, it is shown that if the LRF and EFM are simply scaled by the square-root of Rayleigh number, they perform equally well for flows at other $\mathrm{Ra}$, at least in the investigated range of $ 5 \times 10^5 \le \mathrm{Ra} \le 1.25 \times 10^6$. The GRF method can be applied to develop 1D or 3D ROMs for any turbulent flow, and the calculated LRF and EFM can help with better analyzing and controlling the nonlinear system.

physics.flu-dyn↗

arXiv2018

A test case for application of convolutional neural networks to spatio-temporal climate data: Re-identifying clustered weather patterns

Convolutional neural networks (CNNs) can potentially provide powerful tools for classifying and identifying patterns in climate and environmental data. However, because of the inherent complexities of such data, which are often spatio-temporal, chaotic, and non-stationary, the CNN algorithms must be designed/evaluated for each specific dataset and application. Yet to start, CNN, a supervised technique, requires a large labeled dataset. Labeling demands (human) expert time, which combined with the limited number of relevant examples in this area, can discourage using CNNs for new problems. To address these challenges, here we (1) Propose an effective auto-labeling strategy based on using an unsupervised clustering algorithm and evaluating the performance of CNNs in re-identifying these clusters; (2) Use this approach to label thousands of daily large-scale weather patterns over North America in the outputs of a fully-coupled climate model and show the capabilities of CNNs in re-identifying the 4 clustered regimes. The deep CNN trained with $1000$ samples or more per cluster has an accuracy of $90\%$ or better. Accuracy scales monotonically but nonlinearly with the size of the training set, e.g. reaching $94\%$ with $3000$ training samples per cluster. Effects of architecture and hyperparameters on the performance of CNNs are examined and discussed.

physics.ao-ph↗

arXiv2018

Data-driven reduced modelling of turbulent Rayleigh-Benard convection using DMD-enhanced Fluctuation-Dissipation Theorem

A data-driven, model-free framework is introduced for calculating Reduced-Order Models (ROMs) capable of accurately predicting time-mean responses to external forcings, or forcings needed for specified responses, e.g., for control, in fully turbulent flows. The framework is based on using the Fluctuation-Dissipation Theorem (FDT) in the space of a limited number of modes obtained from Dynamic Mode Decomposition (DMD). Using the DMD modes as the basis functions, rather than the commonly used Proper Orthogonal Decomposition (POD) modes, resolves a previously identified problem in applying FDT to high-dimensional, non-normal turbulent flows. Employing this DMD-enhanced FDT method (FDT$_\mathrm{DMD}$), a 1D linear ROM with horizontally averaged temperature as state vector, is calculated for a 3D Rayleigh-Bénard convection system at the Rayleigh number of $10^6$ using data obtained from Direct Numerical Simulation (DNS). The calculated ROM performs well in various tests for this turbulent flow, suggesting FDT$_\mathrm{DMD}$ as a promising method for developing ROMs for high-dimensional, turbulent systems.

physics.flu-dyn↗

arXiv2018

Quantifying the annular mode dynamics in an idealized atmosphere

The linear response function (LRF) of an idealized GCM, the dry dynamical core with Held-Suarez physics, is used to accurately compute how eddy momentum and heat fluxes change in response to the zonal wind and temperature anomalies of the annular mode at the low-frequency limit. Using these results and knowing the parameterizations of surface friction and thermal radiation in Held-Suarez physics, the contribution of each physical process (meridional and vertical eddy fluxes, surface friction, thermal radiation, and meridional advection) to the annular mode dynamics is quantified. Examining the quasi-geostrophic potential vorticity balance, it is shown that the eddy feedback is positive and increases the persistence of the annular mode by a factor of more than two. Furthermore, how eddy fluxes change in response to only the barotropic component of the annular mode, i.e., vertically averaged zonal wind (and no temperature) anomaly, is also calculated similarly. The response of eddy fluxes to the barotropic-only component of the annular mode is found to be drastically different from the response to the full (barotropic+baroclinic) annular mode anomaly. In the former, the barotropic governor effect significantly suppresses the eddy generation leading to a negative eddy feedback that decreases the persistence of the annular mode by nearly a factor of three. These results suggest that the baroclinic component of the annular mode anomaly, i.e., the increased low-level baroclinicity, is essential for the persistence of the annular mode, consistent with the baroclinic mechanism but not the barotropic mechanism proposed in the previous studies.

physics.ao-ph↗

arXiv2017

Stability of 3D Gaussian vortices in an unbounded, rotating, vertically-stratified, Boussinesq flow: Linear analysis

The linear stability of three-dimensional (3D) vortices in rotating, stratified flows has been studied by analyzing the non-hydrostatic inviscid Boussinesq equations. We have focused on a widely-used model of geophysical and astrophysical vortices, which assumes an axisymmetric Gaussian structure for pressure anomalies in the horizontal and vertical directions. For a range of Rossby number ($-0.5 < Ro < 0.5$) and Burger number ($0.02 < Bu < 2.3$) relevant to observed long-lived vortices, the growth rate and spatial structure of the most unstable eigenmodes have been numerically calculated and presented as a function of $Ro-Bu$. We have found neutrally-stable vortices only over a small region of the $Ro-Bu$ parameter space: cyclones with $Ro \sim 0.02-0.05$ and $Bu \sim 0.85-0.95$. However, we have also found that anticyclones in general have slower growth rates compared to cyclones. In particular, the growth rate of the most unstable eigenmode for anticyclones in a large region of the parameter space (e.g., $Ro<0$ and $0.5 \lesssim Bu \lesssim 1.3$) is slower than $50$ turn-around times of the vortex (which often corresponds to several years for ocean eddies). For cyclones, the region with such slow growth rates is confined to $0<Ro<0.1$ and $0.5 \lesssim Bu \lesssim 1.3$. While most calculations have been done for $f/\bar{N}=0.1$ (where $f$ and $\bar{N}$ are the Coriolis and background Brunt-Väisälä frequencies), we have numerically verified and explained analytically, using non-dimensionalized equations, the insensitivity of the results to reducing $f/\bar{N}$ to the more ocean-relevant value of $0.01$. The results of this paper provide a steppingstone to study the more complicated problems of the stability of geophysical (e.g., those in the atmospheres of giant planets) and astrophysical vortices (in accretion disks).

physics.ao-ph↗

arXiv2016

Quantifying the eddy-jet feedback strength of the annular mode in an idealized GCM and reanalysis data

A linear response function (LRF) that relates the temporal tendency of zonal mean temperature and zonal wind to their anomalies and external forcing is used to accurately quantify the strength of the eddy-jet feedback associated with the annular mode in an idealized GCM. Following a simple feedback model, the results confirm the presence of a positive eddy-jet feedback in the annular mode dynamics, with a feedback strength of 0.137 day$^{-1}$ in the idealized GCM. Statistical methods proposed by earlier studies to quantify the feedback strength are evaluated against results from the LRF. It is argued that the mean-state-independent eddy forcing reduces the accuracy of these statistical methods because of the quasi-oscillatory nature of the eddy forcing. A new method is proposed to approximate the feedback strength as the regression coefficient of low-pass filtered eddy forcing onto low-pass filtered annular mode index, which converges to the value produced by the LRF when timescales longer than 200 days are used for the low-pass filtering. Applying the new low-pass filtering method to the reanalysis data, the feedback strength in the Southern annular mode is found to be 0.121 day$^{-1}$, which is presented as an improvement over previous estimates. This work also highlights the importance of using sub-daily data in the analysis by showing the significant contribution of medium-scale waves of periods less than 2 days to the annular mode dynamics, which was under-appreciated in most of previous research. The present study provides a framework to quantify the eddy-jet feedback strength in models and reanalysis data.

physics.flu-dyn↗

arXiv2016

The linear response function of an idealized atmosphere. Part 1: Construction using Green's functions and applications

A linear response function (LRF) determines the mean-response of a nonlinear climate system to weak imposed forcings, and an eddy flux matrix (EFM) determines the eddy momentum and heat flux responses to mean-flow changes. Neither LRF nor EFM can be calculated from first principles due the lack of a complete theory for turbulent eddies. Here the LRF and EFM for an idealized dry atmosphere are computed by applying numerous localized weak forcings, one at a time, to a GCM with Held-Suarez physics and calculating the mean-responses. The LRF and EFM for zonally-averaged responses are then constructed using these forcings and responses through matrix inversion. Tests demonstrate that LRF and EFM are fairly accurate. Spectral analysis of the LRF shows that the most excitable dynamical mode, the neutral vector, strongly resembles the model's Annular Mode. The framework described here can be employed to compute the LRF/EFM for zonally-asymmetric responses and more complex GCMs. The potential applications of the LRF/EFM constructed here are i) forcing a specified mean-flow for hypothesis-testing, ii) isolating/quantifying the eddy-feedbacks in complex eddy-mean flow interaction problems, and iii) evaluating/improving more generally-applicable methods currently used to construct LRFs or diagnose eddy-feedbacks in comprehensive GCMs or observations. As an example for iii, in Part 2, the LRF is also computed using the fluctuation-dissipation theorem (FDT), and the previously-calculated LRF is exploited to investigate why FDT performs poorly in some cases. It is shown that dimension-reduction using leading EOFs, which is commonly used to construct LRFs from the FDT, can significantly degrade the accuracy due to the non-normality of the operator.

physics.flu-dyn↗