arXiv subjects

Heng Xiao

Publications and source records attributed to Heng Xiao.

At least 37 records · Page 2Linked to original sources

arXiv2021

Frame invariance and scalability of neural operators for partial differential equations

Partial differential equations (PDEs) play a dominant role in the mathematical modeling of many complex dynamical processes. Solving these PDEs often requires prohibitively high computational costs, especially when multiple evaluations must be made for different parameters or conditions. After training, neural operators can provide PDEs solutions significantly faster than traditional PDE solvers. In this work, invariance properties and computational complexity of two neural operators are examined for transport PDE of a scalar quantity. Neural operator based on graph kernel network (GKN) operates on graph-structured data to incorporate nonlocal dependencies. Here we propose a modified formulation of GKN to achieve frame invariance. Vector cloud neural network (VCNN) is an alternate neural operator with embedded frame invariance which operates on point cloud data. GKN-based neural operator demonstrates slightly better predictive performance compared to VCNN. However, GKN requires an excessively high computational cost that increases quadratically with the increasing number of discretized objects as compared to a linear increase for VCNN.

cs.LG↗

arXiv2021

Large-eddy simulations of marine boundary-layer clouds associated with cold air outbreak during the ACTIVATE campaign. Part II: aerosol-meteorology-cloud interaction

Aerosol effects on micro-/macro-physical properties of marine stratocumulus clouds over the Western North Atlantic Ocean (WNAO) are investigated using in-situ measurements and large-eddy simulations (LES) for two cold air outbreak (CAO) cases (February 28 and March 1, 2020) during the Aerosol Cloud meTeorology Interactions oVer the western ATlantic Experiment (ACTIVATE). The LES is able to reproduce the vertical profiles of liquid water content (LWC), effective radius r_eff and the cloud droplet number concentration Nc from fast cloud droplet probe (FCDP) in-situ measurements for both cases. Furthermore, we show that aerosols affect cloud properties (Nc, r_eff, and LWC) via the prescribed bulk hygroscopicity of aerosols and aerosol size distributions characteristics. Nc, r_eff, and liquid water path (LWP) are positively correlated to the bulk hygroscopicity of aerosols and aerosol number concentration (Na) while cloud fractional cover (CFC) is insensitive to the bulk hygroscopicity of aerosols and aerosol size distributions for the two cases. The changes to aerosol size distribution (number concentration, width, and the geometrical diameter) allow us to disentangle aerosol effects on cloud properties from the meteorological effects. We also use the LES results to evaluate cloud properties from two reanalysis products, ERA5 and MERRA-2. Comparing to LES, the ERA5 reanalysis is able to capture the time evolution of LWP and total cloud coverage within the study domain during both CAO cases while MERRA-2 underestimates them.

physics.ao-ph↗

arXiv2021

Frame-independent vector-cloud neural network for nonlocal constitutive modeling on arbitrary grids

Constitutive models are widely used for modeling complex systems in science and engineering, where first-principle-based, well-resolved simulations are often prohibitively expensive. For example, in fluid dynamics, constitutive models are required to describe nonlocal, unresolved physics such as turbulence and laminar-turbulent transition. However, traditional constitutive models based on partial differential equations (PDEs) often lack robustness and are too rigid to accommodate diverse calibration datasets. We propose a frame-independent, nonlocal constitutive model based on a vector-cloud neural network that can be learned with data. The model predicts the closure variable at a point based on the flow information in its neighborhood. Such nonlocal information is represented by a group of points, each having a feature vector attached to it, and thus the input is referred to as vector cloud. The cloud is mapped to the closure variable through a frame-independent neural network, invariant both to coordinate translation and rotation and to the ordering of points in the cloud. As such, the network can deal with any number of arbitrarily arranged grid points and thus is suitable for unstructured meshes in fluid simulations. The merits of the proposed network are demonstrated for scalar transport PDEs on a family of parameterized periodic hill geometries. The vector-cloud neural network is a promising tool not only as nonlocal constitutive models and but also as general surrogate models for PDEs on irregular domains.

physics.flu-dyn↗

arXiv2021

Large-eddy simulations of marine boundary-layer clouds associated with cold air outbreaks during the ACTIVATE campaign-part 1: Case setup and sensitivities to large-scale forcings

Large-eddy simulation (LES) is able to capture key boundary-layer (BL) turbulence and cloud processes. Yet, large-scale forcing and surface turbulent fluxes of sensible and latent heat are often poorly prescribed for LES simulations. We derive these quantities from measurements and reanalysis obtained for two cold air outbreak (CAO) events during Phase I of the Aerosol Cloud meTeorology Interactions oVer the western ATlantic Experiment (ACTIVATE) in February-March 2020. We study the two contrasting CAO cases by performing LES and test the sensitivity of BL structure and clouds to large-scale forcings and turbulent heat fluxes. Profiles of atmospheric state and large-scale divergence and surface turbulent heat fluxes obtained from the reanalysis data ERA5 agree reasonably well with those derived from ACTIVATE field measurements for both cases at the sampling time and location. Therefore, we adopt the time evolving heat fluxes, wind and advective tendencies profiles from ERA5 reanalysis data to drive the LES. We find that large-scale thermodynamic advective tendencies and wind relaxations are important for the LES to capture the evolving observed BL meteorological states characterized by the hourly ERA5 reanalysis data and validated by the observations. We show that the divergence (or vertical velocity) is important in regulating the BL growth driven by surface heat fluxes in LES simulations. The evolution of liquid water path is largely affected by the evolution of surface heat fluxes. The liquid water path simulated in LES agrees reasonably well with the ACTIVATE measurements.This study paves the path to investigate aerosol-cloud-meteorology interactions using LES informed and evaluated by ACTIVATE field measurements.

physics.ao-ph↗

arXiv2021

Recurrent Neural Network for End-to-End Modeling of Laminar-Turbulent Transition

Accurate prediction of laminar-turbulent transition is a critical element of computational fluid dynamics simulations for aerodynamic design across multiple flow regimes. Traditional methods of transition prediction cannot be easily extended to flow configurations where the transition process depends on a large set of parameters. In comparison, neural network methods allow higher dimensional input features to be considered without compromising the efficiency and accuracy of the traditional data driven models. Neural network methods proposed earlier follow a cumbersome methodology of predicting instability growth rates over a broad range of frequencies, which are then processed to obtain the N-factor envelope, and then, the transition location based on the correlating N-factor. This paper presents an end-to-end transition model based on a recurrent neural network, which sequentially processes the mean boundary-layer profiles along the surface of the aerodynamic body to directly predict the N-factor envelope and the transition locations over a two-dimensional airfoil. The proposed transition model has been developed and assessed using a large database of 53 airfoils over a wide range of chord Reynolds numbers and angles of attack. The sequence-to-sequence transduction model proposed herein provides a more direct approach for accurate predictions of the transition location than the earlier neural network methods, which predict the local amplification rate of a single instability mode at a fixed location along the airfoil. The large universe of airfoils encountered in various applications causes additional difficulties. As such, we provide further insights on selecting training datasets from large amounts of available data.

physics.flu-dyn↗

arXiv2021

Learning nonlocal constitutive models with neural networks

Constitutive and closure models play important roles in computational mechanics and computational physics in general. Classical constitutive models for solid and fluid materials are typically local, algebraic equations or flow rules describing the dependence of stress on the local strain and/or strain-rate. Closure models such as those describing Reynolds stress in turbulent flows and laminar--turbulent transition can involve transport PDEs (partial differential equations). Such models play similar roles to constitutive relation, but they are often more challenging to develop and calibrate as they describe nonlocal mappings and often contain many submodels. Inspired by the structure of the exact solutions to linear transport PDEs, we propose a neural network representing a region-to-point mapping to describe such nonlocal constitutive models. The range of nonlocal dependence and the convolution structure are derived from the formal solution to transport equations. The neural network-based nonlocal constitutive model is trained with data. Numerical experiments demonstrate the predictive capability of the proposed method. Moreover, the proposed network learned the embedded submodel without using data from that level, thanks to its interpretable mathematical structure, which makes it a promising alternative to traditional nonlocal constitutive models.

physics.flu-dyn↗

arXiv2021

Ensemble gradient for learning turbulence models from indirect observations

Training data-driven turbulence models with high fidelity Reynolds stress can be impractical and recently such models have been trained with velocity and pressure measurements. For gradient-based optimization, such as training deep learning models, this requires evaluating the sensitivities of the RANS equations. This paper explores the use of an ensemble approximation of the sensitivities of the RANS equations in training data-driven turbulence models with indirect observations. A deep neural network representing the turbulence model is trained using the network's gradients obtained by backpropagation and the ensemble approximation of the RANS sensitivities. Different ensemble approximations are explored and a method based on explicit projection onto the sample space is presented. As validation, the gradient approximations from the different methods are compared to that from the continuous adjoint equations. The ensemble approximation is then used to learn different turbulence models from velocity observations. In all cases, the learned model predicts improved velocities. However, it was observed that once the sensitivity of the velocity to the underlying model becomes small, the approximate nature of the ensemble gradient hinders further optimization of the underlying model. The benefits and limitations of the ensemble gradient approximation are discussed, in particular as compared to the adjoint equations.

physics.flu-dyn↗

arXiv2021

End-to-end differentiable learning of turbulence models from indirect observations

The emerging push of the differentiable programming paradigm in scientific computing is conducive to training deep learning turbulence models using indirect observations. This paper demonstrates the viability of this approach and presents an end-to-end differentiable framework for training deep neural networks to learn eddy viscosity models from indirect observations derived from the velocity and pressure fields. The framework consists of a Reynolds-averaged Navier-Stokes (RANS) solver and a neural-network-represented turbulence model, each accompanied by its derivative computations. For computing the sensitivities of the indirect observations to the Reynolds stress field, we use the continuous adjoint equations for the RANS equations, while the gradient of the neural network is obtained via its built-in automatic differentiation capability. We demonstrate the ability of this approach to learn the true underlying turbulence closure when one exists by training models using synthetic velocity data from linear and nonlinear closures. We also train a linear eddy viscosity model using synthetic velocity measurements from direct numerical simulations of the Navier-Stokes equations for which no true underlying linear closure exists. The trained deep-neural-network turbulence model showed predictive capability on similar flows.

physics.flu-dyn↗

arXiv2021

Assimilation of disparate data for enhanced reconstruction of turbulent mean flows

Reconstruction of turbulent flow based on data assimilation methods is of significant importance for improving the estimation of flow characteristics by incorporating limited observations. Existing works mainly focus on using only one observation data source, e.g., velocity, wall pressure, lift or drag force, to reconstruct the flow. In practical applications observations are disparate data sources that often vary in dimension and quality. Simultaneously incorporating these disparate data is worth investigation to improve the flow reconstruction. In this work, we investigate the disparate data assimilation with ensemble methods to enhance the reconstruction of turbulent mean flows. Specifically, a regularized ensemble Kalman method is employed to incorporate the observation of velocity and different sources of wall quantities (e.g., wall shear stress, wall pressure distribution, lift and drag force). Three numerical examples are used to demonstrate the capability of the proposed framework for assimilating disparate observation data. The first two cases, i.e., a one-dimensional planar channel flow and a two-dimensional transitional flow over plate, are used to incorporate both the sparse velocity and wall friction. In the third case of the flow over periodic hills, the wall pressure distribution and the lift and drag force are regarded as observation in addition to velocity, to recover the flow fields. The results demonstrate the merits of incorporating various disparate data sources to improve the accuracy of the flow-field estimation. The ensemble-based method can assimilate disparate data non-intrusively and robustly without requiring significant changes to the model simulation codes. The method demonstrated here opens up possibilities for assimilating realistic experimental data, which are often disparate.

physics.flu-dyn↗

arXiv2020

DAFI: An Open-Source Framework for Ensemble-Based Data Assimilation and Field Inversion

In many areas of science and engineering, it is a common task to infer physical fields from sparse observations. This paper presents the DAFI code intended as a flexible framework for two broad classes of such inverse problems: data assimilation and field inversion. DAFI generalizes these diverse problems into a general formulation and solves it with ensemble Kalman filters, a family of ensemble-based, derivative-free, Bayesian methods. This Bayesian approach has the added advantage of providing built-in uncertainty quantification. Moreover, the code provides tools for performing common tasks related to random fields, as well as I/O utilities for integration with the open-source finite volume tool OpenFOAM. The code capabilities are showcased through several test cases including state and parameter estimation for the Lorenz dynamic system, field inversion for the diffusion equations, and uncertainty quantification. The object-oriented nature of the code allows for easily interchanging different solution methods and different physics problems. It provides a simple interface for the users to supply their domain-specific physics models. Finally, the code can be used as a test-bed for new ensemble-based data assimilation and field inversion methods.

physics.comp-ph↗

arXiv2020

Enforcing Deterministic Constraints on Generative Adversarial Networks for Emulating Physical Systems

Generative adversarial networks (GANs) were initially proposed to generate images by learning from a large number of samples. Recently, GANs have been used to emulate complex physical systems such as turbulent flows. However, a critical question must be answered before GANs can be considered trusted emulators for physical systems: do GANs-generated samples conform to the various physical constraints? These include both deterministic constraints (e.g., conservation laws) and statistical constraints (e.g., energy spectrum of turbulent flows). The latter have been studied in a companion paper (Wu et al., Enforcing statistical constraints in generative adversarial networks for modeling chaotic dynamical systems. Journal of Computational Physics. 406, 109209, 2020). In the present work, we enforce deterministic yet imprecise constraints on GANs by incorporating them into the loss function of the generator. We evaluate the performance of physics-constrained GANs on two representative tasks with geometrical constraints (generating points on circles) and differential constraints (generating divergence-free flow velocity fields), respectively. In both cases, the constrained GANs produced samples that conform to the underlying constraints rather accurately, even though the constraints are only enforced up to a specified interval. More importantly, the imposed constraints significantly accelerate the convergence and improve the robustness in the training, indicating that they serve as a physics-based regularization. These improvements are noteworthy, as the convergence and robustness are two well-known obstacles in the training of GANs.

physics.comp-ph↗

arXiv2020

Obtaining the mean fields with known Reynolds stresses at steady state

With the rising of modern data science, data--driven turbulence modeling with the aid of machine learning algorithms is becoming a new promising field. Many approaches are able to achieve better Reynolds stress prediction, with much lower modeling error ($ε_M$), than traditional RANS models but they still suffer from numerical error and stability issues when the mean velocity fields are estimated using RANS equations with the predicted Reynolds stresses, illustrating that the error of solving the RANS equations ($ε_P$) is also very important. In the present work, the error $ε_P$ is studied separately by using the Reynolds stresses obtained from direct numerical simulation and we derive the sources of $ε_P$. For the implementations with known Reynolds stresses solely, we suggest to run an adjoint RANS simulation to make first guess on $ν_t^*$ and $S_{ij}^0$. With around 10 iterations, the error could be reduced by about one-order of magnitude in flow over periodic hills. The present work not only provides one robust approach to minimize $ε_P$, which may be very useful for the data-driven turbulence models, but also shows the importance of the nonlinear part of the Reynolds stresses in flow problems with flow separations.

physics.flu-dyn↗

arXiv2020

Convolutional Neural Network for Transition Modeling Based on Linear Stability Theory

Transition prediction is an important aspect of aerodynamic design because of its impact on skin friction and potential coupling with flow separation characteristics. Traditionally, the modeling of transition has relied on correlation-based empirical formulas based on integral quantities such as the shape factor of the boundary layer. However, in many applications of computational fluid dynamics, the shape factor is not straightforwardly available or not well-defined. We propose using the complete velocity profile along with other quantities (e.g., frequency, Reynolds number) to predict the perturbation amplification factor. While this can be achieved with regression models based on a classical fully connected neural network, such a model can be computationally more demanding. We propose a novel convolutional neural network inspired by the underlying physics as described by the stability equations. Specifically, convolutional layers are first used to extract integral quantities from the velocity profiles, and then fully connected layers are used to map the extracted integral quantities, along with frequency and Reynolds number, to the output (amplification ratio). Numerical tests on classical boundary layers clearly demonstrate the merits of the proposed method. More importantly, we demonstrate that, for Tollmien-Schlichting instabilities in two-dimensional, low-speed boundary layers, the proposed network encodes information in the boundary layer profiles into an integral quantity that is strongly correlated to a well-known, physically defined parameter -- the shape factor.

physics.flu-dyn↗

arXiv2020

Evaluation of ensemble methods for quantifying uncertainties in steady-state CFD applications with small ensemble sizes

Bayesian uncertainty quantification (UQ) is of interest to industry and academia as it provides a framework for quantifying and reducing the uncertainty in computational models by incorporating available data. For systems with very high computational costs, for instance, the computational fluid dynamics (CFD) problem, the conventional, exact Bayesian approach such as Markov chain Monte Carlo is intractable. To this end, the ensemble-based Bayesian methods have been used for CFD applications. However, their applicability for UQ has not been fully analyzed and understood thus far. Here, we evaluate the performance of three widely used iterative ensemble-based data assimilation methods, namely ensemble Kalman filter, ensemble randomized maximum likelihood method, and ensemble Kalman filter with multiple data assimilation for UQ problems. We present the derivations of the three ensemble methods from an optimization viewpoint. Further, a scalar case is used to demonstrate the performance of the three different approaches with emphasis on the effects of small ensemble sizes. Finally, we assess the three ensemble methods for quantifying uncertainties in steady-state CFD problems involving turbulent mean flows. Specifically, the Reynolds averaged Navier--Stokes (RANS) equation is considered the forward model, and the uncertainties in the propagated velocity are quantified and reduced by incorporating observation data. The results show that the ensemble methods cannot accurately capture the true posterior distribution, but they can provide a good estimation of the uncertainties even when very limited ensemble sizes are used. Based on the overall performance and efficiency from the comparison, the ensemble randomized maximum likelihood method is identified as the best choice of approximate Bayesian UQ approach~among the three ensemble methods evaluated here.

physics.comp-ph↗

arXiv2020

Regularized Ensemble Kalman Methods for Inverse Problems

Inverse problems are common and important in many applications in computational physics but are inherently ill-posed with many possible model parameters resulting in satisfactory results in the observation space. When solving the inverse problem with adjoint-based optimization, the problem can be regularized by adding additional constraints in the cost function. However, similar regularizations have not been used in ensemble-based methods, where the same optimization is done implicitly through the analysis step rather than through explicit minimization of the cost function. Ensemble-based methods, and in particular ensemble Kalman methods, have gained popularity in practice where physics models typically do not have readily available adjoint capabilities. While the model outputs can be improved by incorporating observations using these methods, the lack of regularization means the inference of the model parameters remains ill-posed. Here we propose a regularized ensemble Kalman method capable of enforcing regularization constraints. Specifically, we derive a modified analysis scheme that implicitly minimizes a cost function with generalized constraints. We demonstrate the method's ability to regularize the inverse problem with three cases of increasing complexity, starting with inferring scalar model parameters. As a final case, we utilize the proposed method to infer the closure field in the Reynolds-averaged Navier--Stokes equations; a problem of significant importance in fluid dynamics and many engineering applications.

physics.comp-ph↗

arXiv2019

Enforcing Boundary Conditions on Physical Fields in Bayesian Inversion

Inverse problems in computational mechanics consist of inferring physical fields that are latent in the model describing some observable fields. For instance, an inverse problem of interest is inferring the Reynolds stress field in the Navier--Stokes equations describing mean fluid velocity and pressure. The physical nature of the latent fields means they have their own set of physical constraints, including boundary conditions. The inherent ill-posedness of inverse problems, however, means that there exist many possible latent fields that do not satisfy their physical constraints while still resulting in a satisfactory agreement in the observation space. These physical constraints must therefore be enforced through the problem formulation. So far there has been no general approach to enforce boundary conditions on latent fields in inverse problems in computational mechanics, with these constraints often simply ignored. In this work we demonstrate how to enforce boundary conditions in Bayesian inversion problems by choice of the statistical model for the latent fields. Specifically, this is done by modifying the covariance kernel to guarantee that all realizations satisfy known values or derivatives at the boundary. As a test case the problem of inferring the eddy viscosity in the Reynolds-averaged Navier--Stokes equations is considered. The results show that enforcing these constraints results in similar improvements in the output fields but with latent fields that behave as expected at the boundaries.

physics.comp-ph↗

arXiv2019

Flows Over Periodic Hills of Parameterized Geometries: A Dataset for Data-Driven Turbulence Modeling From Direct Simulations

Computational fluid dynamics models based on Reynolds-averaged Navier--Stokes equations with turbulence closures still play important roles in engineering design and analysis. However, the development of turbulence models has been stagnant for decades. With recent advances in machine learning, data-driven turbulence models have become attractive alternatives worth further explorations. However, a major obstacle in the development of data-driven turbulence models is the lack of training data. In this work, we survey currently available public turbulent flow databases and conclude that they are inadequate for developing and validating data-driven models. Rather, we need more benchmark data from systematically and continuously varied flow conditions (e.g., Reynolds number and geometry) with maximum coverage in the parameter space for this purpose. To this end, we perform direct numerical simulations of flows over periodic hills with varying slopes, resulting in a family of flows over periodic hills which ranges from incipient to mild and massive separations. We further demonstrate the use of such a dataset by training a machine learning model that predicts Reynolds stress anisotropy based on a set of mean flow features. We expect the generated dataset, along with its design methodology and the example application presented herein, will facilitate development and comparison of future data-driven turbulence models.

physics.flu-dyn↗

arXiv2019

Enforcing Statistical Constraints in Generative Adversarial Networks for Modeling Chaotic Dynamical Systems

Simulating complex physical systems often involves solving partial differential equations (PDEs) with some closures due to the presence of multi-scale physics that cannot be fully resolved. Therefore, reliable and accurate closure models for unresolved physics remains an important requirement for many computational physics problems, e.g., turbulence simulation. Recently, several researchers have adopted generative adversarial networks (GANs), a novel paradigm of training machine learning models, to generate solutions of PDEs-governed complex systems without having to numerically solve these PDEs. However, GANs are known to be difficult in training and likely to converge to local minima, where the generated samples do not capture the true statistics of the training data. In this work, we present a statistical constrained generative adversarial network by enforcing constraints of covariance from the training data, which results in an improved machine-learning-based emulator to capture the statistics of the training data generated by solving fully resolved PDEs. We show that such a statistical regularization leads to better performance compared to standard GANs, measured by (1) the constrained model's ability to more faithfully emulate certain physical properties of the system and (2) the significantly reduced (by up to 80%) training time to reach the solution. We exemplify this approach on the Rayleigh-Benard convection, a turbulent flow system that is an idealized model of the Earth's atmosphere. With the growth of high-fidelity simulation databases of physical systems, this work suggests great potential for being an alternative to the explicit modeling of closures or parameterizations for unresolved physics, which are known to be a major source of uncertainty in simulating multi-scale physical systems, e.g., turbulence or Earth's climate.

physics.comp-ph↗