arXiv subjects

Dongxiao Zhang

Publications and source records attributed to Dongxiao Zhang.

At least 55 records · Page 3Linked to original sources

arXiv2022

Interpretable machine learning optimization (InterOpt) for operational parameters: a case study of highly-efficient shale gas development

An algorithm named InterOpt for optimizing operational parameters is proposed based on interpretable machine learning, and is demonstrated via optimization of shale gas development. InterOpt consists of three parts: a neural network is used to construct an emulator of the actual drilling and hydraulic fracturing process in the vector space (i.e., virtual environment); the Sharpley value method in interpretable machine learning is applied to analyzing the impact of geological and operational parameters in each well (i.e., single well feature impact analysis); and ensemble randomized maximum likelihood (EnRML) is conducted to optimize the operational parameters to comprehensively improve the efficiency of shale gas development and reduce the average cost. In the experiment, InterOpt provides different drilling and fracturing plans for each well according to its specific geological conditions, and finally achieved an average cost reduction of 9.7% for a case study with 104 wells.

cs.LG↗

arXiv2022

Uncertainty quantification of two-phase flow in porous media via coupled-TgNN surrogate model

Uncertainty quantification (UQ) of subsurface two-phase flow usually requires numerous executions of forward simulations under varying conditions. In this work, a novel coupled theory-guided neural network (TgNN) based surrogate model is built to facilitate computation efficiency under the premise of satisfactory accuracy. The core notion of this proposed method is to bridge two separate blocks on top of an overall network. They underlie the TgNN model in a coupled form, which reflects the coupling nature of pressure and water saturation in the two-phase flow equation. The TgNN model not only relies on labeled data, but also incorporates underlying scientific theory and experiential rules (e.g., governing equations, stochastic parameter fields, boundary and initial conditions, well conditions, and expert knowledge) as additional components into the loss function. The performance of the TgNN-based surrogate model for two-phase flow problems is tested by different numbers of labeled data and collocation points, as well as the existence of data noise. The proposed TgNN-based surrogate model offers an effective way to solve the coupled nonlinear two-phase flow problem and demonstrates good accuracy and strong robustness when compared with the purely data-driven surrogate model. By combining the accurate TgNN-based surrogate model with the Monte Carlo method, UQ tasks can be performed at a minimum cost to evaluate statistical quantities. Since the heterogeneity of the random fields strongly impacts the results of the surrogate model, corresponding variance and correlation length are added to the input of the neural network to maintain its predictive capacity. The results show that the TgNN-based surrogate model achieves satisfactory accuracy, stability, and efficiency in UQ problems of subsurface two-phase flow.

physics.flu-dyn↗

arXiv2022

AutoKE: An automatic knowledge embedding framework for scientific machine learning

Imposing physical constraints on neural networks as a method of knowledge embedding has achieved great progress in solving physical problems described by governing equations. However, for many engineering problems, governing equations often have complex forms, including complex partial derivatives or stochastic physical fields, which results in significant inconveniences from the perspective of implementation. In this paper, a scientific machine learning framework, called AutoKE, is proposed, and a reservoir flow problem is taken as an instance to demonstrate that this framework can effectively automate the process of embedding physical knowledge. In AutoKE, an emulator comprised of deep neural networks (DNNs) is built for predicting the physical variables of interest. An arbitrarily complex equation can be parsed and automatically converted into a computational graph through the equation parser module, and the fitness of the emulator to the governing equation is evaluated via automatic differentiation. Furthermore, the fixed weights in the loss function are substituted with adaptive weights by incorporating the Lagrangian dual method. Neural architecture search (NAS) is also introduced into the AutoKE to select an optimal network architecture of the emulator according to the specific problem. Finally, we apply transfer learning to enhance the scalability of the emulator. In experiments, the framework is verified by a series of physical problems in which it can automatically embed physical knowledge into an emulator without heavy hand-coding. The results demonstrate that the emulator can not only make accurate predictions, but also be applied to similar problems with high efficiency via transfer learning.

cs.LG↗

arXiv2022

Inferring electrochemical performance and parameters of Li-ion batteries based on deep operator networks

The Li-ion battery is a complex physicochemical system that generally takes applied current as input and terminal voltage as output. The mappings from current to voltage can be described by several kinds of models, such as accurate but inefficient physics-based models, and efficient but sometimes inaccurate equivalent circuit and black-box models. To realize accuracy and efficiency simultaneously in battery modeling, we propose to build a data-driven surrogate for a battery system while incorporating the underlying physics as constraints. In this work, we innovatively treat the functional mapping from current curve to terminal voltage as a composite of operators, which is approximated by the powerful deep operator network (DeepONet). Its learning capability is firstly verified through a predictive test for Li-ion concentration at two electrodes. In this experiment, the physics-informed DeepONet is found to be more robust than the purely data-driven DeepONet, especially in temporal extrapolation scenarios. A composite surrogate is then constructed for mapping current curve and solid diffusivity to terminal voltage with three operator networks, in which two parallel physics-informed DeepONets are firstly used to predict Li-ion concentration at two electrodes, and then based on their surface values, a DeepONet is built to give terminal voltage predictions. Since the surrogate is differentiable anywhere, it is endowed with the ability to learn from data directly, which was validated by using terminal voltage measurements to estimate input parameters. The proposed surrogate built upon operator networks possesses great potential to be applied in on-board scenarios, such as battery management system, since it integrates efficiency and accuracy by incorporating underlying physics, and also leaves an interface for model refinement through a totally differentiable model structure.

physics.comp-ph↗

arXiv2022

Identification of Physical Processes and Unknown Parameters of 3D Groundwater Contaminant Problems via Theory-guided U-net

Identification of unknown physical processes and parameters of groundwater contaminant sources is a challenging task due to their ill-posed and non-unique nature. Numerous works have focused on determining nonlinear physical processes through model selection methods. However, identifying corresponding nonlinear systems for different physical phenomena using numerical methods can be computationally prohibitive. With the advent of machine learning (ML) algorithms, more efficient surrogate models based on neural networks (NNs) have been developed in various disciplines. In this work, a theory-guided U-net (TgU-net) framework is proposed for surrogate modeling of three-dimensional (3D) groundwater contaminant problems in order to efficiently elucidate their involved processes and unknown parameters. In TgU-net, the underlying governing equations are embedded into the loss function of U-net as soft constraints. For the considered groundwater contaminant problem, sorption is considered to be a potential process of an uncertain type, and three equilibrium sorption isotherm types (i.e., linear, Freundlich, and Langmuir) are considered. Different from traditional approaches in which one model corresponds to one equation, these three sorption types are modeled through only one TgU-net surrogate. The three mentioned sorption terms are integrated into one equation by assigning indicators. Accurate predictions illustrate the satisfactory generalizability and extrapolability of the constructed TgU-net. Furthermore, based on the constructed TgU-net surrogate, a data assimilation method is employed to identify the physical process and parameters simultaneously. This work shows the possibility of governing equation discovery of physical problems that contain multiple and even uncertain processes by using deep learning and data assimilation methods.

physics.comp-ph↗

arXiv2022

Semantic interpretation for convolutional neural networks: What makes a cat a cat?

The interpretability of deep neural networks has attracted increasing attention in recent years, and several methods have been created to interpret the "black box" model. Fundamental limitations remain, however, that impede the pace of understanding the networks, especially the extraction of understandable semantic space. In this work, we introduce the framework of semantic explainable AI (S-XAI), which utilizes row-centered principal component analysis to obtain the common traits from the best combination of superpixels discovered by a genetic algorithm, and extracts understandable semantic spaces on the basis of discovered semantically sensitive neurons and visualization techniques. Statistical interpretation of the semantic space is also provided, and the concept of semantic probability is proposed for the first time. Our experimental results demonstrate that S-XAI is effective in providing a semantic interpretation for the CNN, and offers broad usage, including trustworthiness assessment and semantic sample searching.

cs.LG↗

arXiv2022

Deep learning based closed-loop optimization of geothermal reservoir production

To maximize the economic benefits of geothermal energy production, it is essential to optimize geothermal reservoir management strategies, in which geologic uncertainty should be considered. In this work, we propose a closed-loop optimization framework, based on deep learning surrogates, for the well control optimization of geothermal reservoirs. In this framework, we construct a hybrid convolution-recurrent neural network surrogate, which combines the convolution neural network (CNN) and long short-term memory (LSTM) recurrent network. The convolution structure can extract spatial information of geologic parameter fields and the recurrent structure can approximate sequence-to-sequence mapping. The trained model can predict time-varying production responses (rate, temperature, etc.) for cases with different permeability fields and well control sequences. In the closed-loop optimization framework, production optimization based on the differential evolution (DE) algorithm, and data assimilation based on the iterative ensemble smoother (IES), are performed alternately to achieve real-time well control optimization and geologic parameter estimation as the production proceeds. In addition, the averaged objective function over the ensemble of geologic parameter estimations is adopted to consider geologic uncertainty in the optimization process. Several geothermal reservoir development cases are designed to test the performance of the proposed production optimization framework. The results show that the proposed framework can achieve efficient and effective real-time optimization and data assimilation in the geothermal reservoir production process.

cs.LG↗

arXiv2022

Integration of knowledge and data in machine learning

Scientific research's mandate is to comprehend and explore the world, as well as to improve it based on experience and knowledge. Knowledge embedding and knowledge discovery are two significant methods of integrating knowledge and data. Through knowledge embedding, the barriers between knowledge and data can be eliminated, and machine learning models with physical common sense can be established. Meanwhile, humans' understanding of the world is always limited, and knowledge discovery takes advantage of machine learning to extract new knowledge from observations. Knowledge discovery can not only assist researchers to better grasp the nature of physics, but it can also support them in conducting knowledge embedding research. A closed loop of knowledge generation and usage are formed by combining knowledge embedding with knowledge discovery, which can improve the robustness and accuracy of models and uncover previously unknown scientific principles. This study summarizes and analyzes extant literature, as well as identifies research gaps and future opportunities.

cs.AI↗

arXiv2022

High-throughput discovery of chemical structure-polarity relationships combining automation and machine learning techniques

As an essential attribute of organic compounds, polarity has a profound influence on many molecular properties such as solubility and phase transition temperature. Thin layer chromatography (TLC) represents a commonly used technique for polarity measurement. However, current TLC analysis presents several problems, including the need for a large number of attempts to obtain suitable conditions, as well as irreproducibility due to non-standardization. Herein, we describe an automated experiment system for TLC analysis. This system is designed to conduct TLC analysis automatically, facilitating high-throughput experimentation by collecting large experimental data under standardized conditions. Using these datasets, machine learning (ML) methods are employed to construct surrogate models correlating organic compounds' structures and their polarity using retardation factor (Rf). The trained ML models are able to predict the Rf value curve of organic compounds with high accuracy. Furthermore, the constitutive relationship between the compound and its polarity can also be discovered through these modeling methods, and the underlying mechanism is rationalized through adsorption theories. The trained ML models not only reduce the need for empirical optimization currently required for TLC analysis, but also provide general guidelines for the selection of conditions, making TLC an easily accessible tool for the broad scientific community.

physics.chem-ph↗

arXiv2021

Deep-learning-based upscaling method for geologic models via theory-guided convolutional neural network

Large-scale or high-resolution geologic models usually comprise a huge number of grid blocks, which can be computationally demanding and time-consuming to solve with numerical simulators. Therefore, it is advantageous to upscale geologic models (e.g., hydraulic conductivity) from fine-scale (high-resolution grids) to coarse-scale systems. Numerical upscaling methods have been proven to be effective and robust for coarsening geologic models, but their efficiency remains to be improved. In this work, a deep-learning-based method is proposed to upscale the fine-scale geologic models, which can assist to improve upscaling efficiency significantly. In the deep learning method, a deep convolutional neural network (CNN) is trained to approximate the relationship between the coarse grid of hydraulic conductivity fields and the hydraulic heads, which can then be utilized to replace the numerical solvers while solving the flow equations for each coarse block. In addition, physical laws (e.g., governing equations and periodic boundary conditions) can also be incorporated into the training process of the deep CNN model, which is termed the theory-guided convolutional neural network (TgCNN). With the physical information considered, dependence on the data volume of training the deep learning models can be reduced greatly. Several subsurface flow cases are introduced to test the performance of the proposed deep-learning-based upscaling method, including 2D and 3D cases, and isotropic and anisotropic cases. The results show that the deep learning method can provide equivalent upscaling accuracy to the numerical method, and efficiency can be improved significantly compared to numerical upscaling.

cs.LG↗

arXiv2021

Deep-Learning Discovers Macroscopic Governing Equations for Viscous Gravity Currents from Microscopic Simulation Data

Although deep-learning has been successfully applied in a variety of science and engineering problems owing to its strong high-dimensional nonlinear mapping capability, it is of limited use in scientific knowledge discovery. In this work, we propose a deep-learning based framework to discover the macroscopic governing equation of viscous gravity current based on high-resolution microscopic simulation data without the need for prior knowledge of underlying terms. For two typical scenarios with different viscosity ratios, the deep-learning based equations exactly capture the same dominated terms as the theoretically derived equations for describing long-term asymptotic behaviors, which validates the proposed framework. Unknown macroscopic equations are then obtained for describing short-term behaviors, and additional deep-learned compensation terms are eventually discovered. Comparison of posterior tests shows that the deep-learning based PDEs actually perform better than the theoretically derived PDEs in predicting evolving viscous gravity currents for both long-term and short-term regimes. Moreover, the proposed framework is proven to be very robust against non-biased data noise for training, which is up to 20%. Consequently, the presented deep-learning framework shows considerable potential for discovering unrevealed intrinsic laws in scientific semantic space from raw experimental or simulation results in data space.

physics.comp-ph↗

arXiv2021

Uncertainty quantification and inverse modeling for subsurface flow in 3D heterogeneous formations using a theory-guided convolutional encoder-decoder network

We build surrogate models for dynamic 3D subsurface single-phase flow problems with multiple vertical producing wells. The surrogate model provides efficient pressure estimation of the entire formation at any timestep given a stochastic permeability field, arbitrary well locations and penetration lengths, and a timestep matrix as inputs. The well production rate or bottom hole pressure can then be determined based on Peaceman's formula. The original surrogate modeling task is transformed into an image-to-image regression problem using a convolutional encoder-decoder neural network architecture. The residual of the governing flow equation in its discretized form is incorporated into the loss function to impose theoretical guidance on the model training process. As a result, the accuracy and generalization ability of the trained surrogate models are significantly improved compared to fully data-driven models. They are also shown to have flexible extrapolation ability to permeability fields with different statistics. The surrogate models are used to conduct uncertainty quantification considering a stochastic permeability field, as well as to infer unknown permeability information based on limited well production data and observation data of formation properties. Results are shown to be in good agreement with traditional numerical simulation tools, but computational efficiency is dramatically improved.

cs.CE↗

arXiv2021

Surrogate and inverse modeling for two-phase flow in porous media via theory-guided convolutional neural network

The theory-guided convolutional neural network (TgCNN) framework, which can incorporate discretized governing equation residuals into the training of convolutional neural networks (CNNs), is extended to two-phase porous media flow problems in this work. The two principal variables of the considered problem, pressure and saturation, are approximated simultaneously with two CNNs, respectively. Pressure and saturation are coupled with each other in the governing equations, and thus the two networks are also mutually conditioned in the training process by the discretized governing equations, which also increases the difficulty of model training. The coupled and discretized equations can provide valuable information in the training process. With the assistance of theory-guidance, the TgCNN surrogates can achieve better accuracy than ordinary CNN surrogates in two-phase flow problems. Moreover, a piecewise training strategy is proposed for the scenario with varying well controls, in which the TgCNN surrogates are constructed for different segments on the time dimension and stacked together to predict solutions for the whole time-span. For scenarios with larger variance of the formation property field, the TgCNN surrogates can also achieve satisfactory performance. The constructed TgCNN surrogates are further used for inversion of permeability fields by combining them with the iterative ensemble smoother (IES) algorithm, and sufficient inversion accuracy is obtained with improved efficiency.

physics.geo-ph↗

arXiv2021

An Adaptive Deep Learning Framework for Day-ahead Forecasting of Photovoltaic Power Generation

Accurate forecasts of photovoltaic power generation (PVPG) are essential to optimize operations between energy supply and demand. Recently, the propagation of sensors and smart meters has produced an enormous volume of data, which supports the development of data based PVPG forecasting. Although emerging deep learning (DL) models, such as the long short-term memory (LSTM) model, based on historical data, have provided effective solutions for PVPG forecasting with great successes, these models utilize offline learning. As a result, DL models cannot take advantage of the opportunity to learn from newly-arrived data, and are unable to handle concept drift caused by installing extra PV units and unforeseen PV unit failures. Consequently, to improve day-ahead PVPG forecasting accuracy, as well as eliminate the impacts of concept drift, this paper proposes an adaptive LSTM (AD-LSTM) model, which is a DL framework that can not only acquire general knowledge from historical data, but also dynamically learn specific knowledge from newly-arrived data. A two-phase adaptive learning strategy (TP-ALS) is integrated into AD-LSTM, and a sliding window (SDWIN) algorithm is proposed, to detect concept drift in PV systems. Multiple datasets from PV systems are utilized to assess the feasibility and effectiveness of the proposed approaches. The developed AD-LSTM model demonstrates greater forecasting capability than the offline LSTM model, particularly in the presence of concept drift. Additionally, the proposed AD-LSTM model also achieves superior performance in terms of day-ahead PVPG forecasting compared to other traditional machine learning models and statistical models in the literature.

cs.LG↗

arXiv2021

Constructing Sub-scale Surrogate Model for Proppant Settling in Inclined Fractures from Simulation Data with Multi-fidelity Neural Network

Particle settling in inclined channels is an important phenomenon that occurs during hydraulic fracturing of shale gas production. Generally, in order to accurately simulate the large-scale (field-scale) proppant transport process, constructing a fast and accurate sub-scale proppant settling model, or surrogate model, becomes a critical issue, since mapping between physical parameters and proppant settling velocity is complex. Previously, particle settling has usually been investigated via high-fidelity experiments and meso-scale numerical simulations, both of which are time-consuming. In this work, a new method is proposed and utilized, i.e., the multi-fidelity neural network (MFNN), to construct a settling surrogate model, which could utilize both high-fidelity and low-fidelity (thus, less expensive) data. The results demonstrate that constructing the settling surrogate with the MFNN can reduce the need for high-fidelity data and thus computational cost by 80%, while the accuracy lost is less than 5% compared to a high-fidelity surrogate. Moreover, the investigated particle settling surrogate is applied in macro-scale proppant transport simulation, which shows that the settling model is significant to proppant transport and yields accurate results. This opens novel pathways for rapidly predicting proppant settling velocity in reservoir applications.

physics.flu-dyn↗

arXiv2021

RockGPT: Reconstructing three-dimensional digital rocks from single two-dimensional slice from the perspective of video generation

Random reconstruction of three-dimensional (3D) digital rocks from two-dimensional (2D) slices is crucial for elucidating the microstructure of rocks and its effects on pore-scale flow in terms of numerical modeling, since massive samples are usually required to handle intrinsic uncertainties. Despite remarkable advances achieved by traditional process-based methods, statistical approaches and recently famous deep learning-based models, few works have focused on producing several kinds of rocks with one trained model and allowing the reconstructed samples to satisfy certain given properties, such as porosity. To fill this gap, we propose a new framework, named RockGPT, which is composed of VQ-VAE and conditional GPT, to synthesize 3D samples based on a single 2D slice from the perspective of video generation. The VQ-VAE is utilized to compress high-dimensional input video, i.e., the sequence of continuous rock slices, to discrete latent codes and reconstruct them. In order to obtain diverse reconstructions, the discrete latent codes are modeled using conditional GPT in an autoregressive manner, while incorporating conditional information from a given slice, rock type, and porosity. We conduct two experiments on five kinds of rocks, and the results demonstrate that RockGPT can produce different kinds of rocks with the same model, and the reconstructed samples can successfully meet certain specified porosities. In a broader sense, through leveraging the proposed conditioning scheme, RockGPT constitutes an effective way to build a general model to produce multiple kinds of rocks simultaneously that also satisfy user-defined properties.

eess.IV↗

arXiv2021

Any equation is a forest: Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE)

Partial differential equations (PDEs) are concise and understandable representations of domain knowledge, which are essential for deepening our understanding of physical processes and predicting future responses. However, the PDEs of many real-world problems are uncertain, which calls for PDE discovery. We propose the symbolic genetic algorithm (SGA-PDE) to discover open-form PDEs directly from data without prior knowledge about the equation structure. SGA-PDE focuses on the representation and optimization of PDE. Firstly, SGA-PDE uses symbolic mathematics to realize the flexible representation of any given PDE, transforms a PDE into a forest, and converts each function term into a binary tree. Secondly, SGA-PDE adopts a specially designed genetic algorithm to efficiently optimize the binary trees by iteratively updating the tree topology and node attributes. The SGA-PDE is gradient-free, which is a desirable characteristic in PDE discovery since it is difficult to obtain the gradient between the PDE loss and the PDE structure. In the experiment, SGA-PDE not only successfully discovered nonlinear Burgers' equation, Korteweg-de Vries (KdV) equation, and Chafee-Infante equation, but also handled PDEs with fractional structure and compound functions that cannot be solved by conventional PDE discovery methods.

cs.NE↗

arXiv2021

Digital rock reconstruction with user-defined properties using conditional generative adversarial networks

Uncertainty is ubiquitous with flow in subsurface rocks because of their inherent heterogeneity and lack of in-situ measurements. To complete uncertainty analysis in a multi-scale manner, it is a prerequisite to provide sufficient rock samples. Even though the advent of digital rock technology offers opportunities to reproduce rocks, it still cannot be utilized to provide massive samples due to its high cost, thus leading to the development of diversified mathematical methods. Among them, two-point statistics (TPS) and multi-point statistics (MPS) are commonly utilized, which feature incorporating low-order and high-order statistical information, respectively. Recently, generative adversarial networks (GANs) are becoming increasingly popular since they can reproduce training images with excellent visual and consequent geologic realism. However, standard GANs can only incorporate information from data, while leaving no interface for user-defined properties, and thus may limit the representativeness of reconstructed samples. In this study, we propose conditional GANs for digital rock reconstruction, aiming to reproduce samples not only similar to the real training data, but also satisfying user-specified properties. In fact, the proposed framework can realize the targets of MPS and TPS simultaneously by incorporating high-order information directly from rock images with the GANs scheme, while preserving low-order counterparts through conditioning. We conduct three reconstruction experiments, and the results demonstrate that rock type, rock porosity, and correlation length can be successfully conditioned to affect the reconstructed rock images. Furthermore, in contrast to existing GANs, the proposed conditioning enables learning of multiple rock types simultaneously, and thus invisibly saves computational cost.

cs.CV↗