arXiv subjects

Louis J. Durlofsky

Publications and source records attributed to Louis J. Durlofsky.

30 records · Page 2Linked to original sources

arXiv2022

Deep reinforcement learning for optimal well control in subsurface systems with uncertain geology

A general control policy framework based on deep reinforcement learning (DRL) is introduced for closed-loop decision making in subsurface flow settings. Traditional closed-loop modeling workflows in this context involve the repeated application of data assimilation/history matching and robust optimization steps. Data assimilation can be particularly challenging in cases where both the geological style (scenario) and individual model realizations are uncertain. The closed-loop reservoir management (CLRM) problem is formulated here as a partially observable Markov decision process, with the associated optimization problem solved using a proximal policy optimization algorithm. This provides a control policy that instantaneously maps flow data observed at wells (as are available in practice) to optimal well pressure settings. The policy is represented by a temporal convolution and gated transformer blocks. Training is performed in a preprocessing step with an ensemble of prior geological models, which can be drawn from multiple geological scenarios. Example cases involving the production of oil via water injection, with both 2D and 3D geological models, are presented. The DRL-based methodology is shown to result in an NPV increase of 15% (for the 2D cases) and 33% (3D cases) relative to robust optimization over prior models, and to an average improvement of 4% in NPV relative to traditional CLRM. The solutions from the control policy are found to be comparable to those from deterministic optimization, in which the geological model is assumed to be known, even when multiple geological scenarios are considered. The control policy approach results in a 76% decrease in computational cost relative to traditional CLRM with the algorithms and parameter settings considered in this work.

physics.comp-ph↗

arXiv2022

A new flow-kinematics-based model for time-dependent effective dispersion in mixing-limited reactions

A new upscaling procedure that provides 1D representations of 2D mixing-limited reactive transport systems is developed and applied. A key complication with upscaled models in this setting is that the procedure must differentiate between interface spreading, driven by the spatially variable velocity field, and mixing, in which components contact one another and react. Our model captures the enhanced mixing caused by spreading through use of a time-dependent effective dispersion term. The early-time behavior of this dispersion is driven by flow kinematics, while at late times it reaches a Taylor-dispersion-like limit. The early-time behavior is modeled here using a very fast (purely advective) particle tracking procedure, while late-time effects are estimated from scaling arguments. The only free parameter in the model is the asymptotic effective dispersion. This quantity is determined for a few cases by calibrating 1D results to reference 2D results. For most cases, it is estimated using a fit involving a dimensionless grouping of system variables. Numerical results for bimolecular reaction systems are generated using a pseudo-spectral approach capable of resolving fronts at high Peclet numbers. Results are presented for two different types of 2D velocity fields over a wide range of parameters. The upscaled model is shown to provide highly accurate results for conversion factor, along with reasonable approximations of the spatial distribution of reaction occurrence. The model is also shown to be valid for non-reacting systems, and results for such cases can be used in the calibration step to achieve computational savings.

physics.flu-dyn↗

arXiv2021

Use of low-fidelity models with machine-learning error correction for well placement optimization

Well placement optimization is commonly performed using population-based global stochastic search algorithms. These optimizations are computationally expensive due to the large number of multiphase flow simulations that must be conducted. In this work, we present an optimization framework in which these simulations are performed with low-fidelity (LF) models. These LF models are constructed from the underlying high-fidelity (HF) geomodel using a global transmissibility upscaling procedure. Tree-based machine-learning methods, specifically random forest and light gradient boosting machine, are applied to estimate the error in objective function value (in this case net present value, NPV) associated with the LF models. In the offline (preprocessing) step, preliminary optimizations are performed using LF models, and a clustering procedure is applied to select a representative set of 100--150 well configurations to use for training. HF simulation is then performed for these configurations, and the tree-based models are trained using an appropriate set of features. In the online (runtime) step, optimization with LF models, with the machine-learning correction, is conducted. Differential evolution is used for all optimizations. Results are presented for two example cases involving the placement of vertical wells in 3D bimodal channelized geomodels. We compare the performance of our procedure to optimization using HF models. In the first case, 25 optimization runs are performed with both approaches. Our method provides an overall speedup factor of 46 relative to optimization using HF models, with the best-case NPV within 1% of the HF result. In the second case fewer HF optimization runs are conducted (consistent with actual practice), and the overall speedup factor with our approach is about 8. In this case, the best-case NPV from our procedure exceeds the HF result by 3.8%

physics.geo-ph↗

arXiv2021

Deep-learning-based coupled flow-geomechanics surrogate model for CO$_2$ sequestration

A deep-learning-based surrogate model capable of predicting flow and geomechanical responses in CO2 storage operations is presented and applied. The 3D recurrent R-U-Net model combines deep convolutional and recurrent neural networks to capture the spatial distribution and temporal evolution of saturation, pressure and surface displacement fields. The method is trained using high-fidelity simulation results for 2000 storage-aquifer realizations characterized by multi-Gaussian porosity and log-permeability fields. These numerical solutions are expensive because the domain that must be considered for the coupled problem includes not only the storage aquifer but also a surrounding region, overburden and bedrock. The surrogate model is trained to predict the 3D CO2 saturation and pressure fields in the storage aquifer, and 2D displacement maps at the Earth's surface. Detailed comparisons between surrogate model and full-order simulation results for new (test-case) storage-aquifer realizations are presented. The saturation, pressure and surface displacement fields provided by the surrogate model display a high degree of accuracy, both for individual test-case realizations and for ensemble statistics. Finally, the the recurrent R-U-Net surrogate model is applied with a rejection sampling procedure for data assimilation. Although the observations consist of only a small number of surface displacement measurements, significant uncertainty reduction in pressure buildup at the top of the storage aquifer (caprock) is achieved.

cs.LG↗

arXiv2020

Deep-learning-based surrogate flow modeling and geological parameterization for data assimilation in 3D subsurface flow

Data assimilation in subsurface flow systems is challenging due to the large number of flow simulations often required, and by the need to preserve geological realism in the calibrated (posterior) models. In this work we present a deep-learning-based surrogate model for two-phase flow in 3D subsurface formations. This surrogate model, a 3D recurrent residual U-Net (referred to as recurrent R-U-Net), consists of 3D convolutional and recurrent (convLSTM) neural networks, designed to capture the spatial-temporal information associated with dynamic subsurface flow systems. A CNN-PCA procedure (convolutional neural network post-processing of principal component analysis) for parameterizing complex 3D geomodels is also described. This approach represents a simplified version of a recently developed supervised-learning-based CNN-PCA framework. The recurrent R-U-Net is trained on the simulated dynamic 3D saturation and pressure fields for a set of random `channelized' geomodels (generated using 3D CNN-PCA). Detailed flow predictions demonstrate that the recurrent R-U-Net surrogate model provides accurate results for dynamic states and well responses for new geological realizations, along with accurate flow statistics for an ensemble of new geomodels. The 3D recurrent R-U-Net and CNN-PCA procedures are then used in combination for a challenging data assimilation problem involving a channelized system. Two different algorithms, namely rejection sampling and an ensemble-based method, are successfully applied. The overall methodology described in this paper may enable the assessment and refinement of data assimilation procedures for a range of realistic and challenging subsurface flow problems.

physics.comp-ph↗

arXiv2020

3D CNN-PCA: A Deep-Learning-Based Parameterization for Complex Geomodels

Geological parameterization enables the representation of geomodels in terms of a relatively small set of variables. Parameterization is therefore very useful in the context of data assimilation and uncertainty quantification. In this study, a deep-learning-based geological parameterization algorithm, CNN-PCA, is developed for complex 3D geomodels. CNN-PCA entails the use of convolutional neural networks as a post-processor for the low-dimensional principal component analysis representation of a geomodel. The 3D treatments presented here differ somewhat from those used in the 2D CNN-PCA procedure. Specifically, we introduce a new supervised-learning-based reconstruction loss, which is used in combination with style loss and hard data loss. The style loss uses features extracted from a 3D CNN pretrained for video classification. The 3D CNN-PCA algorithm is applied for the generation of conditional 3D realizations, defined on $60\times60\times40$ grids, for three geological scenarios (binary and bimodal channelized systems, and a three-facies channel-levee-mud system). CNN-PCA realizations are shown to exhibit geological features that are visually consistent with reference models generated using object-based methods. Statistics of flow responses ($\text{P}_{10}$, $\text{P}_{50}$, $\text{P}_{90}$ percentile results) for test sets of 3D CNN-PCA models are shown to be in consistent agreement with those from reference geomodels. Lastly, CNN-PCA is successfully applied for history matching with ESMDA for the bimodal channelized system.

cs.CV↗

arXiv2020

Data-Space Inversion Using a Recurrent Autoencoder for Time-Series Parameterization

Data-space inversion (DSI) and related procedures represent a family of methods applicable for data assimilation in subsurface flow settings. These methods differ from model-based techniques in that they provide only posterior predictions for quantities (time series) of interest, not posterior models with calibrated parameters. DSI methods require a large number of flow simulations to first be performed on prior geological realizations. Given observed data, posterior predictions can then be generated directly. DSI operates in a Bayesian setting and provides posterior samples of the data vector. In this work we develop and evaluate a new approach for data parameterization in DSI. Parameterization reduces the number of variables to determine in the inversion, and it maintains the physical character of the data variables. The new parameterization uses a recurrent autoencoder (RAE) for dimension reduction, and a long-short-term memory (LSTM) network to represent flow-rate time series. The RAE-based parameterization is combined with an ensemble smoother with multiple data assimilation (ESMDA) for posterior generation. Results are presented for two- and three-phase flow in a 2D channelized system and a 3D multi-Gaussian model. The RAE procedure, along with existing DSI treatments, are assessed through comparison to reference rejection sampling (RS) results. The new DSI methodology is shown to consistently outperform existing approaches, in terms of statistical agreement with RS results. The method is also shown to accurately capture derived quantities, which are computed from variables considered directly in DSI. This requires correlation and covariance between variables to be properly captured, and accuracy in these relationships is demonstrated. The RAE-based parameterization developed here is clearly useful in DSI, and it may also find application in other subsurface flow problems.

stat.ML↗

arXiv2020

Well control optimization using a two-step surrogate treatment

Large numbers of flow simulations are typically required for determining optimal well settings. These simulations are often computationally demanding, which poses challenges for the optimizations. In this paper we present a new two-step surrogate treatment (ST) that reduces the computational expense associated with well control optimization. The method is applicable for oil production via waterflood, with well rates optimized at a single control period. The two-step ST entails two separate optimizations, which can both be performed very efficiently. In the first optimization, optimal well-rate ratios (i.e., the fraction of total injection or production associated with each well) are determined such that a measure of velocity variability over the field is minimized, leading to more uniform sweep. In the second step, overall injection and production rates are determined. The flow physics in the first step is highly simplified, while the actual physical system is simulated in the second step. Near-globally-optimal results can be determined in both cases, as the first optimization is posed as a QP problem, and the second step entails just a single optimization variable. Under full parallelization, the overall elapsed time for the ST corresponds to the runtime for 1-2 full-order simulations. Results are presented for multiple well configurations, for 2D and 3D channelized models, and comparisons with formal optimization procedures (MADS and adjoint-gradient) are conducted. Three different fluid mobility ratios (M = 1, 3 and 5) are considered. Optimization results demonstrate that the two-step ST provides results in reasonable agreement with those from MADS and adjoint-gradient, with speedups of 5x or more. We also show that the ST is applicable in the inner-loop in field development optimization, where it will be especially useful since many different well configurations must be evaluated.

math.OC↗

arXiv2019

A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems

A deep-learning-based surrogate model is developed and applied for predicting dynamic subsurface flow in channelized geological models. The surrogate model is based on deep convolutional and recurrent neural network architectures, specifically a residual U-Net and a convolutional long short term memory recurrent network. Training samples entail global pressure and saturation maps, at a series of time steps, generated by simulating oil-water flow in many (1500 in our case) realizations of a 2D channelized system. After training, the `recurrent R-U-Net' surrogate model is shown to be capable of accurately predicting dynamic pressure and saturation maps and well rates (e.g., time-varying oil and water rates at production wells) for new geological realizations. Assessments demonstrating high surrogate-model accuracy are presented for an individual geological realization and for an ensemble of 500 test geomodels. The surrogate model is then used for the challenging problem of data assimilation (history matching) in a channelized system. For this study, posterior reservoir models are generated using the randomized maximum likelihood method, with the permeability field represented using the recently developed CNN-PCA parameterization. The flow responses required during the data assimilation procedure are provided by the recurrent R-U-Net. The overall approach is shown to lead to substantial reduction in prediction uncertainty. High-fidelity numerical simulation results for the posterior geomodels (generated by the surrogate-based data assimilation procedure) are shown to be in essential agreement with the recurrent R-U-Net predictions. The accuracy and dramatic speedup provided by the surrogate model suggest that it may eventually enable the application of more formal posterior sampling methods in realistic problems.

cs.LG↗

arXiv2019

Deep-learning-based reduced-order modeling for subsurface flow simulation

A new deep-learning-based reduced-order modeling (ROM) framework is proposed for application in subsurface flow simulation. The reduced-order model is based on an existing embed-to-control (E2C) framework and includes an auto-encoder, which projects the system to a low-dimensional subspace, and a linear transition model, which approximates the evolution of the system states in low dimension. In addition to the loss function for data mismatch considered in the original E2C framework, we introduce a physics-based loss function that penalizes predictions that are inconsistent with the governing flow equations. The loss function is also modified to emphasize accuracy in key well quantities of interest (e.g., fluid production rates). The E2C ROM is shown to be very analogous to an existing ROM, POD-TPWL, which has been extensively developed for subsurface flow simulation. The new ROM is applied to oil-water flow in a heterogeneous reservoir, with flow driven by nine wells operating under time-varying control specifications. A total of 300 high-fidelity training simulations are performed in the offline stage, and the network training requires 10-12~minutes on a Tesla V100 GPU node. Online (runtime) computations achieve speedups of $\mathcal{O}$(1000) relative to full-order simulations. Extensive test case results, with well controls varied over large ranges, are presented. Accurate ROM predictions are achieved for global saturation and pressure fields at particular times, and for injection and production well responses as a function of time. Error is shown to increase when 100 or 200 (rather than 300) training runs are used to construct the E2C ROM.

physics.comp-ph↗

arXiv2018

A Deep-Learning-Based Geological Parameterization for History Matching Complex Models

A new low-dimensional parameterization based on principal component analysis (PCA) and convolutional neural networks (CNN) is developed to represent complex geological models. The CNN-PCA method is inspired by recent developments in computer vision using deep learning. CNN-PCA can be viewed as a generalization of an existing optimization-based PCA (O-PCA) method. Both CNN-PCA and O-PCA entail post-processing a PCA model to better honor complex geological features. In CNN-PCA, rather than use a histogram-based regularization as in O-PCA, a new regularization involving a set of metrics for multipoint statistics is introduced. The metrics are based on summary statistics of the nonlinear filter responses of geological models to a pre-trained deep CNN. In addition, in the CNN-PCA formulation presented here, a convolutional neural network is trained as an explicit transform function that can post-process PCA models quickly. CNN-PCA is shown to provide both unconditional and conditional realizations that honor the geological features present in reference SGeMS geostatistical realizations for a binary channelized system. Flow statistics obtained through simulation of random CNN-PCA models closely match results for random SGeMS models for a demanding case in which O-PCA models lead to significant discrepancies. Results for history matching are also presented. In this assessment CNN-PCA is applied with derivative-free optimization, and a subspace randomized maximum likelihood method is used to provide multiple posterior models. Data assimilation and significant uncertainty reduction are achieved for existing wells, and physically reasonable predictions are also obtained for new wells. Finally, the CNN-PCA method is extended to a more complex non-stationary bimodal deltaic fan system, and is shown to provide high-quality realizations for this challenging example.

stat.ML↗

arXiv2017

Error modeling for surrogates of dynamical systems using machine learning

A machine-learning-based framework for modeling the error introduced by surrogate models of parameterized dynamical systems is proposed. The framework entails the use of high-dimensional regression techniques (e.g., random forests, LASSO) to map a large set of inexpensively computed `error indicators' (i.e., features) produced by the surrogate model at a given time instance to a prediction of the surrogate-model error in a quantity of interest (QoI). This eliminates the need for the user to hand-select a small number of informative features. The methodology requires a training set of parameter instances at which the time-dependent surrogate-model error is computed by simulating both the high-fidelity and surrogate models. Using these training data, the method first determines regression-model locality (via classification or clustering), and subsequently constructs a `local' regression model to predict the time-instantaneous error within each identified region of feature space. We consider two uses for the resulting error model: (1) as a correction to the surrogate-model QoI prediction at each time instance, and (2) as a way to statistically model arbitrary functions of the time-dependent surrogate-model error (e.g., time-integrated errors). We apply the proposed framework to model errors in reduced-order models of nonlinear oil--water subsurface flow simulations. The reduced-order models used in this work entail application of trajectory piecewise linearization with proper orthogonal decomposition. When the first use of the method is considered, numerical experiments demonstrate consistent improvement in accuracy in the time-instantaneous QoI prediction relative to the original surrogate model, across a large number of test cases. When the second use is considered, results show that the proposed method provides accurate statistical predictions of the time- and well-averaged errors.

math.NA↗