arXiv subjects

Bjarne Grimstad

Publications and source records attributed to Bjarne Grimstad.

15 recordsLinked to original sources

arXiv2025

Multi-task and few-shot learning in virtual flow metering

Recent literature has explored various ways to improve soft sensors by utilizing learning algorithms with transferability. A performance gain is generally attained when knowledge is transferred among strongly related soft sensor learning tasks. One setting where it is reasonable to expect strongly related tasks, is when learning soft sensors for separate process units that are of the same type. Applying methods that exploit transferability in this setting leads to what we call multi-unit soft sensing. This paper formulates a probabilistic, hierarchical model for multi-unit soft sensing. The model is implemented using a deep neural network. The proposed learning method is studied empirically on a large-scale industrial case by developing virtual flow meters (a type of soft sensor) for 80 petroleum wells. We investigate how the model generalizes with the number of wells/units. We demonstrate that multi-unit models learned from data from many wells permit few-shot learning of virtual flow meters for new wells. Surprisingly, regarding the difficulty of the tasks, few-shot learning on 1-3 data points often leads to high performance on new wells.

stat.ML↗

arXiv2025

On a closed-loop identification challenge in feedback optimization

Feedback optimization has emerged as an effective strategy for steady-state optimization of dynamical systems. By exploiting models of the steady-state input-output sensitivity, methods of this type are often sample efficient, and their use of feedback ensures that they are robust against model error. Still, this robustness has its limitations, and the dependence on a model may hinder convergence in settings with high model error. We investigate here the effect of a particular type of model error: bias due to identifying the model from closed-loop data. Our main results are a sufficient convergence condition, and a converse divergence condition. The convergence condition requires a matrix which depends on the closed-loop sensitivity and a noise-to-signal ratio of the data generating system to be positive definite. The negative definiteness of the same matrix characterizes an extreme case where the bias due to closed-loop data results in divergence of model-based feedback optimization.

math.OC↗

arXiv2025

Multi-task neural networks by learned contextual inputs

This paper explores learned-context neural networks. It is a multi-task learning architecture based on a fully shared neural network and an augmented input vector containing trainable task parameters. The architecture is interesting due to its powerful task adaption mechanism, which facilitates a low-dimensional task parameter space. Theoretically, we show that a scalar task parameter is sufficient for universal approximation of all tasks, which is not necessarily the case for more common architectures. Empirically it is shown that, for homogeneous tasks, the dimension of the task parameter may vary with the complexity of the tasks, but a small task parameter space is generally viable. The task parameter space is found to be well-behaved, which simplifies workflows related to updating models as new data arrives, and learning new tasks with the shared parameters are frozen. Additionally, the architecture displays robustness towards datasets where tasks have few data points. The architecture's performance is compared to similar neural network architectures on ten datasets, with competitive results.

cs.LG↗

arXiv2024

An updated look on the convergence and consistency of data-driven dynamical models

Deep sequence models are receiving significant interest in current machine learning research. By representing probability distributions that are fit to data using maximum likelihood estimation, such models can model data on general observation spaces (both continuous and discrete-valued). Furthermore, they can be applied to a wide range of modelling problems, including modelling of dynamical systems which are subject to control. The problem of learning data-driven models of systems subject to control is well studied in the field of system identification. In particular, there exist theoretical convergence and consistency results which can be used to analyze model behaviour and guide model development. However, these results typically concern models which provide point predictions of continuous-valued variables. Motivated by this, we derive convergence and consistency results for a class of nonlinear probabilistic models defined on a general observation space. The results rely on stability and regularity assumptions, and can be used to derive consistency conditions and bias expressions for nonlinear probabilistic models of systems under control. We illustrate the results on examples from linear system identification and Markov chains on finite state spaces.

eess.SY↗

arXiv2024

A deep latent variable model for semi-supervised multi-unit soft sensing in industrial processes

In many industrial processes, an apparent lack of data limits the development of data-driven soft sensors. There are, however, often opportunities to learn stronger models by being more data-efficient. To achieve this, one can leverage knowledge about the data from which the soft sensor is learned. Taking advantage of properties frequently possessed by industrial data, we introduce a deep latent variable model for semi-supervised multi-unit soft sensing. This hierarchical, generative model is able to jointly model different units, as well as learning from both labeled and unlabeled data. An empirical study of multi-unit soft sensing is conducted using two datasets: a synthetic dataset of single-phase fluid flow, and a large, real dataset of multi-phase flow in oil and gas wells. We show that by combining semi-supervised and multi-task learning, the proposed model achieves superior results, outperforming current leading methods for this soft sensing problem. We also show that when a model has been trained on a multi-unit dataset, it may be finetuned to previously unseen units using only a handful of data points. In this finetuning procedure, unlabeled data improve soft sensor performance; remarkably, this is true even when no labeled data are available.

stat.ML↗

arXiv2024

Flow Fusion, Exploiting Measurement Redundancy for Smarter Allocation

In petroleum production systems, continuous multiphase flow rates are essential for efficient operation. They provide situational awareness, enable production optimization, improve reservoir management and planning, and form the basis for allocation. Furthermore, they can be crucial to ensure a fair revenue split between stakeholders for complex production systems where operators share the facilities. Yet, due to complex multiphase flow dynamics and uncertain subsurface fluid properties, the flow rates are challenging to obtain with high accuracy. Consequently, flow rate measurement and estimation solutions, such as multiphase flow meters and virtual flow meters, have different degrees of accuracy and suitability, and impact production decisions and production allocation accordingly. We propose a field-proven, data-driven framework for reconciliation and allocation. With data validation and reconciliation as the theoretical backbone, the solution exploits measurement redundancy to fuse together relevant flow rate information to infer the most likely flow rates in the production system based on quantifiable uncertainties. The framework consists of four modules: data-processing, uncertainty estimation, reconciliation, and gross error detection. The latter, being the focus of this paper, is a means to identify and mitigate the effect of measurements subject to systematic error, which can invalidate the reconciliation. In this paper, we highlight that a combination of statistical tests and supporting logic for gross error detection and elimination can be beneficial in obtaining a more justifiable production allocation. Using the maximum power measurement test, the module can be limited in its ability to pinpoint the erroneous measurement. Yet, it is demonstrated that the detections can be convenient indications of gross errors and where these might reside in the production system.

eess.SP↗

arXiv2023

Sequential Monte Carlo applied to virtual flow meter calibration

Soft-sensors are gaining popularity due to their ability to provide estimates of key process variables with little intervention required on the asset and at a low cost. In oil and gas production, virtual flow metering (VFM) is a popular soft-sensor that attempts to estimate multiphase flow rates in real time. VFMs are based on models, and these models require calibration. The calibration is highly dependent on the application, both due to the great diversity of the models, and in the available measurements. The most accurate calibration is achieved by careful tuning of the VFM parameters to well tests, but this can be work intensive, and not all wells have frequent well test data available. This paper presents a calibration method based on the measurement provided by the production separator, and the assumption that the observed flow should be equal to the sum of flow rates from each individual well. This allows us to jointly calibrate the VFMs continuously. The method applies Sequential Monte Carlo (SMC) to infer a tuning factor and the flow composition for each well. The method is tested on a case with ten wells, using both synthetic and real data. The results are promising and the method is able to provide reasonable estimates of the parameters without relying on well tests. However, some challenges are identified and discussed, particularly related to the process noise and how to manage varying data quality.

cs.LG↗

arXiv2022

Adjustment formulas for learning causal steady-state models from closed-loop operational data

Steady-state models which have been learned from historical operational data may be unfit for model-based optimization unless correlations in the training data which are introduced by control are accounted for. Using recent results from work on structural dynamical causal models, we derive a formula for adjusting for this control confounding, enabling the estimation of a causal steady-state model from closed-loop steady-state data. The formula assumes that the available data have been gathered under some fixed control law. It works by estimating and taking into account the disturbance which the controller is trying to counteract, and enables learning from data gathered under both feedforward and feedback control.

eess.SY↗

arXiv2022

Passive learning to address nonstationarity in virtual flow metering applications

Steady-state process models are common in virtual flow meter applications due to low computational complexity, and low model development and maintenance cost. Nevertheless, the prediction performance of steady-state models typically degrades with time due to the inherent nonstationarity of the underlying process being modeled. Few studies have investigated how learning methods can be applied to sustain the prediction accuracy of steady-state virtual flow meters. This paper explores passive learning, where the model is frequently calibrated to new data, as a way to address nonstationarity and improve long-term performance. An advantage with passive learning is that it is compatible with models used in the industry. Two passive learning methods, periodic batch learning and online learning, are applied with varying calibration frequency to train virtual flow meters. Six different model types, ranging from data-driven to first-principles, are trained on historical production data from 10 petroleum wells. The results are two-fold: first, in the presence of frequently arriving measurements, frequent model updating sustains an excellent prediction performance over time; second, in the presence of intermittent and infrequently arriving measurements, frequent updating in addition to the utilization of expert knowledge is essential to increase the performance accuracy. The investigation may be of interest to experts developing soft-sensors for nonstationary processes, such as virtual flow meters.

eess.SY↗

arXiv2021

On gray-box modeling for virtual flow metering

A virtual flow meter (VFM) enables continuous prediction of flow rates in petroleum production systems. The predicted flow rates may aid the daily control and optimization of a petroleum asset. Gray-box modeling is an approach that combines mechanistic and data-driven modeling. The objective is to create a computationally feasible VFM for use in real-time applications, with high prediction accuracy and scientifically consistent behavior. This article investigates five different gray-box model types in an industrial case study using real, historical production data from 10 petroleum wells, spanning at most four years of production. The results are diverse with an oil flow rate prediction error in the range of 1.8%-40.6%. Further, the study casts light upon the nontrivial task of balancing learning from both physics and data. Consequently, providing general recommendations towards the suitability of different hybrid models is challenging. Nevertheless, the results are promising and indicate that gray-box VFMs may reduce the prediction error of a mechanistic VFM while remaining scientifically consistent. The findings motivate further experimentation with gray-box VFM models and suggest several future research directions to improve upon the performance and scientific consistency.

cs.LG↗

arXiv2021

Multi-task learning for virtual flow metering

Virtual flow metering (VFM) is a cost-effective and non-intrusive technology for inferring multiphase flow rates in petroleum assets. Inferences about flow rates are fundamental to decision support systems that operators extensively rely on. Data-driven VFM, where mechanistic models are replaced with machine learning models, has recently gained attention due to its promise of lower maintenance costs. While excellent performances in small sample studies have been reported in the literature, there is still considerable doubt about the robustness of data-driven VFM. In this paper, we propose a new multi-task learning (MTL) architecture for data-driven VFM. Our method differs from previous methods in that it enables learning across oil and gas wells. We study the method by modeling 55 wells from four petroleum assets and compare the results with two single-task baseline models. Our findings show that MTL improves robustness over single-task methods, without sacrificing performance. MTL yields a 25-50% error reduction on average for the assets where single-task architectures are struggling.

cs.LG↗

arXiv2021

Bayesian Neural Networks for Virtual Flow Metering: An Empirical Study

Recent works have presented promising results from the application of machine learning (ML) to the modeling of flow rates in oil and gas wells. Encouraging results and advantageous properties of ML models, such as computationally cheap evaluation and ease of calibration to new data, have sparked optimism for the development of data-driven virtual flow meters (VFMs). Data-driven VFMs are developed in the small data regime, where it is important to question the uncertainty and robustness of models. The modeling of uncertainty may help to build trust in models, which is a prerequisite for industrial applications. The contribution of this paper is the introduction of a probabilistic VFM based on Bayesian neural networks. Uncertainty in the model and measurements is described, and the paper shows how to perform approximate Bayesian inference using variational inference. The method is studied by modeling on a large and heterogeneous dataset, consisting of 60 wells across five different oil and gas assets. The predictive performance is analyzed on historical and future test data, where an average error of 4-6% and 8-13% is achieved for the 50% best performing models, respectively. Variational inference appears to provide more robust predictions than the reference approach on future data. Prediction performance and uncertainty calibration is explored in detail and discussed in light of four data challenges. The findings motivate the development of alternative strategies to improve the robustness of data-driven VFMs.

cs.LG↗

arXiv2021

Identifiability and physical interpretability of hybrid, gray-box models -- a case study

Model identifiability concerns the uniqueness of uncertain model parameters to be estimated from available process data and is often thought of as a prerequisite for the physical interpretability of a model. Nevertheless, model identifiability may be challenging to obtain in practice due to both stochastic and deterministic uncertainties, e.g. low data variability, noisy measurements, erroneous model structure, and stochasticity and locality of the optimization algorithm. For gray-box, hybrid models, model identifiability is rarely obtainable due to a high number of parameters. We illustrate through an industrial case study - modeling of a production choke valve in a petroleum well - that physical interpretability may be preserved even for non-identifiable models with adequate parameter regularization in the estimation problem. To this end, in a real industrial scenario, it may be beneficial for the model's predictive performance to develop hybrid over mechanistic models, as the model flexibility is higher. Modeling of six petroleum wells on the asset Edvard Grieg using historical production data show a 35\% reduction in the median prediction error across the wells comparing a hybrid to a mechanistic model. On the other hand, both the predictive performance and physical interpretability of the developed models are influenced by the available data. The findings encourage research into online learning and other hybrid model variants to improve the results.

eess.SY↗

arXiv2020

Developing a Hybrid Data-Driven, Mechanistic Virtual Flow Meter -- a Case Study

Virtual flow meters, mathematical models predicting production flow rates in petroleum assets, are useful aids in production monitoring and optimization. Mechanistic models based on first-principles are most common, however, data-driven models exploiting patterns in measurements are gaining popularity. This research investigates a hybrid modeling approach, utilizing techniques from both the aforementioned areas of expertise, to model a well production choke. The choke is represented with a simplified set of first-principle equations and a neural network to estimate the valve flow coefficient. Historical production data from the petroleum platform Edvard Grieg is used for model validation. Additionally, a mechanistic and a data-driven model are constructed for comparison of performance. A practical framework for development of models with varying degree of hybridity and stochastic optimization of its parameters is established. Results of the hybrid model performance are promising albeit with considerable room for improvements.

eess.SY↗

arXiv2019

ReLU Networks as Surrogate Models in Mixed-Integer Linear Programs

We consider the embedding of piecewise-linear deep neural networks (ReLU networks) as surrogate models in mixed-integer linear programming (MILP) problems. A MILP formulation of ReLU networks has recently been applied by many authors to probe for various model properties subject to input bounds. The formulation is obtained by programming each ReLU operator with a binary variable and applying the big-M method. The efficiency of the formulation hinges on the tightness of the bounds defined by the big-M values. When ReLU networks are embedded in a larger optimization problem, the presence of output bounds can be exploited in bound tightening. To this end, we devise and study several bound tightening procedures that consider both input and output bounds. Our numerical results show that bound tightening may reduce solution times considerably, and that small-sized ReLU networks are suitable as surrogate models in mixed-integer linear programs.

math.OC↗