arXiv subjects

Adil Rasheed

Publications and source records attributed to Adil Rasheed.

At least 55 records · Page 3Linked to original sources

arXiv2023

Sparse deep neural networks for modeling aluminum electrolysis dynamics

Deep neural networks have become very popular in modeling complex nonlinear processes due to their extraordinary ability to fit arbitrary nonlinear functions from data with minimal expert intervention. However, they are almost always overparameterized and challenging to interpret due to their internal complexity. Furthermore, the optimization process to find the learned model parameters can be unstable due to the process getting stuck in local minima. In this work, we demonstrate the value of sparse regularization techniques to significantly reduce the model complexity. We demonstrate this for the case of an aluminium extraction process, which is highly nonlinear system with many interrelated subprocesses. We trained a densely connected deep neural network to model the process and then compared the effects of sparsity promoting l1 regularization on generalizability, interpretability, and training stability. We found that the regularization significantly reduces model complexity compared to a corresponding dense neural network. We argue that this makes the model more interpretable, and show that training an ensemble of sparse neural networks with different parameter initializations often converges to similar model structures with similar learned input features. Furthermore, the empirical study shows that the resulting sparse models generalize better from small training sets than their dense counterparts.

physics.chem-ph↗

arXiv2023

PoroTwin: A digital twin for a FluidFlower rig

We present a framework for integrated experiments and simulations of tracer transport in heterogeneous porous media using digital twin technology. The physical asset in our setup is a meter-scale FluidFlower rig. The digital twin consists of a traditional physics-based forward simulation tool and a correction technique which compensates for mismatches between simulation results and observations. The latter augments the range of the physics-based simulation and allows us to bridge the gap between simulation and experiments in a quantitative sense. We describe the setup of the physical and digital twin, including data transfer protocols using cloud technology. The accuracy of the digital twin is demonstrated on a case with artificially high diffusion that must be compensated by the correction approach, as well as by simulations in geologically complex media. The digital twin is then applied to control tracer transport by manipulating fluid injection and production in the experimental rig, thereby enabling two-way coupling between the physical and digital twins.

physics.flu-dyn↗

arXiv2022

Ship Performance Monitoring using Machine-learning

The hydrodynamic performance of a sea-going ship varies over its lifespan due to factors like marine fouling and the condition of the anti-fouling paint system. In order to accurately estimate the power demand and fuel consumption for a planned voyage, it is important to assess the hydrodynamic performance of the ship. The current work uses machine-learning (ML) methods to estimate the hydrodynamic performance of a ship using the onboard recorded in-service data. Three ML methods, NL-PCR, NL-PLSR and probabilistic ANN, are calibrated using the data from two sister ships. The calibrated models are used to extract the varying trend in ship's hydrodynamic performance over time and predict the change in performance through several propeller and hull cleaning events. The predicted change in performance is compared with the corresponding values estimated using the fouling friction coefficient ($ΔC_F$). The ML methods are found to be performing well while modelling the hydrodynamic state variables of the ships with probabilistic ANN model performing the best, but the results from NL-PCR and NL-PLSR are not far behind, indicating that it may be possible to use simple methods to solve such problems with the help of domain knowledge.

stat.ML↗

arXiv2022

Prospects of federated machine learning in fluid dynamics

Physics-based models have been mainstream in fluid dynamics for developing predictive models. In recent years, machine learning has offered a renaissance to the fluid community due to the rapid developments in data science, processing units, neural network based technologies, and sensor adaptations. So far in many applications in fluid dynamics, machine learning approaches have been mostly focused on a standard process that requires centralizing the training data on a designated machine or in a data center. In this letter, we present a federated machine learning approach that enables localized clients to collaboratively learn an aggregated and shared predictive model while keeping all the training data on each edge device. We demonstrate the feasibility and prospects of such decentralized learning approach with an effort to forge a deep learning surrogate model for reconstructing spatiotemporal fields. Our results indicate that federated machine learning might be a viable tool for designing highly accurate predictive decentralized digital twins relevant to fluid dynamics.

cs.LG↗

arXiv2022

Decentralized digital twins of complex dynamical systems

In this paper, we introduce a decentralized digital twin (DDT) framework for dynamical systems and discuss the prospects of the DDT modeling paradigm in computational science and engineering applications. The DDT approach is built on a federated learning concept, a branch of machine learning that encourages knowledge sharing without sharing the actual data. This approach enables clients to collaboratively learn an aggregated model while keeping all the training data on each client. We demonstrate the feasibility of the DDT framework with various dynamical systems, which are often considered prototypes for modeling complex transport phenomena in spatiotemporally extended systems. Our results indicate that federated machine learning might be a key enabler for designing highly accurate decentralized digital twins in complex nonlinear spatiotemporal systems.

cs.LG↗

arXiv2022

Variational multiscale reinforcement learning for discovering reduced order closure models of nonlinear spatiotemporal transport systems

A central challenge in the computational modeling and simulation of a multitude of science applications is to achieve robust and accurate closures for their coarse-grained representations due to underlying highly nonlinear multiscale interactions. These closure models are common in many nonlinear spatiotemporal systems to account for losses due to reduced order representations, including many transport phenomena in fluids. Previous data-driven closure modeling efforts have mostly focused on supervised learning approaches using high fidelity simulation data. On the other hand, reinforcement learning (RL) is a powerful yet relatively uncharted method in spatiotemporally extended systems. In this study, we put forth a modular dynamic closure modeling and discovery framework to stabilize the Galerkin projection based reduced order models that may arise in many nonlinear spatiotemporal dynamical systems with quadratic nonlinearity. However, a key element in creating a robust RL agent is to introduce a feasible reward function, which can be constituted of any difference metrics between the RL model and high fidelity simulation data. First, we introduce a multi-modal RL (MMRL) to discover mode-dependant closure policies that utilize the high fidelity data in rewarding our RL agent. We then formulate a variational multiscale RL (VMRL) approach to discover closure models without requiring access to the high fidelity data in designing the reward function. Specifically, our chief innovation is to leverage variational multiscale formalism to quantify the difference between modal interactions in Galerkin systems. Our results in simulating the viscous Burgers equation indicate that the proposed VMRL method leads to robust and accurate closure parameterizations, and it may potentially be used to discover scale-aware closure models for complex dynamical systems.

cs.LG↗

arXiv2022

Combining physics-based and data-driven techniques for reliable hybrid analysis and modeling using the corrective source term approach

Upcoming technologies like digital twins, autonomous, and artificial intelligent systems involving safety-critical applications require models which are accurate, interpretable, computationally efficient, and generalizable. Unfortunately, the two most commonly used modeling approaches, physics-based modeling (PBM) and data-driven modeling (DDM) fail to satisfy all these requirements. In the current work, we demonstrate how a hybrid approach combining the best of PBM and DDM can result in models which can outperform them both. We do so by combining partial differential equations based on first principles describing partially known physics with a black box DDM, in this case, a deep neural network model compensating for the unknown physics. First, we present a mathematical argument for why this approach should work and then apply the hybrid approach to model two dimensional heat diffusion problem with an unknown source term. The result demonstrates the method's superior performance in terms of accuracy, and generalizability. Additionally, it is shown how the DDM part can be interpreted within the hybrid framework to make the overall approach reliable.

cs.LG↗

arXiv2022

Physics Guided Machine Learning for Variational Multiscale Reduced Order Modeling

We propose a new physics guided machine learning (PGML) paradigm that leverages the variational multiscale (VMS) framework and available data to dramatically increase the accuracy of reduced order models (ROMs) at a modest computational cost. The hierarchical structure of the ROM basis and the VMS framework enable a natural separation of the resolved and unresolved ROM spatial scales. Modern PGML algorithms are used to construct novel models for the interaction among the resolved and unresolved ROM scales. Specifically, the new framework builds ROM operators that are closest to the true interaction terms in the VMS framework. Finally, machine learning is used to reduce the projection error and further increase the ROM accuracy. Our numerical experiments for a two-dimensional vorticity transport problem show that the novel PGML-VMS-ROM paradigm maintains the low computational cost of current ROMs, while significantly increasing the ROM accuracy.

physics.flu-dyn↗

arXiv2022

Physics guided neural networks for modelling of non-linear dynamics

The success of the current wave of artificial intelligence can be partly attributed to deep neural networks, which have proven to be very effective in learning complex patterns from large datasets with minimal human intervention. However, it is difficult to train these models on complex dynamical systems from data alone due to their low data efficiency and sensitivity to hyperparameters and initialisation. This work demonstrates that injection of partially known information at an intermediate layer in a DNN can improve model accuracy, reduce model uncertainty, and yield improved convergence during the training. The value of these physics-guided neural networks has been demonstrated by learning the dynamics of a wide variety of nonlinear dynamical systems represented by five well-known equations in nonlinear systems theory: the Lotka-Volterra, Duffing, Van der Pol, Lorenz, and Henon-Heiles systems.

cs.LG↗

arXiv2022

Data Processing Framework for Ship Performance Analysis

The hydrodynamic performance of a sea-going ship can be analysed using the data obtained from the ship. Such data can be gathered from different sources, like onboard recorded in-service data, AIS data, and noon reports. Each of these sources is known to have their inherent problems. The current work gives a brief introduction to these data sources as well as the common problems associated with them, along with some examples. In order to resolve most of these problems, a streamlined semi-automatic data processing framework for fast data processing is developed and presented here. The data processing framework can be used to process the data obtained from any of the above three mentioned sources. The framework incorporates processing steps like interpolating weather hindcast (metocean) data to ship's location in time, deriving additional features, validating data, estimating resistance components, data cleaning, and outlier detection. A brief description of each of the processing steps is provided with examples from existing datasets. The processed data can be further used to analyse the hydrodynamic performance of a ship.

cs.DB↗

arXiv2022

Frame invariant neural network closures for Kraichnan turbulence

Numerical simulations of geophysical and atmospheric flows have to rely on parameterizations of subgrid scale processes due to their limited spatial resolution. Despite substantial progress in developing parameterization (or closure) models for subgrid scale (SGS) processes using physical insights and mathematical approximations, they remain imperfect and can lead to inaccurate predictions. In recent years, machine learning has been successful in extracting complex patterns from high-resolution spatio-temporal data, leading to improved parameterization models, and ultimately better coarse grid prediction. However, the inability to satisfy known physics and poor generalization hinders the application of these models for real-world problems. In this work, we propose a frame invariant closure approach to improve the accuracy and generalizability of deep learning-based subgrid scale closure models by embedding physical symmetries directly into the structure of the neural network. Specifically, we utilized specialized layers within the convolutional neural network in such a way that desired constraints are theoretically guaranteed without the need for any regularization terms. We demonstrate our framework for a two-dimensional decaying turbulence test case mostly characterized by the forward enstrophy cascade. We show that our frame invariant SGS model (i) accurately predicts the subgrid scale source term, (ii) respects the physical symmetries such as translation, Galilean, and rotation invariance, and (iii) is numerically stable when implemented in coarse-grid simulation with generalization to different initial conditions and Reynolds number. This work builds a bridge between extensive physics-based theories and data-driven modeling paradigms, and thus represents a promising step towards the development of physically consistent data-driven turbulence closure models.

physics.flu-dyn↗

arXiv2021

Deep neural network enabled corrective source term approach to hybrid analysis and modeling

In this work, we introduce, justify and demonstrate the Corrective Source Term Approach (CoSTA) -- a novel approach to Hybrid Analysis and Modeling (HAM). The objective of HAM is to combine physics-based modeling (PBM) and data-driven modeling (DDM) to create generalizable, trustworthy, accurate, computationally efficient and self-evolving models. CoSTA achieves this objective by augmenting the governing equation of a PBM model with a corrective source term generated using a deep neural network. In a series of numerical experiments on one-dimensional heat diffusion, CoSTA is found to outperform comparable DDM and PBM models in terms of accuracy -- often reducing predictive errors by several orders of magnitude -- while also generalizing better than pure DDM. Due to its flexible but solid theoretical foundation, CoSTA provides a modular framework for leveraging novel developments within both PBM and DDM. Its theoretical foundation also ensures that CoSTA can be used to model any system governed by (deterministic) partial differential equations. Moreover, CoSTA facilitates interpretation of the DNN-generated source term within the context of PBM, which results in improved explainability of the DNN. These factors make CoSTA a potential door-opener for data-driven techniques to enter high-stakes applications previously reserved for pure PBM.

cs.NE↗

arXiv2021

Nonlinear proper orthogonal decomposition for convection-dominated flows

Autoencoder techniques find increasingly common use in reduced order modeling as a means to create a latent space. This reduced order representation offers a modular data-driven modeling approach for nonlinear dynamical systems when integrated with a time series predictive model. In this letter, we put forth a nonlinear proper orthogonal decomposition (POD) framework, which is an end-to-end Galerkin-free model combining autoencoders with long short-term memory networks for dynamics. By eliminating the projection error due to the truncation of Galerkin models, a key enabler of the proposed nonintrusive approach is the kinematic construction of a nonlinear mapping between the full-rank expansion of the POD coefficients and the latent space where the dynamics evolve. We test our framework for model reduction of a convection-dominated system, which is generally challenging for reduced order models. Our approach not only improves the accuracy, but also significantly reduces the computational cost of training and testing.

physics.flu-dyn↗

arXiv2021

Multi-fidelity information fusion with concatenated neural networks

Recently, computational modeling has shifted towards the use of deep learning, and other data-driven modeling frameworks. Although this shift in modeling holds promise in many applications like design optimization and real-time control by lowering the computational burden, training deep learning models needs a huge amount of data. This big data is not always available for scientific problems and leads to poorly generalizable data-driven models. This gap can be furnished by leveraging information from physics-based models. Exploiting prior knowledge about the problem at hand, this study puts forth a concatenated neural network approach to build more tailored, effective, and efficient machine learning models. For our analysis, without losing its generalizability and modularity, we focus on the development of predictive models for laminar and turbulent boundary layer flows. In particular, we combine the self-similarity solution and power-law velocity profile (low-fidelity models) with the noisy data obtained either from experiments or computational fluid dynamics simulations (high-fidelity models) through a concatenated neural network. We illustrate how the knowledge from these simplified models results in reducing uncertainties associated with deep learning models. The proposed framework produces physically consistent models that attempt to achieve better generalization than data-driven models obtained purely based on data. While we demonstrate our framework for a problem relevant to fluid mechanics, its workflow and principles can be adopted for many scientific problems where empirical models are prevalent. In line with grand demands in novel physics-guided machine learning principles, this work builds a bridge between extensive physics-based theories and data-driven modeling paradigms and paves the way for using hybrid modeling approaches for next-generation digital twin technologies.

physics.flu-dyn↗

arXiv2021

On closures for reduced order models $-$ A spectrum of first-principle to machine-learned avenues

For over a century, reduced order models (ROMs) have been a fundamental discipline of theoretical fluid mechanics. Early examples include Galerkin models inspired by the Orr-Sommerfeld stability equation and numerous vortex models, of which the von Kármán vortex street is one of the most prominent. Subsequent ROMs typically relied on first principles, like mathematical Galerkin models, weakly nonlinear stability theory, and two- and three-dimensional vortex models. Aubry et al. [N. Aubry, P. Holmes, J. Lumley, and E. Stone, Journal of Fluid Mechanics, 192, 115-173 (1988)] pioneered data-driven proper orthogonal decomposition (POD) modeling. In early POD modeling, available data was used to build an optimal basis, which was then utilized in a classical Galerkin procedure to construct the ROM. But data has made a profound impact on ROMs beyond the Galerkin expansion. In this paper, we take a modest step and illustrate the impact of data-driven modeling on one significant ROM area. Specifically, we focus on ROM closures, which are correction terms that are added to classical ROMs in order to model the effect of the discarded ROM modes in under-resolved simulations. Through simple examples, we illustrate the main modeling principles used to construct classical ROMs, motivate and introduce modern ROM closures, and show how data-driven modeling, artificial intelligence, and machine learning have changed the standard ROM methodology over the last two decades. Finally, we outline our vision on how state-of-the-art data-driven modeling can continue to reshape the field of reduced order modeling.

physics.flu-dyn↗

arXiv2021

A nonintrusive hybrid neural-physics modeling of incomplete dynamical systems: Lorenz equations

This work presents a hybrid modeling approach to data-driven learning and representation of unknown physical processes and closure parameterizations. These hybrid models are suitable for situations where the mechanistic description of dynamics of some variables is unknown, but reasonably accurate observational data can be obtained for the evolution of the state of the system. In this work, we propose machine learning to account for missing physics and then data assimilation to correct the prediction. In particular, we devise an effective methodology based on a recurrent neural network to model the unknown dynamics. A long short-term memory (LSTM) based correction term is added to the predictive model in order to take into account hidden physics. Since LSTM introduces a black-box approach for the unknown part of the model, we investigate whether the proposed hybrid neural-physical model can be further corrected through a sequential data assimilation step. We apply this framework to the weakly nonlinear Lorenz model that displays quasiperiodic oscillations, the highly nonlinear Lorenz model, and two-scale Lorenz model. The hybrid neural-physics model yields accurate results for the weakly nonlinear Lorenz model with the predicted state close to the true Lorenz model trajectory. For the highly nonlinear Lorenz model and the two-scale Lorenz model, the hybrid neural-physics model deviates from the true state due to the accumulation of prediction error from one time step to the next time step. The ensemble Kalman filter approach takes into account the prediction error and updates the diverged prediction using available observations in order to provide a more accurate state estimate. The successful synergistic integration of neural network and data assimilation for low-dimensional system shows the potential benefits of the proposed hybrid-neural physics model for complex systems.

physics.comp-ph↗

arXiv2021

Model fusion with physics-guided machine learning

The unprecedented amount of data generated from experiments, field observations, and large-scale numerical simulations at a wide range of spatio-temporal scales have enabled the rapid advancement of data-driven and especially deep learning models in the field of fluid mechanics. Although these methods are proven successful for many applications, there is a grand challenge of improving their generalizability. This is particularly essential when data-driven models are employed within outer-loop applications like optimization. In this work, we put forth a physics-guided machine learning (PGML) framework that leverages the interpretable physics-based model with a deep learning model. The PGML framework is capable of enhancing the generalizability of data-driven models and effectively protect against or inform about the inaccurate predictions resulting from extrapolation. We apply the PGML framework as a novel model fusion approach combining the physics-based Galerkin projection model and long-short term memory (LSTM) network for parametric model order reduction of fluid flows. We demonstrate the improved generalizability of the PGML framework against a purely data-driven approach through the injection of physics features into intermediate LSTM layers. Our quantitative analysis shows that the overall model uncertainty can be reduced through the PGML approach especially for test data coming from a distribution different than the training data. Moreover, we demonstrate that our approach can be used as an inverse diagnostic tool providing a confidence score associated with models and observations. The proposed framework also allows for multi-fidelity computing by making use of low-fidelity models in the online deployment of quantified data-driven models.

physics.comp-ph↗

arXiv2021

Hybrid analysis and modeling, eclecticism, and multifidelity computing toward digital twin revolution

Most modeling approaches lie in either of the two categories: physics-based or data-driven. Recently, a third approach which is a combination of these deterministic and statistical models is emerging for scientific applications. To leverage these developments, our aim in this perspective paper is centered around exploring numerous principle concepts to address the challenges of (i) trustworthiness and generalizability in developing data-driven models to shed light on understanding the fundamental trade-offs in their accuracy and efficiency, and (ii) seamless integration of interface learning and multifidelity coupling approaches that transfer and represent information between different entities, particularly when different scales are governed by different physics, each operating on a different level of abstraction. Addressing these challenges could enable the revolution of digital twin technologies for scientific and engineering applications.

physics.comp-ph↗