arXiv subjects

Akhil Ahmed

Publications and source records attributed to Akhil Ahmed.

6 recordsLinked to original sources

arXiv2024

PC-Gym: Benchmark Environments For Process Control Problems

PC-Gym is an open-source tool for developing and evaluating reinforcement learning (RL) algorithms in chemical process control. It features environments that simulate various chemical processes, incorporating nonlinear dynamics, disturbances, and constraints. The tool includes customizable constraint handling, disturbance generation, reward function design, and enables comparison of RL algorithms against Nonlinear Model Predictive Control (NMPC) across different scenarios. Case studies demonstrate the framework's effectiveness in evaluating RL approaches for systems like continuously stirred tank reactors, multistage extraction processes, and crystallization reactors. The results reveal performance gaps between RL algorithms and NMPC oracles, highlighting areas for improvement and enabling benchmarking. By providing a standardized platform, PC-Gym aims to accelerate research at the intersection of machine learning, control, and process systems engineering. By connecting theoretical RL advances with practical industrial process control applications, offering researchers a tool for exploring data-driven control solutions.

eess.SY↗

arXiv2024

Control-Informed Reinforcement Learning for Chemical Processes

This work proposes a control-informed reinforcement learning (CIRL) framework that integrates proportional-integral-derivative (PID) control components into the architecture of deep reinforcement learning (RL) policies. The proposed approach augments deep RL agents with a PID controller layer, incorporating prior knowledge from control theory into the learning process. CIRL improves performance and robustness by combining the best of both worlds: the disturbance-rejection and setpoint-tracking capabilities of PID control and the nonlinear modeling capacity of deep RL. Simulation studies conducted on a continuously stirred tank reactor system demonstrate the improved performance of CIRL compared to both conventional model-free deep RL and static PID controllers. CIRL exhibits better setpoint-tracking ability, particularly when generalizing to trajectories outside the training distribution, suggesting enhanced generalization capabilities. Furthermore, the embedded prior control knowledge within the CIRL policy improves its robustness to unobserved system disturbances. The control-informed RL framework combines the strengths of classical control and reinforcement learning to develop sample-efficient and robust deep reinforcement learning algorithms, with potential applications in complex industrial systems.

eess.SY↗

arXiv2024

ARRTOC: Adversarially Robust Real-Time Optimization and Control

Real-Time Optimization (RTO) plays a crucial role in the process operation hierarchy by determining optimal set-points for the lower-level controllers. However, at the control layer, these set-points may be difficult to track due to challenges in implementation as a result of disturbances, measurement noise, and actuator performance limitations. To address this, in this paper, we present the Adversarially Robust Real-Time Optimization and Control (ARRTOC) algorithm. ARRTOC addresses this issue by finding set-points which are both optimal and inherently robust to implementation errors at the control layers. ARRTOC draws inspiration from adversarial machine learning, offering a novel constrained Adversarially Robust Optimization (ARO) solution applied to the RTO layer. By integrating controller design with RTO, ARRTOC enhances overall system performance and robustness by ensuring the chosen set-points are tailored to the underlying controller designs. To validate our claims, we present three case studies: an illustrative example, a bioreactor case study, and a multi-loop evaporator process. The proposed approach is found to improve RTO objectives, such as profit, by as much as $50\%$ in some case studies compared to RTO formulations which ignore the performance of the control layers.

eess.SY↗

arXiv2023

Exploring the Dynamics of the Specialty Insurance Market Using a Novel Discrete Event Simulation Framework: a Lloyd's of London Case Study

This research presents a novel Discrete Event Simulation (DES) of the Lloyd's of London specialty insurance market, exploring complex market dynamics that have not been previously studied quantitatively. The proof-of-concept model allows for the simulation of various scenarios that capture important market phenomena such as the underwriting cycle, the impact of risk syndication, and the importance of appropriate exposure management. Despite minimal calibration, our model has shown that it is a valuable tool for understanding and analysing the Lloyd's of London specialty insurance market, particularly in terms of identifying areas for further investigation for regulators and participants of the market alike. The results generate the expected behaviours that, syndicates (insurers) are less likely to go insolvent if they adopt sophisticated exposure management practices, catastrophe events lead to more defined patterns of cyclicality and cause syndicates to substantially increase their premiums offered. Lastly, syndication enhances the accuracy of actuarial price estimates and narrows the divergence among syndicates. Overall, this research offers a new perspective on the Lloyd's of London market and demonstrates the potential of individual-based modelling (IBM) for understanding complex financial systems.

q-fin.GN↗

arXiv2022

Learning Linear Representations of Nonlinear Dynamics Using Deep Learning

The vast majority of systems of practical interest are characterised by nonlinear dynamics. This renders the control and optimization of such systems a complex task due to their nonlinear behaviour. Additionally, standard methods such as linearizing around a fixed point may not be an effective strategy for many systems, thus requiring an alternative approach. For this reason, we propose a new deep learning framework to discover a transformation of a nonlinear dynamical system to an equivalent higher dimensional linear representation. We demonstrate that the resulting learned linear representation accurately captures the dynamics of the original system for a wider range of conditions than standard linearization. As a result of this, we show that the learned linear model can subsequently be used for the successful control of the original system. We demonstrate this by applying the proposed framework to two examples; one from the literature and a more complex example in the form of a Continuous Stirred Tank Reactor (CSTR).

eess.SY↗

arXiv2021

Application of Gaussian Processes to online approximation of compressor maps for load-sharing in a compressor station

Devising optimal operating strategies for a compressor station relies on the knowledge of compressor characteristics. As the compressor characteristics change with time and use, it is necessary to provide accurate models of the characteristics that can be used in optimization of the operating strategy. This paper proposes a new algorithm for online learning of the characteristics of the compressors using Gaussian Processes. The performance of the new approximation is shown in a case study with three compressors. The case study shows that Gaussian Processes accurately capture the characteristics of compressors even if no knowledge about the characteristics is initially available. The results show that the flexible nature of Gaussian Processes allows them to adapt to the data online making them amenable for use in real-time optimization problems.

cs.CE↗