arXiv subjects

Subhrakanti Dey

Publications and source records attributed to Subhrakanti Dey.

At least 37 records · Page 2Linked to original sources

arXiv2023

Deterministic policy gradient based optimal control with probabilistic constraints

This paper studies a deep deterministic policy gradient (DDPG) based actor critic (AC) reinforcement learning (RL) technique to control a linear discrete-time system with a quadratic control cost while ensuring a constraint on the probability of potentially risky or undesirable events. The proposed methodology can be applied to both known and unknown system models with minor adjustments to the reward structure (negative cost). The problem is formulated by considering the average expected quadratic cost of the states and inputs over an infinite time horizon. Risky or undesirable events are represented as functions of the states at the next time step exceeding a user-defined limit. Two strategies are employed to manage the probabilistic constraint in scenarios of known and unknown system models. In the case of a known system model, the probabilistic constraint is replaced with an upper bound, such as the Chernoff bound. For unknown system models, the expected value of the indicator function of the occurrence of the risky or undesirable event is used. We have adopted a deterministic policy gradient (DPG) based AC method to derive a parameterised optimal policy. Extensive numerical simulations are performed using a second- and a fourth-order system, and the proposed method is compared with the standard risk-neutral linear quadratic regulator (LQR) and a chance-constrained model predictive control (MPC) method. The results demonstrate the effectiveness of the proposed approach in both known and unknown system model scenarios.

eess.SY↗

arXiv2023

Output Feedback Reinforcement Learning with Parameter Optimisation for Temperature Control in a Material Extrusion Additive Manufacturing system

With the rapid development of Additive Manufacturing (AM) comes an urgent need for advanced monitoring and control of the process. Many aspects of the AM process play a significant role in the efficiency, accuracy and repeatability of the process, with temperature regulation being one of the most important ones. In this work, we solve the problem of optimal tracking control for a state space temperature model of a Big Area Additive Manufacturing (BAAM) system. In particular, we address the problem of designing a Linear Quadratic Tracking (LQT) controller when access to the exact system state is not possible, except in the form of measurements. We initially solve the problem with a model-based approach based on reinforcement learning concepts, with state estimation through an observer. We then design a model-free reinforcement-learning based controller with an internal state estimation step and demonstrate its performance through a simulator of the systems' behaviour. Our results showcase the possibility of achieving comparable results while learning optimal policies directly from process data, without the need for an accurate, intricate model of the process. We consider this outcome to be a significant stride towards autonomous intelligent manufacturing.

math.OC↗

arXiv2023

FedZeN: Towards superlinear zeroth-order federated learning via incremental Hessian estimation

Federated learning is a distributed learning framework that allows a set of clients to collaboratively train a model under the orchestration of a central server, without sharing raw data samples. Although in many practical scenarios the derivatives of the objective function are not available, only few works have considered the federated zeroth-order setting, in which functions can only be accessed through a budgeted number of point evaluations. In this work we focus on convex optimization and design the first federated zeroth-order algorithm to estimate the curvature of the global objective, with the purpose of achieving superlinear convergence. We take an incremental Hessian estimator whose error norm converges linearly, and we adapt it to the federated zeroth-order setting, sampling the random search directions from the Stiefel manifold for improved performance. In particular, both the gradient and Hessian estimators are built at the central server in a communication-efficient and privacy-preserving way by leveraging synchronized pseudo-random number generators. We provide a theoretical analysis of our algorithm, named FedZeN, proving local quadratic convergence with high probability and global linear convergence up to zeroth-order precision. Numerical simulations confirm the superlinear convergence rate and show that our algorithm outperforms the federated zeroth-order methods available in the literature.

cs.LG↗

arXiv2023

Network-GIANT: Fully distributed Newton-type optimization via harmonic Hessian consensus

This paper considers the problem of distributed multi-agent learning, where the global aim is to minimize a sum of local objective (empirical loss) functions through local optimization and information exchange between neighbouring nodes. We introduce a Newton-type fully distributed optimization algorithm, Network-GIANT, which is based on GIANT, a Federated learning algorithm that relies on a centralized parameter server. The Network-GIANT algorithm is designed via a combination of gradient-tracking and a Newton-type iterative algorithm at each node with consensus based averaging of local gradient and Newton updates. We prove that our algorithm guarantees semi-global and exponential convergence to the exact solution over the network assuming strongly convex and smooth loss functions. We provide empirical evidence of the superior convergence performance of Network-GIANT over other state-of-art distributed learning algorithms such as Network-DANE and Newton-Raphson Consensus.

math.OC↗

arXiv2023

Data-driven Linear Quadratic Tracking based Temperature Control of a Big Area Additive Manufacturing System

Designing efficient closed-loop control algorithms is a key issue in Additive Manufacturing (AM), as various aspects of the AM process require continuous monitoring and regulation, with temperature being a particularly significant factor. Here we study closed-loop control of a state space temperature model with a focus on both model-based and data-driven methods. We demonstrate these approaches using a simulator of the temperature evolution in the extruder of a Big Area Additive Manufacturing system (BAAM). We perform an in-depth comparison of the performance of these methods using the simulator. We find that we can learn an effective controller using solely simulated process data. Our approach achieves parity in performance compared to model-based controllers and so lessens the need for estimating a large number of parameters of the intricate and complicated process model. We believe this result is an important step towards autonomous intelligent manufacturing.

math.OC↗

arXiv2023

Q-SHED: Distributed Optimization at the Edge via Hessian Eigenvectors Quantization

Edge networks call for communication efficient (low overhead) and robust distributed optimization (DO) algorithms. These are, in fact, desirable qualities for DO frameworks, such as federated edge learning techniques, in the presence of data and system heterogeneity, and in scenarios where internode communication is the main bottleneck. Although computationally demanding, Newton-type (NT) methods have been recently advocated as enablers of robust convergence rates in challenging DO problems where edge devices have sufficient computational power. Along these lines, in this work we propose Q-SHED, an original NT algorithm for DO featuring a novel bit-allocation scheme based on incremental Hessian eigenvectors quantization. The proposed technique is integrated with the recent SHED algorithm, from which it inherits appealing features like the small number of required Hessian computations, while being bandwidth-versatile at a bit-resolution level. Our empirical evaluation against competing approaches shows that Q-SHED can reduce by up to 60% the number of communication rounds required for convergence.

eess.SY↗

arXiv2023

Quickest Detection of Deception Attacks on Cyber-Physical Systems with a Parsimonious Watermarking Policy

The addition of a physical watermarking signal to the control input increases the detection probability of data deception attacks at the expense of increased control cost. In this paper, we propose a parsimonious policy to reduce the average number of watermarking events when the attack is not present, which in turn reduces the control cost. We model the system as a stochastic optimal control problem and apply the dynamic programming to minimize the average detection delay (ADD) for fixed upper bounds on false alarm rate (FAR) and increased control cost. The optimal solution results in a two threshold policy on the posterior probability of attack, which is derived from the Shiryaev statistics for sequential change detection assuming the change point is a random variable with a geometric distribution. We derive approximate expressions of ADD and FAR applying the non-linear renewal theory. The relationship between the average number of watermarking added before the attack and the increase in control cost is also derived. We design the optimal watermarking that maximizes the Kullback-Leibler divergence for a fixed increase in the control cost. Simulation studies are performed to illustrate and validate the theoretical results.

math.OC↗

arXiv2023

LQG Control Over SWIPT-enabled Wireless Communication Network

In this paper, we consider using simultaneous wireless information and power transfer (SWIPT) to recharge the sensor in the LQG control, which provides a new approach to prolonging the network lifetime. We analyze the stability of the proposed system model and show that there exist two critical values for the power splitting ratio α. Then, we propose an optimization problem to derive the optimal value of α. This problem is non-convex but its numerical solution can be derived by our proposed algorithm efficiently. Moreover, we provide the feasible condition of the proposed optimization problem. Finally, simulation results are presented to verify and illustrate the main theoretical results.

eess.SY↗

arXiv2022

Optimal age-specific vaccination control for COVID-19: an Irish case study

The outbreak of a novel coronavirus causing severe acute respiratory syndrome in December 2019 has escalated into a worldwide pandemic. In this work, we propose a compartmental model to describe the dynamics of transmission of infection and use it to obtain the optimal vaccination control. The model accounts for the various stages of the vaccination and the optimisation is focused on minimising the infections to protect the population and relieve the healthcare system. As a case study we selected the Republic of Ireland. We use data provided by Ireland's COVID-19 Data-Hub and simulate the evolution of the pandemic with and without the vaccination in place for two different scenarios, one representative of a national lockdown situation and the other indicating looser restrictions in place. One of the main findings of our work is that the optimal approach would involve a vaccination programme where the older population is vaccinated in larger numbers earlier while simultaneously part of the younger population also gets vaccinated to lower the risk of transmission between groups. We compare our simulated results with that of the vaccination policy taken by the Irish government to explore the advantages of our optimisation method. Our comparison suggests that a similar reduction in cases may have been possible even with a reduced set of vaccinations being available for use.

math.OC↗

arXiv2022

SHED: A Newton-type algorithm for federated learning based on incremental Hessian eigenvector sharing

There is a growing interest in the distributed optimization framework that goes under the name of Federated Learning (FL). In particular, much attention is being turned to FL scenarios where the network is strongly heterogeneous in terms of communication resources (e.g., bandwidth) and data distribution. In these cases, communication between local machines (agents) and the central server (Master) is a main consideration. In this work, we present SHED, an original communication-constrained Newton-type (NT) algorithm designed to accelerate FL in such heterogeneous scenarios. SHED is by design robust to non i.i.d. data distributions, handles heterogeneity of agents' communication resources (CRs), only requires sporadic Hessian computations, and achieves super-linear convergence. This is possible thanks to an incremental strategy, based on eigendecomposition of the local Hessian matrices, which exploits (possibly) outdated second-order information. The proposed solution is thoroughly validated on real datasets by assessing (i) the number of communication rounds required for convergence, (ii) the overall amount of data transmitted and (iii) the number of local Hessian computations. For all these metrics, the proposed approach shows superior performance against state-of-the art techniques like GIANT and FedNL.

cs.LG↗

arXiv2022

A Differential Private Method for Distributed Optimization in Directed Networks via State Decomposition

In this paper, we study the problem of consensus-based distributed optimization where a network of agents, abstracted as a directed graph, aims to minimize the sum of all agents' cost functions collaboratively. In existing distributed optimization approaches (Push-Pull/AB) for directed graphs, all agents exchange their states with neighbors to achieve the optimal solution with a constant stepsize, which may lead to the disclosure of sensitive and private information. For privacy preservation, we propose a novel state-decomposition based gradient tracking approach (SD-Push-Pull) for distributed optimzation over directed networks that preserves differential privacy, which is a strong notion that protects agents' privacy against an adversary with arbitrary auxiliary information. The main idea of the proposed approach is to decompose the gradient state of each agent into two sub-states. Only one substate is exchanged by the agent with its neighbours over time, and the other one is kept private. That is to say, only one substate is visible to an adversary, protecting the privacy from being leaked. It is proved that under certain decomposition principles, a bound for the sub-optimality of the proposed algorithm can be derived and the differential privacy is achieved simultaneously. Moreover, the trade-off between differential privacy and the optimization accuracy is also characterized. Finally, a numerical simulation is provided to illustrate the effectiveness of the proposed approach.

eess.SY↗

arXiv2022

Transmission Power Allocation for Remote Estimation with Multi-packet Reception Capabilities

In this paper we consider the problem of transmission power allocation for remote estimation of a dynamical system in the case where the estimator is able to simultaneously receive packets from multiple interfering sensors, as it is possible e.g. with the latest wireless technologies such as 5G and WiFi. To this end we introduce a general model where packet arrival probabilities are determined based on the received Signal-to-Interference-and-Noise Ratio and with two different receivers design schemes, one implementing standard multi-packet reception technique and one implementing Successive Interference Cancellation decoding algorithm in addition. Then we cast the power allocation problem as an optimization task where the mean error covariance at the remote estimator is minimized, while penalizing the mean transmission power consumption. For the infinite-horizon problem we show the existence of a stationary optimal policy, while for the finite-horizon case we derive some structural properties under the special scenario where the overall system to be estimated can be seen as a set of independent subsystems. Numerical simulations illustrate the improvement given by the proposed receivers over orthogonal schemes that schedules only one sensor transmission at a time in order to avoid interference.

eess.SY↗

arXiv2021

Privacy-Preserving Push-sum Average Consensus via State Decomposition

Average consensus is extensively used in distributed networks for computation and control, where all the agents constantly communicate with each other and update their states in order to reach an agreement. Under a general average consensus algorithm, information exchanged through wireless or wired communication networks could lead to the disclosure of sensitive and private information. In this paper, we propose a privacy-preserving push-sum approach for directed networks that can protect the privacy of all agents while achieving average consensus simultaneously. Each node decomposes its initial state arbitrarily into two substates, and their average equals to the initial state, guaranteeing that the agent's state will converge to the accurate average consensus. Only one substate is exchanged by the node with its neighbours over time, and the other one is reserved. That is to say, only the exchanged substate would be visible to an adversary, preventing the initial state information from leakage. Different from the existing state-decomposition approach which only applies to undirected graphs, our proposed approach is applicable to strongly connected digraphs. In addition, in direct contrast to offset-adding based privacy-preserving push-sum algorithm, which is vulnerable to an external eavesdropper, our proposed approach can ensure privacy against both an honest-but-curious node and an external eavesdropper. A numerical simulation is provided to illustrate the effectiveness of the proposed approach.

eess.SY↗

arXiv2021

Quickest Detection of Deception Attacks in Networked Control Systems with Physical Watermarking

In this paper, we propose and analyze an attack detection scheme for securing the physical layer of a networked control system against attacks where the adversary replaces the true observations with stationary false data. An independent and identically distributed watermarking signal is added to the optimal linear quadratic Gaussian (LQG) control inputs, and a cumulative sum (CUSUM) test is carried out using the joint distribution of the innovation signal and the watermarking signal for quickest attack detection. We derive the expressions of the supremum of the average detection delay (SADD) for a multi-input and multi-output (MIMO) system under the optimal and sub-optimal CUSUM tests. The SADD is asymptotically inversely proportional to the expected Kullback-Leibler divergence (KLD) under certain conditions. The expressions for the MIMO case are simplified for multi-input and single-output systems and explored further to distil design insights. We provide insights into the design of an optimal watermarking signal to maximize KLD for a given fixed increase in LQG control cost when there is no attack. Furthermore, we investigate how the attacker and the control system designer can accomplish their respective objectives by changing the relative power of the attack signal and the watermarking signal. Simulations and numerical studies are carried out to validate the theoretical results.

math.OC↗

arXiv2020

Sequential detection of Replay attacks

One of the most studied forms of attacks on the cyber-physical systems is the replay attack. The statistical similarities of the replay signal and the true observations make the replay attack difficult to detect. In this paper, we have addressed the problem of replay attack detection by adding watermarking to the control inputs and then performed resilient detection using cumulative sum (CUSUM) test on the joint statistics of the innovation signal and the watermarking signal. We derive the expression of the Kullback-Liebler divergence (KLD) between the two joint distributions before and after the replay attack, which is asymptotically inversely proportional to the detection delay. We perform structural analysis of the derived KLD expression and suggest a technique to improve the KLD for the systems with relative degree greater than one. A scheme to find the optimal watermarking signal variance for a fixed increase in the control cost to maximize the KLD under the CUSUM test is presented. We provide various numerical simulation results to support our theory. The proposed method is also compared with a state-of-the-art method.

math.OC↗

arXiv2020

Stealthy hacking and secrecy of controlled state estimation systems with random dropouts

We study the maximum information gain that an adversary may obtain through hacking without being detected. Consider a dynamical process observed by a sensor that transmits a local estimate of the system state to a remote estimator according to some reference transmission policy across a packet-dropping wireless channel equipped with acknowledgments (ACK). An adversary overhears the transmissions and proactively hijacks the sensor to reprogram its transmission policy. We define perfect secrecy as keeping the averaged expected error covariance bounded at the legitimate estimator and unbounded at the adversary. By analyzing the stationary distribution of the expected error covariance, we show that perfect secrecy can be attained for unstable systems only if the ACK channel has no packet dropouts. In other situations, we prove that independent of the reference policy and the detection methods, perfect secrecy is not attainable. For this scenario, we formulate a constrained Markov decision process to derive the optimal transmission policy that the adversary should implement at the sensor, and devise a Stackelberg game to derive the optimal reference policy for the legitimate estimator.

eess.SY↗

arXiv2020

A Game-Theoretic Approach to Covert Communications

This paper considers a game-theoretic formulation of the covert communications problem with finite blocklength, where the transmitter (Alice) can randomly vary her transmit power in different blocks, while the warden (Willie) can randomly vary his detection threshold in different blocks. In this two player game, the payoff for Alice is a combination of the coding rate to the receiver (Bob) and the detection error probability at Willie, while the payoff for Willie is the negative of his detection error probability. Nash equilibrium solutions to the game are obtained, and shown to be efficiently computable using linear programming. For less covert requirements, our game-theoretic approach can achieve significantly higher coding rates than uniformly distributed transmit powers. We then consider the situation with an additional jammer, where Alice and the jammer can both vary their powers. We pose a two player game where Alice and the jammer jointly comprise one player, with Willie the other player. The use of a jammer is shown in numerical simulations to lead to further significant performance improvements.

cs.IT↗

arXiv2020

Asymptotic Performance Analysis of Non-Bayesian Quickest Change Detection with an Energy Harvesting Sensor

In this paper, we consider a non-Bayesian sequential change detection based on the Cumulative Sum (CUSUM) algorithm employed by an energy harvesting sensor where the distributions before and after the change are assumed to be known. In a slotted discrete-time model, the sensor, exclusively powered by randomly available harvested energy, obtains a sample and computes the log-likelihood ratio of the two distributions if it has enough energy to sense and process a sample. If it does not have enough energy in a given slot, it waits until it harvests enough energy to perform the task in a future time slot. We derive asymptotic expressions for the expected detection delay (when a change actually occurs), and the asymptotic tail distribution of the run-length to a false alarm (when a change never happens). We show that when the average harvested energy ($\bar H$) is greater than or equal to the energy required to sense and process a sample ($E_s$), standard existing asymptotic results for the CUSUM test apply since the energy storage level at the sensor is greater than $E_s$ after a sufficiently long time. However, when the $\bar H < E_s$, the energy storage level can be modelled by a positive Harris recurrent Markov chain with a unique stationary distribution. Using asymptotic results from Markov random walk theory and associated nonlinear Markov renewal theory, we establish asymptotic expressions for the expected detection delay and asymptotic exponentiality of the tail distribution of the run-length to a false alarm in this non-trivial case. Numerical results are provided to support the theoretical results.

stat.AP↗