arXiv subjects

Hamidreza Modares

Publications and source records attributed to Hamidreza Modares.

34 records · Page 2Linked to original sources

arXiv2022

Deterministic and Stochastic Fixed-time Stability of Discrete-time Autonomous Systems

This paper studies deterministic and stochastic fixed-time stability of autonomous nonlinear discrete-time (DT) systems. Lyapunov conditions are first presented under which the fixed-time stability of deterministic DT system is certified. Extensions to systems under deterministic perturbations as well as stochastic noise are then considered. For the former, the sensitivity to perturbations for fixed-time stable DT systems is analyzed, and it is shown that fixed-time attractiveness is resulted from the presented Lyapunov conditions. For the latter, sufficient Lyapunov conditions for fixed-time stability in probability of nonlinear stochastic DT systems are presented. The fixed upper bound of the settling-time function is derived for both fixed-time stable and fixed-time attractive systems, and the stochastic settling-time function fixed upper bound is derived for stochastic DT systems. Illustrative examples are given along with simulation results to verify the introduced results.

eess.SY↗

arXiv2022

Data Driven Safe Gain-Scheduling Control

Data-based safe gain-scheduling controllers are presented for discrete-time linear parameter-varying systems (LPV) with polytopic models. First, $λ$-contractivity conditions are provided under which safety and stability of the LPV systems are unified through Minkowski functions of the safe sets. Then, to bypass the requirement to identify the system dynamics, a data-based representation of the closed-loop LPV system is provided to directly exploit collected data and construct a safe controller. It is shown that weaker data richness requirements are needed to directly learn a closed-loop safe control policy than to identify the LPV system. The closed-loop data-based representation is leveraged to directly design data-driven gain-scheduling controllers that guarantee $λ$-contractiveness, and, thus, invariance of the safe sets. It is also shown that the problem of designing a data-driven gain-scheduling controller for a polyhedral (ellipsoidal) safe set amounts to a linear program (a semi-definite program). A simulation example is provided to show the effectiveness of the presented approach.

eess.SY↗

arXiv2022

A Convex Optimization Approach for Control of Linear Quadratic Systems with Multiplicative Noise via System Level Synthesis

This paper presents a convex optimization-based solution to the design of state-feedback controllers for solving the linear quadratic regulator (LQR) problem of uncertain discrete-time systems with multiplicative noise. To synthesize a tractable solution, the recently developed system level synthesis (SLS) framework is leveraged. It is shown that SLS shifts the controller synthesis task from the design of a robust controller to the design of the entire set-valued closed-loop system responses. To this end, the closed-loop system response is entirely characterized by probabilistic set-valued maps from the additive noise to control actions and states. A bi-level convex optimization over the achievable set-valued closed-loop responses is then developed to optimize the expected value of the LQR cost against the worst-case closed-loop system response. The solution to this robust optimization problem may be too conservative since it aims at enforcing the design constraints for all possible system realizations. To deal with this issue, the presented optimization problem is next reformulated as a chance-constrained program (CCP) in which the guarantees are not intended in a deterministic sense of satisfaction against all possible closed-loop system responses, but are instead intended in a probabilistic sense of satisfaction against all but a small fraction of the system responses. To approximately solve the CCP without the requirement of knowing the probabilistic description of the uncertainty in the system matrices, the so-called scenario optimization approach is employed, which provides probabilistic guarantees based on a finite number of system realizations and results in a convex optimization program with moderate computational complexity. Finally, numerical simulations are presented to illustrate the theoretical findings.

eess.SY↗

arXiv2022

Performance Analysis of Event-Triggered Consensus Control for Multi-agent Systems under Cyber-Physical Attacks

This work presents a rigorous analysis of the adverse effects of cyber-physical attacks on the performance of multi-agent consensus with event-triggered control protocols. It is shown how a strategic malicious attack on sensors and actuators can deceive the triggering condition of both state-based event-triggered mechanism and combinational state-based event-triggered mechanism, which are commonplace and widely used in the literature. More precisely, it is first shown that a deception attack in the case of combinational state-based event-triggered mechanism can result in a non-triggering misbehavior, in the sense that the compromised agent does not trigger any event and consequently results in partial feedback disconnectivity by preventing information from reaching the local neighbors of the compromised agent. This indicates that the combinational state-based event-triggered mechanism can be leveraged by the attacker to harm the network connectivity by rendering the recent data unavailable to agents. It is then shown that the deception attack in the case of state-based event-triggered mechanism can result in a continuous-triggering misbehavior in the sense that the event-triggered mechanism continuously generates triggering events resulting in undesirable phenomenon of Zeno behavior. Finally, numerical simulations are presented to illustrate the theoretical findings.

eess.SY↗

arXiv2021

Secure Event-Triggered Distributed Kalman Filters for State Estimation over Wireless Sensor Networks

In this paper, we analyze the adverse effects of cyber-physical attacks as well as mitigate their impacts on the event-triggered distributed Kalman filter (DKF). We first show that although event-triggered mechanisms are highly desirable, the attacker can leverage the event-triggered mechanism to cause non-triggering misbehavior which significantly harms the network connectivity and its collective observability. We also show that an attacker can mislead the event-triggered mechanism to achieve continuous-triggering misbehavior which not only drains the communication resources but also harms the network's performance. An information-theoretic approach is presented next to detect attacks on both sensors and communication channels. In contrast to the existing results, the restrictive Gaussian assumption on the attack signal's probability distribution is not required. To mitigate attacks, a meta-Bayesian approach is presented that incorporates the outcome of the attack detection mechanism to perform second-order inference. The proposed second-order inference forms confidence and trust values about the truthfulness or legitimacy of sensors' own estimates and those of their neighbors, respectively. Each sensor communicates its confidence to its neighbors. Sensors then incorporate the confidence they receive from their neighbors and the trust they formed about their neighbors into their posterior update laws to successfully discard corrupted information. Finally, the simulation result validates the effectiveness of the presented resilient event-triggered DKF.

eess.SY↗

arXiv2021

Fully-Heterogeneous Containment Control of a Network of Leader-Follower Systems

This paper develops a distributed solution to the fully-heterogeneous containment control problem (CCP), for which not only the followers' dynamics but also the leaders' dynamics are non-identical. A novel formulation of the fully-heterogeneous CCP is first presented in which each follower constructs its virtual exo-system. To build these virtual exo-systems by followers, a novel distributed algorithm is developed to calculate the so-called normalized level of influences (NLIs) of all leaders on each follower and a novel adaptive distributed observer is designed to estimate the dynamics and states of all leaders that have an influence on each follower. Then, a distributed control protocol is proposed based on the cooperative output regulation framework, utilizing this virtual exo-system. Based on estimations of leaders' dynamics and states and NLIs of leaders on each follower, the solutions of the so-called linear regulator equations are calculated in a distributed manner, and consequently, a distributed control protocol is designed for solving the output containment problem. Finally, theoretical results are verified by performing numerical simulations.

cs.MA↗

arXiv2021

Assured Learning-enabled Autonomy: A Metacognitive Reinforcement Learning Framework

Reinforcement learning (RL) agents with pre-specified reward functions cannot provide guaranteed safety across variety of circumstances that an uncertain system might encounter. To guarantee performance while assuring satisfaction of safety constraints across variety of circumstances, an assured autonomous control framework is presented in this paper by empowering RL algorithms with metacognitive learning capabilities. More specifically, adapting the reward function parameters of the RL agent is performed in a metacognitive decision-making layer to assure the feasibility of RL agent. That is, to assure that the learned policy by the RL agent satisfies safety constraints specified by signal temporal logic while achieving as much performance as possible. The metacognitive layer monitors any possible future safety violation under the actions of the RL agent and employs a higher-layer Bayesian RL algorithm to proactively adapt the reward function for the lower-layer RL agent. To minimize the higher-layer Bayesian RL intervention, a fitness function is leveraged by the metacognitive layer as a metric to evaluate success of the lower-layer RL agent in satisfaction of safety and liveness specifications, and the higher-layer Bayesian RL intervenes only if there is a risk of lower-layer RL failure. Finally, a simulation example is provided to validate the effectiveness of the proposed approach.

cs.AI↗

arXiv2021

A Convex Programming Approach to Data-Driven Risk-Averse Reinforcement Learning

This paper presents a model-free reinforcement learning (RL) algorithm to solve the risk-averse optimal control (RAOC) problem for discrete-time nonlinear systems. While successful RL algorithms have been presented to learn optimal control solutions under epistemic uncertainties (i.e., lack of knowledge of system dynamics), they do so by optimizing the expected utility of outcomes, which ignores the variance of cost under aleatory uncertainties (i.e., randomness). Performance-critical systems, however, must not only optimize the expected performance, but also reduce its variance to avoid performance fluctuation during RL's course of operation. To solve the RAOC problem, this paper presents the following three variants of RL algorithms and analyze their advantages and preferences for different situations/systems: 1) a one-shot static convex program -based RL, 2) an iterative value iteration (VI) algorithm that solves a linear programming (LP) optimization at each iteration, and 3) an iterative policy iteration (PI) algorithm that solves a convex optimization at each iteration and guarantees the stability of the consecutive control policies. Convergence of the exact optimization problems, which are infinite-dimensional in all three cases, to the optimal risk-averse value function is shown. To turn these optimization problems into standard optimization problems with finite decision variables and constraints, function approximation for value estimations as well as constraint sampling are leveraged. Data-driven implementations of these algorithms are provided based on Q-function which enables learning the optimal value without any knowledge of the system dynamics. The performance of the approximated solutions is also verified through a weighted sup-norm bound and the Lyapunov bound. A simulation example is provided to verify the effectiveness of the presented approach.

eess.SY↗

arXiv2021

Data-driven Dynamic Multi-objective Optimal Control: An Aspiration-satisfying Reinforcement Learning Approach

This paper presents an iterative data-driven algorithm for solving dynamic multi-objective (MO) optimal control problems arising in control of nonlinear continuous-time systems. It is first shown that the Hamiltonian functional corresponding to each objective can be leveraged to compare the performance of admissible policies. Hamiltonian-inequalities are then used for which their satisfaction guarantees satisfying the objectives' aspirations. An aspiration-satisfying dynamic optimization framework is then presented to optimize the main objective while satisfying the aspiration of other objectives. Relation to satisficing (good enough) decision-making framework is shown. A Sum-of-Square (SOS) based iterative algorithm is developed to solve the formulated aspiration-satisfying MO optimization. To obviate the requirement of complete knowledge of the system dynamics, a data-driven satisficing reinforcement learning approach is proposed to solve the SOS optimization problem in real-time using only the information of the system trajectories measured during a time interval without having full knowledge of the system dynamics. Finally, two simulation examples are provided to show the effectiveness of the proposed algorithm.

eess.SY↗

arXiv2020

A Satisficing Control Design Framework with Safety and Performance Guarantees for Constrained Systems under Disturbances

This paper presents a safe robust policy iteration (SR-PI) algorithm to design controllers with satisficing (good enough) performance and safety guarantee. This is in contrast to standard PI-based control design methods with no safety certification. It also moves away from existing safe control design approaches that perform pointwise optimization and are thus myopic. Safety assurance requires satisfying a control barrier function (CBF), which might be in conflict with the performance-driven Lyapunov solution to the Bellman equation arising in each iteration of the PI. Therefore, a new development is required to robustly certify the safety of an improved policy at each iteration of the PI. The proposed SR-PI algorithm unifies performance guarantee (provided by a Bellman inequality) with safety guarantee (provided by a robust CBF) at each iteration. The Bellman inequality resembles the satisficing decision making framework and parameterizes the sacrifice on the performance with an aspiration level when there is a conflict with safety. This aspiration level is optimized at each iteration to minimize the sacrifice on the performance. It is shown that the presented satisficing control policies obtained at each iteration of the SR-PI guarantees robust safety and performance. Robust stability is also guaranteed when there is no conflict with safety. Sum of squares (SOS) program is employed to implement the proposed SR-PI algorithm iteratively. Finally, numerical simulations are carried out to illustrate the proposed satisficing control framework.

eess.SY↗

arXiv2020

Adaptive Finite-time Disturbance Rejection for Nonlinear Systems using an Experience-Replay based Disturbance Observer

Control systems are inevitably affected by external disturbances, and a major objective of the control design is to attenuate or eliminate their adverse effects on the system performance. This paper presents a disturbance rejection approach with two main improvements over existing results: 1) it relaxes the requirement of calculating or measuring the state derivatives, which are not available for measurement, and their calculation is corrupted by noise, and 2) it achieves finite-time disturbance rejection and control. To this end, the disturbance is first modeled by an unknown dynamics, and an adaptive disturbance observer is proposed to estimate it. A filtered regressor form is leveraged to model the nonlinear system and the unknown disturbance. It is shown that using this filtered regressor form, the disturbance is estimated using only measured state of the regressor. That is, contrary to the existing results on disturbance rejection, the presented approach does not require the state derivative measurements. To improve the convergence speed of the disturbance estimation, an adaptive law, equipped with experience replay, is presented. The disturbance observer is then augmented with an adaptive integral terminal sliding mode control to assure the finite-time convergence of tracking error to zero. A verifiable rank condition on the history of the past experience used by the experience-replay technique provides a sufficient condition for convergence. Compared to the existing results, neither the knowledge of the disturbance dynamics nor the state derivatives are required, and finite-time stability is guaranteed. A simulation example illustrates the effectiveness of the proposed approach.

eess.SY↗

arXiv2019

Attack Analysis and Resilient Control Design for Discrete-time Distributed Multi-agent Systems

This work presents a rigorous analysis of the adverse effects of cyber-physical attacks on discrete-time distributed multi-agent systems, and propose a mitigation approach for attacks on sensors and actuators. First, we show how an attack on a compromised agent can propagate and affect intact agents that are reachable from it. This is, an attack on a single node snowballs into a network-wide attack and can even destabilize the entire system. Moreover, we show that the attacker can bypass the robust $H_{\infty}$ control protocol and make it entirely ineffective in attenuating the effect of the adversarial input on the system performance. Finally, to overcome adversarial effects of attacks on sensors and actuators, a distributed adaptive attack compensator is designed by estimating the normal expected behavior of agents. The adaptive attack compensator is augmented with the controller and it is shown that the proposed controller achieves secure consensus in presence of the attacks on sensors and actuators. This controller does not require to make any restrictive assumption on the number of agents or agent's neighbors under direct effect of adversarial input. Moreover, it recovers compromised agents under actuator attacks and avoids propagation of attacks on sensors without removing compromised agents. The effectiveness of the proposed controller and analysis is validated on a network of Sentry autonomous underwater vehicles subject to attacks under different scenarios.

eess.SY↗

arXiv2019

Resilient Synchronization of Distributed Multi-agent Systems under Attacks

In this paper, we first address adverse effects of cyber-physical attacks on distributed synchronization of multi-agent systems, by providing conditions under which an attacker can destabilize the underlying network, as well as another set of conditions under which local neighborhood tracking errors of intact agents converge to zero. Based on this analysis, we propose a Kullback-Liebler divergence based criterion in view of which each agent detects its neighbors' misbehavior and, consequently, forms a self-belief about the trustworthiness of the information it receives. Agents continuously update their self-beliefs and communicate them with their neighbors to inform them of the significance of their outgoing information. Moreover, if the self-belief of an agent is low, it forms trust on its neighbors. Agents incorporate their neighbors' self-beliefs and their own trust values on their control protocols to slow down and mitigate attacks. We show that using the proposed resilient approach, an agent discards the information it receives from a neighbor only if its neighbor is compromised, and not solely based on the discrepancy among neighbors' information, which might be caused by legitimate changes, and not attacks. The proposed approach is guaranteed to work under mild connectivity assumptions.

cs.MA↗

arXiv2018

Resilient Autonomous Control of Distributed Multi-agent Systems in Contested Environments

An autonomous and resilient controller is proposed for leader-follower multi-agent systems under uncertainties and cyber-physical attacks. The leader is assumed non-autonomous with a nonzero control input, which allows changing the team behavior or mission in response to environmental changes. A resilient learning-based control protocol is presented to find optimal solutions to the synchronization problem in the presence of attacks and system dynamic uncertainties. An observer-based distributed H_infinity controller is first designed to prevent propagating the effects of attacks on sensors and actuators throughout the network, as well as to attenuate the effect of these attacks on the compromised agent itself. Non-homogeneous game algebraic Riccati equations are derived to solve the H_infinity optimal synchronization problem and off-policy reinforcement learning is utilized to learn their solution without requiring any knowledge of the agent's dynamics. A trust-confidence based distributed control protocol is then proposed to mitigate attacks that hijack the entire node and attacks on communication links. A confidence value is defined for each agent based solely on its local evidence. The proposed resilient reinforcement learning algorithm employs the confidence value of each agent to indicate the trustworthiness of its own information and broadcast it to its neighbors to put weights on the data they receive from it during and after learning. If the confidence value of an agent is low, it employs a trust mechanism to identify compromised agents and remove the data it receives from them from the learning process. Simulation results are provided to show the effectiveness of the proposed approach.

cs.MA↗

arXiv2018

Observer-based Adaptive Optimal Output Containment Control problem of Linear Heterogeneous Multi-agent Systems with Relative Output Measurements

This paper develops an optimal relative output-feedback based solution to the containment control problem of linear heterogeneous multi-agent systems. A distributed optimal control protocol is presented for the followers to not only assure that their outputs fall into the convex hull of the leaders' output (i.e., the desired or safe region), but also optimizes their transient performance. The proposed optimal control solution is composed of a feedback part, depending of the followers' state, and a feed-forward part, depending on the convex hull of the leaders' state. To comply with most real-world applications, the feedback and feed-forward states are assumed to be unavailable and are estimated using two distributed observers. That is, since the followers cannot directly sense their absolute states, a distributed observer is designed that uses only relative output measurements with respect to their neighbors (measured for example by using range sensors in robotic) and the information which is broadcasted by their neighbors to estimate their states. Moreover, another adaptive distributed observer is designed that uses exchange of information between followers over a communication network to estimate the convex hull of the leaders' state. The proposed observer relaxes the restrictive requirement of knowing the complete knowledge of the leaders' dynamics by all followers. An off-policy reinforcement learning algorithm on an actor-critic structure is next developed to solve the optimal containment control problem online, using relative output measurements and without requirement of knowing the leaders' dynamics by all followers. Finally, the theoretical results are verified by numerical simulations.

eess.SY↗

arXiv2017

Attack Analysis for Distributed Control Systems: An Internal Model Principle Approach

Although adverse effects of attacks have been acknowledged in many cyber-physical systems, there is no system-theoretic comprehension of how a compromised agent can leverage communication capabilities to maximize the damage in distributed multi-agent systems. A rigorous analysis of cyber-physical attacks enables us to increase the system awareness against attacks and design more resilient control protocols. To this end, we will take the role of the attacker to identify the worst effects of attacks on root nodes and non-root nodes in a distributed control system. More specifically, we show that a stealthy attack on root nodes can mislead the entire network to a wrong understanding of the situation and even destabilize the synchronization process. This will be called the internal model principle for the attacker and will intensify the urgency of designing novel control protocols to mitigate these types of attacks.

eess.SY↗