arXiv subjects

Andrew Clark

Publications and source records attributed to Andrew Clark.

At least 55 records · Page 3Linked to original sources

arXiv2021

Safety-Critical Control Synthesis for Unknown Sampled-Data Systems via Control Barrier Functions

As the complexity of control systems increases, safety becomes an increasingly important property since safety violations can damage the plant and put the system operator in danger. When the system dynamics are unknown, safety-critical synthesis becomes more challenging. Additionally, modern systems are controlled digitally and hence behave as sampled-data systems, i.e., the system dynamics evolve continuously while the control input is applied at discrete time steps. In this paper, we study the problem of control synthesis for safety-critical sampled-data systems with unknown dynamics. We overcome the challenges introduced by sampled-data implementation and unknown dynamics by constructing a set of control barrier function (CBF)-based constraints. By satisfying the constructed CBF constraint at each sampling time, we guarantee the unknown sampled-data system is safe for all time. We formulate a non-convex program to solve for the control signal at each sampling time. We decompose the non-convex program into two convex sub-problems. We illustrate the proposed approach using a numerical case study.

eess.SY↗

arXiv2021

A Game-Theoretic Framework for Controlled Islanding in the Presence of Adversaries

Controlled islanding effectively mitigates cascading failures by partitioning the power system into a set of disjoint islands. In this paper, we study the controlled islanding problem of a power system under disturbances introduced by a malicious adversary. We formulate the interaction between the grid operator and adversary using a game-theoretic framework. The grid operator first computes a controlled islanding strategy, along with the power generation for the post-islanding system to guarantee stability. The adversary observes the strategies of the grid operator. The adversary then identifies critical substations of the power system to compromise and trips the transmission lines that are connected with compromised substations. For our game formulation, we propose a double oracle algorithm based approach that solves the best response for each player. We show that the best responses for the grid operator and adversary can be formulated as mixed integer linear programs. In addition, the best response of the adversary is equivalent to a submodular maximization problem under a cardinality constraint, which can be approximated up to a $(1-\frac{1}{e})$ optimality bound in polynomial time. We compare the proposed approach with a baseline where the grid operator computes an islanding strategy by minimizing the power flow disruption without considering the possible response from the adversary. We evaluate both approaches using IEEE 9-bus, 14-bus, 30-bus, 39-bus, 57-bus, and 118-bus power system case study data. Our proposed approach achieves better performance than the baseline in about $44\%$ of test cases, and on average it incurs about 12.27 MW less power flow disruption.

eess.SY↗

arXiv2021

Dynamic Information Flow Tracking for Detection of Advanced Persistent Threats: A Stochastic Game Approach

Advanced Persistent Threats (APTs) are stealthy customized attacks by intelligent adversaries. This paper deals with the detection of APTs that infiltrate cyber systems and compromise specifically targeted data and/or infrastructures. Dynamic information flow tracking is an information trace-based detection mechanism against APTs that taints suspicious information flows in the system and generates security analysis for unauthorized use of tainted data. In this paper, we develop an analytical model for resource-efficient detection of APTs using an information flow tracking game. The game is a nonzero-sum, turn-based, stochastic game with asymmetric information as the defender cannot distinguish whether an incoming flow is malicious or benign and hence has only partial state observation. We analyze equilibrium of the game and prove that a Nash equilibrium is given by a solution to the minimum capacity cut set problem on a flow-network derived from the system, where the edge capacities are obtained from the cost of performing security analysis. Finally, we implement our algorithm on the real-world dataset for a data exfiltration attack augmented with false-negative and false-positive rates and compute an optimal defender strategy.

cs.GT↗

arXiv2021

Verification and Synthesis of Control Barrier Functions

Control systems often must satisfy strict safety requirements over an extended operating lifetime. Control Barrier Functions (CBFs) are a promising recent approach to constructing simple and safe control policies. This paper proposes a framework for verifying that a CBF guarantees safety for all time and synthesizing CBFs with verifiable safety in polynomial control systems. Our approach is to show that safety of CBFs is equivalent to the non-existence of solutions to a family of polynomial equations, and then prove that this nonexistence is equivalent to a pair of sum-of-squares constraints via the Positivstellensatz of algebraic geometry. We develop this Positivstellensatz to verify CBFs, as well as generalization to high-degree systems and multiple CBF constraints. We then propose a set of heuristics for CBF synthesis, including a general alternating-descent heuristic, a specialized approach for compact safe regions, and an approach for convex unsafe regions. Our approach is illustrated on two numerical examples.

eess.SY↗

arXiv2021

Scalable Planning in Multi-Agent MDPs

Multi-agent Markov Decision Processes (MMDPs) arise in a variety of applications including target tracking, control of multi-robot swarms, and multiplayer games. A key challenge in MMDPs occurs when the state and action spaces grow exponentially in the number of agents, making computation of an optimal policy computationally intractable for medium- to large-scale problems. One property that has been exploited to mitigate this complexity is transition independence, in which each agent's transition probabilities are independent of the states and actions of other agents. Transition independence enables factorization of the MMDP and computation of local agent policies but does not hold for arbitrary MMDPs. In this paper, we propose an approximate transition dependence property, called $δ$-transition dependence and develop a metric for quantifying how far an MMDP deviates from transition independence. Our definition of $δ$-transition dependence recovers transition independence as a special case when $δ$ is zero. We develop a polynomial time algorithm in the number of agents that achieves a provable bound on the global optimum when the reward functions are monotone increasing and submodular in the agent actions. We evaluate our approach on two case studies, namely, multi-robot control and multi-agent patrolling example.

cs.MA↗

arXiv2021

Reinforcement Learning Beyond Expectation

The inputs and preferences of human users are important considerations in situations where these users interact with autonomous cyber or cyber-physical systems. In these scenarios, one is often interested in aligning behaviors of the system with the preferences of one or more human users. Cumulative prospect theory (CPT) is a paradigm that has been empirically shown to model a tendency of humans to view gains and losses differently. In this paper, we consider a setting where an autonomous agent has to learn behaviors in an unknown environment. In traditional reinforcement learning, these behaviors are learned through repeated interactions with the environment by optimizing an expected utility. In order to endow the agent with the ability to closely mimic the behavior of human users, we optimize a CPT-based cost. We introduce the notion of the CPT-value of an action taken in a state, and establish the convergence of an iterative dynamic programming-based approach to estimate this quantity. We develop two algorithms to enable agents to learn policies to optimize the CPT-vale, and evaluate these algorithms in environments where a target state has to be reached while avoiding obstacles. We demonstrate that behaviors of the agent learned using these algorithms are better aligned with that of a human user who might be placed in the same environment, and is significantly improved over a baseline that optimizes an expected utility.

cs.LG↗

arXiv2020

Secure Control in Partially Observable Environments to Satisfy LTL Specifications

This paper studies the synthesis of control policies for an agent that has to satisfy a temporal logic specification in a partially observable environment, in the presence of an adversary. The interaction of the agent (defender) with the adversary is modeled as a partially observable stochastic game. The goal is to generate a defender policy to maximize satisfaction of a given temporal logic specification under any adversary policy. The search for policies is limited to the space of finite state controllers, which leads to a tractable approach to determine policies. We relate the satisfaction of the specification to reaching (a subset of) recurrent states of a Markov chain. We present an algorithm to determine a set of defender and adversary finite state controllers of fixed sizes that will satisfy the temporal logic specification, and prove that it is sound. We then propose a value-iteration algorithm to maximize the probability of satisfying the temporal logic specification under finite state controllers of fixed sizes. Lastly, we extend this setting to the scenario where the size of the finite state controller of the defender can be increased to improve the satisfaction probability. We illustrate our approach with an example.

eess.SY↗

arXiv2020

Control Barrier Functions for Stochastic Systems

Control Barrier Functions (CBFs) aim to ensure safety by constraining the control input at each time step so that the system state remains within a desired safe region. This paper presents a framework for CBFs in stochastic systems in the presence of Gaussian process and measurement noise. We first consider the case where the system state is known at each time step, and present reciprocal and zero CBF constructions that guarantee safety with probability 1. We extend our results to high relative degree systems with linear dynamics and affine safety constraints. We then develop CBFs for incomplete state information environments, in which the state must be estimated using sensors that are corrupted by Gaussian noise. We prove that our proposed CBF ensures safety with probability 1 when the state estimate is within a given bound of the true state, which can be achieved using an Extended Kalman Filter when the system is linear or the process and measurement noise are sufficiently small. We propose control policies that combine these CBFs with Control Lyapunov Functions in order to jointly ensure safety and stochastic stability. Our results are validated via numerical study on a multi-agent collision avoidance example.

math.OC↗

arXiv2020

West Australian Pandemic Response: The Black Swan of Black Swans

The COVID-19 Pandemic has been described as the global challenge of our time, an enormous human tragedy with dramatic economic impacts. This paper describes the response and expected recovery process for Western Australia, where a rapid and effective response was implemented. This has enabled an early transition into an expected recovery both in health and economic terms. The positive lessons learned from this experience are documented as they emerge in order to support other states and nations as they address this issue globally in the near-term and consider enduring improvements for the longer term. While the authors have personal experience in the WA context, wider observations across Australia and selected international benchmarks are also included. Key lessons include the importance of good health advice in Australia's interest; timely, synchronized and aligned action at all levels of government; a program of well communicated, aligned health and economic measures which support all in society allowing a very high level of appropriate community behaviour, ensuring the health system was not overloaded; innovation in telehealth, testing, pandemic modelling, and integrated operations which also allowed essential industries to continue; and strong border and travel controls with highly effective isolation preventing community spread, ultimately enabling rapid elimination of the disease from the hospital system. In combination, these demonstrate that in the case of Western Australia the result of first eliminating the disease from the community, and then reopening the economy progressively at a strong pace, has enabled a world leading outcome in both in health and economic terms. The lessons from this experience are widely applicable, shareable both as supporting service to other regions and through knowledge transfer.

q-bio.PE↗

arXiv2020

Control Barrier Functions for Abstraction-Free Control Synthesis under Temporal Logic Constraints

Temporal logic has been widely used to express complex task specifications for cyber-physical systems (CPSs). One way to synthesize a controller for CPS under temporal logic constraints is to first abstract the CPS as a discrete transition system, and then apply formal methods. This approach, however, is computationally demanding and its scalability suffers due to the curse of dimensionality. In this paper, we propose a control barrier function (CBF) approach to abstraction-free control synthesis under a linear temporal logic (LTL) constraint. We first construct the deterministic Rabin automaton of the specification and compute an accepting run. We then compute a sequence of LTL formulae, each of which must be satisfied during a particular time interval, and prove that satisfying the sequence of formulae is sufficient to satisfy the LTL specification. Finally, we compute a control policy for satisfying each formula by constructing an appropriate CBF. We present a quadratic program to compute the controllers, and show the controllers synthesized using the proposed approach guarantees the system to satisfy the LTL specification, provided the quadratic program is feasible at each time step. A numerical case study is presented to demonstrate the proposed approach.

eess.SY↗

arXiv2020

Privacy-Preserving Resilience of Cyber-Physical Systems to Adversaries

A cyber-physical system (CPS) is expected to be resilient to more than one type of adversary. In this paper, we consider a CPS that has to satisfy a linear temporal logic (LTL) objective in the presence of two kinds of adversaries. The first adversary has the ability to tamper with inputs to the CPS to influence satisfaction of the LTL objective. The interaction of the CPS with this adversary is modeled as a stochastic game. We synthesize a controller for the CPS to maximize the probability of satisfying the LTL objective under any policy of this adversary. The second adversary is an eavesdropper who can observe labeled trajectories of the CPS generated from the previous step. It could then use this information to launch other kinds of attacks. A labeled trajectory is a sequence of labels, where a label is associated to a state and is linked to the satisfaction of the LTL objective at that state. We use differential privacy to quantify the indistinguishability between states that are related to each other when the eavesdropper sees a labeled trajectory. Two trajectories of equal length will be differentially private if they are differentially private at each state along the respective trajectories. We use a skewed Kantorovich metric to compute distances between probability distributions over states resulting from actions chosen according to policies from related states in order to quantify differential privacy. Moreover, we do this in a manner that does not affect the satisfaction probability of the LTL objective. We validate our approach on a simulation of a UAV that has to satisfy an LTL objective in an adversarial environment.

eess.SY↗

arXiv2020

Submodular Input Selection for Synchronization in Kuramoto Networks

Synchronization is an essential property of engineered and natural networked dynamical systems. The Kuramoto model of nonlinear synchronization has been widely studied in applications including entrainment of clock cells in brain networks and power system stability. Synchronization of Kuramoto networks has been found to be challenging in the presence of signed couplings between oscillators and when the network includes oscillators with heterogeneous natural frequencies. In this paper, we study the problem of minimum-set control input selection for synchronizing signed Kuramoto networks. We first derive sufficient conditions for synchronization in homogeneous as well as heterogeneous Kuramoto networks using a passivity-based framework. We then develop a submodular algorithm for selecting a minimum set of control inputs for a given Kuramoto network. We evaluate our approach through a numerical study on multiple classes of graphs, including undirected, directed, and cycle graphs.

eess.SY↗

arXiv2020

Time-resolved formation of excited atomic and molecular states in XUV-induced nanoplasmas in ammonia clusters

High intensity XUV radiation from a free-electron (FEL) was used to create a nanoplasma inside ammonia clusters with the intent of studying the resulting electron-ion interactions and their interplay with plasma evolution. In a plasma-like state, electrons with kinetic energy lower than the local collective Coulomb potential of the positive ionic core are trapped in the cluster and take part in secondary processes (e.g. electron-impact excitation/ionization and electron-ion recombination) which lead to subsequent excited and neutral molecular fragmentation. Using a time-delayed UV laser, the dynamics of the excited atomic and molecular states are probed from -0.1 ps to 18 ps. We identify three different phases of molecular fragmentation that are clearly distinguished by the effect of the probe laser on the ionic and electronic yield. We propose a simple model to rationalize our data and further identify two separate channels leading to the formation of excited hydrogen.

physics.atm-clus↗

arXiv2020

Control Synthesis for Cyber-Physical Systems to Satisfy Metric Interval Temporal Logic Objectives under Timing and Actuator Attacks

This paper studies the synthesis of controllers for cyber-physical systems (CPSs) that are required to carry out complex tasks that are time-sensitive, in the presence of an adversary. The task is specified as a formula in metric interval temporal logic (MITL). The adversary is assumed to have the ability to tamper with the control input to the CPS and also manipulate timing information perceived by the CPS. In order to model the interaction between the CPS and the adversary, and also the effect of these two classes of attacks, we define an entity called a durational stochastic game (DSG). DSGs probabilistically capture transitions between states in the environment, and also the time taken for these transitions. With the policy of the defender represented as a finite state controller (FSC), we present a value-iteration based algorithm that computes an FSC that maximizes the probability of satisfying the MITL specification under the two classes of attacks. A numerical case-study on a signalized traffic network is presented to illustrate our results.

eess.SY↗

arXiv2020

FRESH: Interactive Reward Shaping in High-Dimensional State Spaces using Human Feedback

Reinforcement learning has been successful in training autonomous agents to accomplish goals in complex environments. Although this has been adapted to multiple settings, including robotics and computer games, human players often find it easier to obtain higher rewards in some environments than reinforcement learning algorithms. This is especially true of high-dimensional state spaces where the reward obtained by the agent is sparse or extremely delayed. In this paper, we seek to effectively integrate feedback signals supplied by a human operator with deep reinforcement learning algorithms in high-dimensional state spaces. We call this FRESH (Feedback-based REward SHaping). During training, a human operator is presented with trajectories from a replay buffer and then provides feedback on states and actions in the trajectory. In order to generalize feedback signals provided by the human operator to previously unseen states and actions at test-time, we use a feedback neural network. We use an ensemble of neural networks with a shared network architecture to represent model uncertainty and the confidence of the neural network in its output. The output of the feedback neural network is converted to a shaping reward that is augmented to the reward provided by the environment. We evaluate our approach on the Bowling and Skiing Atari games in the arcade learning environment. Although human experts have been able to achieve high scores in these environments, state-of-the-art deep learning algorithms perform poorly. We observe that FRESH is able to achieve much higher scores than state-of-the-art deep learning algorithms in both environments. FRESH also achieves a 21.4% higher score than a human expert in Bowling and does as well as a human expert in Skiing.

cs.AI↗

arXiv2019

A Differentially Private Incentive Design for Traffic Offload to Public Transportationx

Increasingly large trip demands have strained urban transportation capacity, which consequently leads to traffic congestion and rapid growth of greenhouse gas emissions. In this work, we focus on achieving sustainable transportation by incentivizing passengers to switch from private cars to public transport. We address the following challenges. First, the passengers incur inconvenience costs when changing their transit behaviors due to delay and discomfort, and thus need to be reimbursed. Second, the inconvenience cost, however, is unknown to the government when choosing the incentives. Furthermore, changing transit behaviors raises privacy concerns from passengers. An adversary could infer personal information, (e.g., daily routine, region of interest, and wealth), by observing the decisions made by the government, which are known to the public. We adopt the concept of differential privacy and propose privacy-preserving incentive designs under two settings, denoted as two-way communication and one-way communication. Under two-way communication, passengers submit bids and then the government determines the incentives, whereas in one-way communication the government simply sets a price without acquiring information from the passengers. Under one-way communication, we focus on how the government should design the incentives without revealing passengers' inconvenience costs while still preserving differential privacy. We formulate the problem as a convex program, and propose a differentially private and near-optimal solution algorithm. A numerical case study using Caltrans Performance Measurement System (PeMS) data source is presented as evaluation. The results show that the proposed approaches achieve a win-win situation in which both the government and passengers obtain non-negative utilities.

cs.CY↗

arXiv2019

Linear Temporal Logic Satisfaction in Adversarial Environments using Secure Control Barrier Certificates

This paper studies the satisfaction of a class of temporal properties for cyber-physical systems (CPSs) over a finite-time horizon in the presence of an adversary, in an environment described by discrete-time dynamics. The temporal logic specification is given in safe-LTL_F, a fragment of linear temporal logic over traces of finite length. The interaction of the CPS with the adversary is modeled as a two-player zero-sum discrete-time dynamic stochastic game with the CPS as defender. We formulate a dynamic programming based approach to determine a stationary defender policy that maximized the probability of satisfaction of a safe-LTL_F formula over a finite time-horizon under any stationary adversary policy. We introduce secure control barrier certificates (S-CBCs), a generalization of barrier certificates and control barrier certificates that accounts for the presence of an adversary, and use S-CBCs to provide a lower bound on the above satisfaction probability. When the dynamics of the evolution of the system state has a specific underlying structure, we present a way to determine an S-CBC as a polynomial in the state variables using sum-of-squares optimization. An illustrative example demonstrates our approach.

eess.SY↗

arXiv2019

Potential-Based Advice for Stochastic Policy Learning

This paper augments the reward received by a reinforcement learning agent with potential functions in order to help the agent learn (possibly stochastic) optimal policies. We show that a potential-based reward shaping scheme is able to preserve optimality of stochastic policies, and demonstrate that the ability of an agent to learn an optimal policy is not affected when this scheme is augmented to soft Q-learning. We propose a method to impart potential based advice schemes to policy gradient algorithms. An algorithm that considers an advantage actor-critic architecture augmented with this scheme is proposed, and we give guarantees on its convergence. Finally, we evaluate our approach on a puddle-jump grid world with indistinguishable states, and the continuous state and action mountain car environment from classical control. Our results indicate that these schemes allow the agent to learn a stochastic optimal policy faster and obtain a higher average reward.

cs.LG↗