arXiv subjects

Luyao Niu

Publications and source records attributed to Luyao Niu.

At least 55 records · Page 3Linked to original sources

arXiv2022

LDL: A Defense for Label-Based Membership Inference Attacks

The data used to train deep neural network (DNN) models in applications such as healthcare and finance typically contain sensitive information. A DNN model may suffer from overfitting. Overfitted models have been shown to be susceptible to query-based attacks such as membership inference attacks (MIAs). MIAs aim to determine whether a sample belongs to the dataset used to train a classifier (members) or not (nonmembers). Recently, a new class of label based MIAs (LAB MIAs) was proposed, where an adversary was only required to have knowledge of predicted labels of samples. Developing a defense against an adversary carrying out a LAB MIA on DNN models that cannot be retrained remains an open problem. We present LDL, a light weight defense against LAB MIAs. LDL works by constructing a high-dimensional sphere around queried samples such that the model decision is unchanged for (noisy) variants of the sample within the sphere. This sphere of label-invariance creates ambiguity and prevents a querying adversary from correctly determining whether a sample is a member or a nonmember. We analytically characterize the success rate of an adversary carrying out a LAB MIA when LDL is deployed, and show that the formulation is consistent with experimental observations. We evaluate LDL on seven datasets -- CIFAR-10, CIFAR-100, GTSRB, Face, Purchase, Location, and Texas -- with varying sizes of training data. All of these datasets have been used by SOTA LAB MIAs. Our experiments demonstrate that LDL reduces the success rate of an adversary carrying out a LAB MIA in each case. We empirically compare LDL with defenses against LAB MIAs that require retraining of DNN models, and show that LDL performs favorably despite not needing to retrain the DNNs.

cs.LG↗

arXiv2022

A Timing-Based Framework for Designing Resilient Cyber-Physical Systems under Safety Constraint

Cyber-physical systems (CPS) are required to satisfy safety constraints in various application domains such as robotics, industrial manufacturing systems, and power systems. Faults and cyber attacks have been shown to cause safety violations, which can damage the system and endanger human lives. Resilient architectures have been proposed to ensure safety of CPS under such faults and attacks via methodologies including redundancy and restarting from safe operating conditions. The existing resilient architectures for CPS utilize different mechanisms to guarantee safety, and currently there is no approach to compare them. Moreover, the analysis and design undertaken for CPS employing one architecture is not readily extendable to another. In this paper, we propose a timing-based framework for CPS employing various resilient architectures and develop a common methodology for safety analysis and computation of control policies and design parameters. Using the insight that the cyber subsystem operates in one out of a finite number of statuses, we first develop a hybrid system model that captures CPS adopting any of these architectures. Based on the hybrid system, we formulate the problem of joint computation of control policies and associated timing parameters for CPS to satisfy a given safety constraint and derive sufficient conditions for the solution. Utilizing the derived conditions, we provide an algorithm to compute control policies and timing parameters relevant to the employed architecture. We also note that our solution can be applied to a wide class of CPS with polynomial dynamics and also allows incorporation of new architectures. We verify our proposed framework by performing a case study on adaptive cruise control of vehicles.

eess.SY↗

arXiv2022

Abstraction-Free Control Synthesis to Satisfy Temporal Logic Constraints under Sensor Faults and Attacks

We study the problem of synthesizing a controller to satisfy a complex task in the presence of sensor faults and attacks. We model the task using Gaussian distribution temporal logic (GDTL), and propose a solution approach that does not rely on computing any finite abstraction to model the system. We decompose the GDTL specification into a sequence of reach-avoid sub-tasks. We develop a class of fault-tolerant finite time convergence control barrier functions (CBFs) to guarantee that a dynamical system reaches a set within finite time almost surely in the presence of malicious attacks. We use the fault-tolerant finite time convergence CBFs to guarantee the satisfaction of `reach' property. We ensure `avoid' part in each sub-task using fault-tolerant zeroing CBFs. These fault-tolerant CBFs formulate a set of linear constraints on the control input for each sub-task. We prove that if the error incurred by system state estimation is bounded by a certain threshold, then our synthesized controller fulfills each reach-avoid sub-task almost surely for any possible sensor fault and attack, and thus the GDTL specification is satisfied with probability one. We demonstrate our proposed approach using a numerical study on the coordination of two wheeled mobile robots.

eess.SY↗

arXiv2022

Barrier Certificate based Safe Control for LiDAR-based Systems under Sensor Faults and Attacks

Autonomous Cyber-Physical Systems (CPS) fuse proprioceptive sensors such as GPS and exteroceptive sensors including Light Detection and Ranging (LiDAR) and cameras for state estimation and environmental observation. It has been shown that both types of sensors can be compromised by malicious attacks, leading to unacceptable safety violations. We study the problem of safety-critical control of a LiDAR-based system under sensor faults and attacks. We propose a framework consisting of fault tolerant estimation and fault tolerant control. The former reconstructs a LiDAR scan with state estimations, and excludes the possible faulty estimations that are not aligned with LiDAR measurements. We also verify the correctness of LiDAR scans by comparing them with the reconstructed ones and removing the possibly compromised sector in the scan. Fault tolerant control computes a control signal with the remaining estimations at each time step. We prove that the synthesized control input guarantees system safety using control barrier certificates. We validate our proposed framework using a UAV delivery system in an urban environment. We show that our proposed approach guarantees safety for the UAV whereas a baseline fails.

eess.SY↗

arXiv2022

Game of Trojans: A Submodular Byzantine Approach

Machine learning models in the wild have been shown to be vulnerable to Trojan attacks during training. Although many detection mechanisms have been proposed, strong adaptive attackers have been shown to be effective against them. In this paper, we aim to answer the questions considering an intelligent and adaptive adversary: (i) What is the minimal amount of instances required to be Trojaned by a strong attacker? and (ii) Is it possible for such an attacker to bypass strong detection mechanisms? We provide an analytical characterization of adversarial capability and strategic interactions between the adversary and detection mechanism that take place in such models. We characterize adversary capability in terms of the fraction of the input dataset that can be embedded with a Trojan trigger. We show that the loss function has a submodular structure, which leads to the design of computationally efficient algorithms to determine this fraction with provable bounds on optimality. We propose a Submodular Trojan algorithm to determine the minimal fraction of samples to inject a Trojan trigger. To evade detection of the Trojaned model, we model strategic interactions between the adversary and Trojan detection mechanism as a two-player game. We show that the adversary wins the game with probability one, thus bypassing detection. We establish this by proving that output probability distributions of a Trojan model and a clean model are identical when following the Min-Max (MM) Trojan algorithm. We perform extensive evaluations of our algorithms on MNIST, CIFAR-10, and EuroSAT datasets. The results show that (i) with Submodular Trojan algorithm, the adversary needs to embed a Trojan trigger into a very small fraction of samples to achieve high accuracy on both Trojan and clean samples, and (ii) the MM Trojan algorithm yields a trained Trojan model that evades detection with probability 1.

cs.LG↗

arXiv2022

LQG Reference Tracking with Safety and Reachability Guarantees under Unknown False Data Injection Attacks

We investigate a linear quadratic Gaussian (LQG) tracking problem with safety and reachability constraints in the presence of an adversary who mounts an FDI attack on an unknown set of sensors. For each possible set of compromised sensors, we maintain a state estimator disregarding the sensors in that set, and calculate the optimal LQG control input at each time based on this estimate. We propose a control policy which constrains the control input to lie within a fixed distance of the optimal control input corresponding to each state estimate. The control input is obtained at each time step by solving a quadratically constrained quadratic program (QCQP). We prove that our policy can achieve a desired probability of safety and reachability using the barrier certificate method. Our control policy is evaluated via a numerical case study.

eess.SY↗

arXiv2022

A Compositional Approach to Safety-Critical Resilient Control for Systems with Coupled Dynamics

Complex, interconnected Cyber-physical Systems (CPS) are increasingly common in applications including smart grids and transportation. Ensuring safety of interconnected systems whose dynamics are coupled is challenging because the effects of faults and attacks in one sub-system can propagate to other sub-systems and lead to safety violations. In this paper, we study the problem of safety-critical control for CPS with coupled dynamics when some sub-systems are subject to failure or attack. We first propose resilient-safety indices (RSIs) for the faulty or compromised sub-systems that bound the worst-case impacts of faulty or compromised sub-systems on a set of specified safety constraints. By incorporating the RSIs, we provide a sufficient condition for the synthesis of control policies in each failure- and attack- free sub-systems. The synthesized control policies compensate for the impacts of the faulty or compromised sub-systems to guarantee safety. We formulate sum-of-square optimization programs to compute the RSIs and the safety-ensuring control policies. We present a case study that applies our proposed approach on the temperature regulation of three coupled rooms. The case study demonstrates that control policies obtained using our algorithm guarantee system's safety constraints.

eess.SY↗

arXiv2022

An Analytical Framework for Control Synthesis of Cyber-Physical Systems with Safety Guarantee

Cyber-physical systems (CPS) are required to operate safely under fault and malicious attacks. The simplex architecture and the recently proposed cyber resilient architectures, e.g., Byzantine fault tolerant++ (BFT++), provide safety for CPS under faults and malicious cyber attacks, respectively. However, these existing architectures make use of different timing parameters and implementations to provide safety, and are seemingly unrelated. In this paper, we propose an analytical framework to represent the simplex, BFT++ and other practical cyber resilient architectures (CRAs). We construct a hybrid system that models CPS adopting any of these architectures. We derive sufficient conditions via our proposed framework under which a control policy is guaranteed to be safe. We present an algorithm to synthesize the control policy. We validate the proposed framework using a case study on lateral control of a Boeing 747, and demonstrate that our proposed approach ensures safety of the system.

eess.SY↗

arXiv2021

Safety-Critical Control Synthesis for Unknown Sampled-Data Systems via Control Barrier Functions

As the complexity of control systems increases, safety becomes an increasingly important property since safety violations can damage the plant and put the system operator in danger. When the system dynamics are unknown, safety-critical synthesis becomes more challenging. Additionally, modern systems are controlled digitally and hence behave as sampled-data systems, i.e., the system dynamics evolve continuously while the control input is applied at discrete time steps. In this paper, we study the problem of control synthesis for safety-critical sampled-data systems with unknown dynamics. We overcome the challenges introduced by sampled-data implementation and unknown dynamics by constructing a set of control barrier function (CBF)-based constraints. By satisfying the constructed CBF constraint at each sampling time, we guarantee the unknown sampled-data system is safe for all time. We formulate a non-convex program to solve for the control signal at each sampling time. We decompose the non-convex program into two convex sub-problems. We illustrate the proposed approach using a numerical case study.

eess.SY↗

arXiv2021

A Game-Theoretic Framework for Controlled Islanding in the Presence of Adversaries

Controlled islanding effectively mitigates cascading failures by partitioning the power system into a set of disjoint islands. In this paper, we study the controlled islanding problem of a power system under disturbances introduced by a malicious adversary. We formulate the interaction between the grid operator and adversary using a game-theoretic framework. The grid operator first computes a controlled islanding strategy, along with the power generation for the post-islanding system to guarantee stability. The adversary observes the strategies of the grid operator. The adversary then identifies critical substations of the power system to compromise and trips the transmission lines that are connected with compromised substations. For our game formulation, we propose a double oracle algorithm based approach that solves the best response for each player. We show that the best responses for the grid operator and adversary can be formulated as mixed integer linear programs. In addition, the best response of the adversary is equivalent to a submodular maximization problem under a cardinality constraint, which can be approximated up to a $(1-\frac{1}{e})$ optimality bound in polynomial time. We compare the proposed approach with a baseline where the grid operator computes an islanding strategy by minimizing the power flow disruption without considering the possible response from the adversary. We evaluate both approaches using IEEE 9-bus, 14-bus, 30-bus, 39-bus, 57-bus, and 118-bus power system case study data. Our proposed approach achieves better performance than the baseline in about $44\%$ of test cases, and on average it incurs about 12.27 MW less power flow disruption.

eess.SY↗

arXiv2021

Scalable Planning in Multi-Agent MDPs

Multi-agent Markov Decision Processes (MMDPs) arise in a variety of applications including target tracking, control of multi-robot swarms, and multiplayer games. A key challenge in MMDPs occurs when the state and action spaces grow exponentially in the number of agents, making computation of an optimal policy computationally intractable for medium- to large-scale problems. One property that has been exploited to mitigate this complexity is transition independence, in which each agent's transition probabilities are independent of the states and actions of other agents. Transition independence enables factorization of the MMDP and computation of local agent policies but does not hold for arbitrary MMDPs. In this paper, we propose an approximate transition dependence property, called $δ$-transition dependence and develop a metric for quantifying how far an MMDP deviates from transition independence. Our definition of $δ$-transition dependence recovers transition independence as a special case when $δ$ is zero. We develop a polynomial time algorithm in the number of agents that achieves a provable bound on the global optimum when the reward functions are monotone increasing and submodular in the agent actions. We evaluate our approach on two case studies, namely, multi-robot control and multi-agent patrolling example.

cs.MA↗

arXiv2021

Reinforcement Learning Beyond Expectation

The inputs and preferences of human users are important considerations in situations where these users interact with autonomous cyber or cyber-physical systems. In these scenarios, one is often interested in aligning behaviors of the system with the preferences of one or more human users. Cumulative prospect theory (CPT) is a paradigm that has been empirically shown to model a tendency of humans to view gains and losses differently. In this paper, we consider a setting where an autonomous agent has to learn behaviors in an unknown environment. In traditional reinforcement learning, these behaviors are learned through repeated interactions with the environment by optimizing an expected utility. In order to endow the agent with the ability to closely mimic the behavior of human users, we optimize a CPT-based cost. We introduce the notion of the CPT-value of an action taken in a state, and establish the convergence of an iterative dynamic programming-based approach to estimate this quantity. We develop two algorithms to enable agents to learn policies to optimize the CPT-vale, and evaluate these algorithms in environments where a target state has to be reached while avoiding obstacles. We demonstrate that behaviors of the agent learned using these algorithms are better aligned with that of a human user who might be placed in the same environment, and is significantly improved over a baseline that optimizes an expected utility.

cs.LG↗

arXiv2020

Secure Control in Partially Observable Environments to Satisfy LTL Specifications

This paper studies the synthesis of control policies for an agent that has to satisfy a temporal logic specification in a partially observable environment, in the presence of an adversary. The interaction of the agent (defender) with the adversary is modeled as a partially observable stochastic game. The goal is to generate a defender policy to maximize satisfaction of a given temporal logic specification under any adversary policy. The search for policies is limited to the space of finite state controllers, which leads to a tractable approach to determine policies. We relate the satisfaction of the specification to reaching (a subset of) recurrent states of a Markov chain. We present an algorithm to determine a set of defender and adversary finite state controllers of fixed sizes that will satisfy the temporal logic specification, and prove that it is sound. We then propose a value-iteration algorithm to maximize the probability of satisfying the temporal logic specification under finite state controllers of fixed sizes. Lastly, we extend this setting to the scenario where the size of the finite state controller of the defender can be increased to improve the satisfaction probability. We illustrate our approach with an example.

eess.SY↗

arXiv2020

Control Barrier Functions for Abstraction-Free Control Synthesis under Temporal Logic Constraints

Temporal logic has been widely used to express complex task specifications for cyber-physical systems (CPSs). One way to synthesize a controller for CPS under temporal logic constraints is to first abstract the CPS as a discrete transition system, and then apply formal methods. This approach, however, is computationally demanding and its scalability suffers due to the curse of dimensionality. In this paper, we propose a control barrier function (CBF) approach to abstraction-free control synthesis under a linear temporal logic (LTL) constraint. We first construct the deterministic Rabin automaton of the specification and compute an accepting run. We then compute a sequence of LTL formulae, each of which must be satisfied during a particular time interval, and prove that satisfying the sequence of formulae is sufficient to satisfy the LTL specification. Finally, we compute a control policy for satisfying each formula by constructing an appropriate CBF. We present a quadratic program to compute the controllers, and show the controllers synthesized using the proposed approach guarantees the system to satisfy the LTL specification, provided the quadratic program is feasible at each time step. A numerical case study is presented to demonstrate the proposed approach.

eess.SY↗

arXiv2020

Privacy-Preserving Resilience of Cyber-Physical Systems to Adversaries

A cyber-physical system (CPS) is expected to be resilient to more than one type of adversary. In this paper, we consider a CPS that has to satisfy a linear temporal logic (LTL) objective in the presence of two kinds of adversaries. The first adversary has the ability to tamper with inputs to the CPS to influence satisfaction of the LTL objective. The interaction of the CPS with this adversary is modeled as a stochastic game. We synthesize a controller for the CPS to maximize the probability of satisfying the LTL objective under any policy of this adversary. The second adversary is an eavesdropper who can observe labeled trajectories of the CPS generated from the previous step. It could then use this information to launch other kinds of attacks. A labeled trajectory is a sequence of labels, where a label is associated to a state and is linked to the satisfaction of the LTL objective at that state. We use differential privacy to quantify the indistinguishability between states that are related to each other when the eavesdropper sees a labeled trajectory. Two trajectories of equal length will be differentially private if they are differentially private at each state along the respective trajectories. We use a skewed Kantorovich metric to compute distances between probability distributions over states resulting from actions chosen according to policies from related states in order to quantify differential privacy. Moreover, we do this in a manner that does not affect the satisfaction probability of the LTL objective. We validate our approach on a simulation of a UAV that has to satisfy an LTL objective in an adversarial environment.

eess.SY↗

arXiv2020

Control Synthesis for Cyber-Physical Systems to Satisfy Metric Interval Temporal Logic Objectives under Timing and Actuator Attacks

This paper studies the synthesis of controllers for cyber-physical systems (CPSs) that are required to carry out complex tasks that are time-sensitive, in the presence of an adversary. The task is specified as a formula in metric interval temporal logic (MITL). The adversary is assumed to have the ability to tamper with the control input to the CPS and also manipulate timing information perceived by the CPS. In order to model the interaction between the CPS and the adversary, and also the effect of these two classes of attacks, we define an entity called a durational stochastic game (DSG). DSGs probabilistically capture transitions between states in the environment, and also the time taken for these transitions. With the policy of the defender represented as a finite state controller (FSC), we present a value-iteration based algorithm that computes an FSC that maximizes the probability of satisfying the MITL specification under the two classes of attacks. A numerical case-study on a signalized traffic network is presented to illustrate our results.

eess.SY↗

arXiv2019

A Differentially Private Incentive Design for Traffic Offload to Public Transportationx

Increasingly large trip demands have strained urban transportation capacity, which consequently leads to traffic congestion and rapid growth of greenhouse gas emissions. In this work, we focus on achieving sustainable transportation by incentivizing passengers to switch from private cars to public transport. We address the following challenges. First, the passengers incur inconvenience costs when changing their transit behaviors due to delay and discomfort, and thus need to be reimbursed. Second, the inconvenience cost, however, is unknown to the government when choosing the incentives. Furthermore, changing transit behaviors raises privacy concerns from passengers. An adversary could infer personal information, (e.g., daily routine, region of interest, and wealth), by observing the decisions made by the government, which are known to the public. We adopt the concept of differential privacy and propose privacy-preserving incentive designs under two settings, denoted as two-way communication and one-way communication. Under two-way communication, passengers submit bids and then the government determines the incentives, whereas in one-way communication the government simply sets a price without acquiring information from the passengers. Under one-way communication, we focus on how the government should design the incentives without revealing passengers' inconvenience costs while still preserving differential privacy. We formulate the problem as a convex program, and propose a differentially private and near-optimal solution algorithm. A numerical case study using Caltrans Performance Measurement System (PeMS) data source is presented as evaluation. The results show that the proposed approaches achieve a win-win situation in which both the government and passengers obtain non-negative utilities.

cs.CY↗

arXiv2019

Linear Temporal Logic Satisfaction in Adversarial Environments using Secure Control Barrier Certificates

This paper studies the satisfaction of a class of temporal properties for cyber-physical systems (CPSs) over a finite-time horizon in the presence of an adversary, in an environment described by discrete-time dynamics. The temporal logic specification is given in safe-LTL_F, a fragment of linear temporal logic over traces of finite length. The interaction of the CPS with the adversary is modeled as a two-player zero-sum discrete-time dynamic stochastic game with the CPS as defender. We formulate a dynamic programming based approach to determine a stationary defender policy that maximized the probability of satisfaction of a safe-LTL_F formula over a finite time-horizon under any stationary adversary policy. We introduce secure control barrier certificates (S-CBCs), a generalization of barrier certificates and control barrier certificates that accounts for the presence of an adversary, and use S-CBCs to provide a lower bound on the above satisfaction probability. When the dynamics of the evolution of the system state has a specific underlying structure, we present a way to determine an S-CBC as a polynomial in the state variables using sum-of-squares optimization. An illustrative example demonstrates our approach.

eess.SY↗