arXiv subjects

Quanyan Zhu

Publications and source records attributed to Quanyan Zhu.

At least 217 records · Page 12Linked to original sources

arXiv2019

Cyber Insurance

This chapter will first present a principal-agent game-theoretic model to capture the interactions between one insurer and one user. The insurer is deemed as the principal who does not have incomplete information about user's security policies. The user, which refers to the infrastructure operator or the customer, implements his local protection and pays a premium to the insurer. The insurer designs an incentive compatible insurance mechanism that includes the premium and the coverage policy, while the user determines whether to participate in the insurance and his effort to defend against attacks. The chapter will also focus on an attack-aware cyber insurance model by introducing the adversarial behaviors into the framework. The behavior of an attacker determines the type of cyber threats, e.g. denial of service (DoS) attacks, data breaches, phishing and spoofing. The distinction of threat types plays a role in determining the type of losses and the coverage policies. The data breaches can lead to not only financial losses but also damage of the reputations. The coverage may only cover certain agreed percentage of the financial losses.

cs.CY↗

arXiv2019

PhD Forum: Enabling Autonomic IoT for Smart Urban Services

The development of autonomous cyber-physical systems (CPS) and advances towards the fifth generation (5G) of wireless technology is promising to revolutionize many industry verticals such as Healthcare, Transportation, Energy, Retail Services, Building Automation, Education, etc., leading to the realization of the smart city paradigm. The Internet of Things (IoT), enables powerful and unprecedented capabilities for intelligent and autonomous operation. We leverage ideas from Network Science, Optimization & Decision Theory, Incentive Mechanism Design, and Data Science/Machine Learning to achieve key design goals, in IoT-enabled urban systems, such as efficiency, security & resilience, and economics.

eess.SP↗

arXiv2019

Control of Multi-Layer Mobile Autonomous Systems in Adversarial Environments: A Games-in-Games Approach

Mobile autonomous system (MAS) becomes pervasive especially in the vehicular and robotic networks. Multiple heterogeneous MAS networks can be integrated together as a multi-layer MAS network to offer holistic services. The network connectivity of multi-layer MAS plays an important role in the information exchange between agents within and across different layers of the network. In this paper, we establish a games-in-games framework to capture the uncoordinated nature of decision makings under adversarial environment at different layers. Specifically, each network operator controls the mobile agents in his own subnetwork and designs a secure strategy to maximize the global network connectivity by considering the behavior of jamming attackers that aim to disconnect the network. The solution concept of meta-equilibrium is proposed to characterize the system-of-systems behavior of the autonomous agents. For online implementation of the control, we design a resilient algorithm that improves the network algebraic connectivity iteratively. We show that the designed algorithm converges to a meta-equilibrium asymptotically. Finally, we use case studies of a two-layer MAS network to corroborate the security and agile resilience of the network controlled by the proposed strategy.

eess.SY↗

arXiv2019

RIoTS: Risk Analysis of IoT Supply Chain Threats

Securing the supply chain of information and communications technology (ICT) has recently emerged as a critical concern for national security and integrity. With the proliferation of Internet of Things (IoT) devices and their increasing role in controlling real world infrastructure, there is a need to analyze risks in networked systems beyond established security analyses. Existing methods in literature typically leverage attack and fault trees to analyze malicious activity and its impact. In this paper, we develop RIoTS, a security risk assessment framework borrowing from system reliability theory to incorporate the supply chain. We also analyze the impact of grouping within suppliers that may pose hidden risks to the systems from malicious supply chain actors. The results show that the proposed analysis is able to reveal hidden threats posed to the IoT ecosystem from potential supplier collusion.

cs.CR↗

arXiv2019

FlipIn: A Game-Theoretic Cyber Insurance Framework for Incentive-Compatible Cyber Risk Management of Internet of Things

Internet of Things (IoT) is highly vulnerable to emerging Advanced Persistent Threats (APTs) that are often operated by well-resourced adversaries. Achieving perfect security for IoT networks is often cost-prohibitive if not impossible. Cyber insurance is a valuable mechanism to mitigate cyber risks for IoT systems. In this work, we propose a bi-level game-theoretic framework called FlipIn to design incentive-compatible and welfare-maximizing cyber insurance contracts. The framework captures the strategic interactions among APT attackers, IoT defenders, and cyber insurance insurers, and incorporates influence networks to assess the systemic cyber risks of interconnected IoT devices. The FlipIn framework formulates a game over networks within a principal-agent problem of moral-hazard type to design a cyber risk-aware insurance contract. We completely characterize the equilibrium solutions of the bi-level games for a network of distributed defenders and a semi-homogeneous centralized defender and show that the optimal insurance contracts cover half of the defenders' losses. Our framework predicts the risk compensation of defenders and the Peltzman effect of insurance. We study a centralized security management scenario and its decentralized counterpart, and leverage numerical experiments to show that network connectivity plays an important role in the security of the IoT devices and the insurability of both distributed and centralized defenders.

cs.CR↗

arXiv2019

Adaptive Honeypot Engagement through Reinforcement Learning of Semi-Markov Decision Processes

A honeynet is a promising active cyber defense mechanism. It reveals the fundamental Indicators of Compromise (IoCs) by luring attackers to conduct adversarial behaviors in a controlled and monitored environment. The active interaction at the honeynet brings a high reward but also introduces high implementation costs and risks of adversarial honeynet exploitation. In this work, we apply infinite-horizon Semi-Markov Decision Process (SMDP) to characterize a stochastic transition and sojourn time of attackers in the honeynet and quantify the reward-risk trade-off. In particular, we design adaptive long-term engagement policies shown to be risk-averse, cost-effective, and time-efficient. Numerical results have demonstrated that our adaptive engagement policies can quickly attract attackers to the target honeypot and engage them for a sufficiently long period to obtain worthy threat information. Meanwhile, the penetration probability is kept at a low level. The results show that the expected utility is robust against attackers of a large range of persistence and intelligence. Finally, we apply reinforcement learning to the SMDP to solve the curse of modeling. Under a prudent choice of the learning rate and exploration policy, we achieve a quick and robust convergence of the optimal policy and value.

cs.CR↗

arXiv2019

A Dynamic Games Approach to Proactive Defense Strategies against Advanced Persistent Threats in Cyber-Physical Systems

Advanced Persistent Threats (APTs) have recently emerged as a significant security challenge for a cyber-physical system due to their stealthy, dynamic and adaptive nature. Proactive dynamic defenses provide a strategic and holistic security mechanism to increase the costs of attacks and mitigate the risks. This work proposes a dynamic game framework to model a long-term interaction between a stealthy attacker and a proactive defender. The stealthy and deceptive behaviors are captured by the multi-stage game of incomplete information, where each player has his own private information unknown to the other. Both players act strategically according to their beliefs which are formed by the multi-stage observation and learning. The perfect Bayesian Nash equilibrium provides a useful prediction of both players' policies because no players benefit from unilateral deviations from the equilibrium. We propose an iterative algorithm to compute the perfect Bayesian Nash equilibrium and use the Tennessee Eastman process as a benchmark case study. Our numerical experiment corroborates the analytical results and provides further insights into the design of proactive defense-in-depth strategies.

cs.GT↗

arXiv2019

Dynamic Games for Secure and Resilient Control System Design

Modern control systems are featured by their hierarchical structure composing of cyber, physical, and human layers. The intricate dependencies among multiple layers and units of modern control systems require an integrated framework to address cross-layer design issues related to security and resilience challenges. To this end, game theory provides a bottom-up modeling paradigm to capture the strategic interactions among multiple components of the complex system and enables a holistic view to understand and design cyber-physical-human control systems. In this review, we first provide a multi-layer perspective toward increasingly complex and integrated control systems and then introduce several variants of dynamic games for modeling different layers of control systems. We present game-theoretic methods for understanding the fundamental tradeoffs of robustness, security, and resilience and developing a clean-slate cross-layer approach to enhance the system performance in various adversarial environments. This review also includes three quintessential research problems that represent three research directions where dynamic game approaches can bridge between multiple research areas and make significant contributions to the design of modern control systems. The paper is concluded with a discussion on emerging areas of research that crosscut dynamic games and control systems.

eess.SY↗

arXiv2019

On Convergence Rate of Adaptive Multiscale Value Function Approximation For Reinforcement Learning

In this paper, we propose a generic framework for devising an adaptive approximation scheme for value function approximation in reinforcement learning, which introduces multiscale approximation. The two basic ingredients are multiresolution analysis as well as tree approximation. Starting from simple refinable functions, multiresolution analysis enables us to construct a wavelet system from which the basis functions are selected adaptively, resulting in a tree structure. Furthermore, we present the convergence rate of our multiscale approximation which does not depend on the regularity of basis functions.

cs.LG↗

arXiv2019

Deceptive Reinforcement Learning Under Adversarial Manipulations on Cost Signals

This paper studies reinforcement learning (RL) under malicious falsification on cost signals and introduces a quantitative framework of attack models to understand the vulnerabilities of RL. Focusing on $Q$-learning, we show that $Q$-learning algorithms converge under stealthy attacks and bounded falsifications on cost signals. We characterize the relation between the falsified cost and the $Q$-factors as well as the policy learned by the learning agent which provides fundamental limits for feasible offensive and defensive moves. We propose a robust region in terms of the cost within which the adversary can never achieve the targeted policy. We provide conditions on the falsified cost which can mislead the agent to learn an adversary's favored policy. A numerical case study of water reservoir control is provided to show the potential hazards of RL in learning-based control systems and corroborate the results.

cs.LG↗

arXiv2019

Dynamic Contract Design for Systemic Cyber Risk Management of Interdependent Enterprise Networks

The interconnectivity of cyber and physical systems and Internet of things has created ubiquitous concerns of cyber threats for enterprise system managers. It is common that the asset owners and enterprise network operators need to work with cybersecurity professionals to manage the risk by remunerating them for their efforts that are not directly observable. In this paper, we use a principal-agent framework to capture the service relationships between the two parties, i.e., the asset owner (principal) and the cyber risk manager (agent). Specifically, we consider a dynamic systemic risk management problem with asymmetric information where the principal can only observe cyber risk outcomes of the enterprise network rather than directly the efforts that the manager expends on protecting the resources. Under this information pattern, the principal aims to minimize the systemic cyber risks by designing a dynamic contract that specifies the compensation flows and the anticipated efforts of the manager by taking into account his incentives and rational behaviors. We formulate a bi-level mechanism design problem for dynamic contract design within the framework of a class of stochastic differential games. We show that the principal has rational controllability of the systemic risk by designing an incentive compatible estimator of the agent's hidden efforts. We characterize the optimal solution by reformulating the problem as a stochastic optimal control program which can be solved using dynamic programming. We further investigate a benchmark scenario with complete information and identify conditions that yield zero information rent and lead to a new certainty equivalence principle for principal-agent problems. Finally, case studies over networked systems are carried out to illustrate the theoretical results obtained.

eess.SY↗

arXiv2019

IoT Supply Chain Security: Overview, Challenges, and the Road Ahead

Supply chain is emerging as the next frontier of threats in the rapidly evolving IoT ecosystem. It is fundamentally more complex compared to traditional ICT systems. We analyze supply chain risks in IoT systems and their unique aspects, discuss research challenges in supply chain security, and identify future research directions.

cs.CR↗

arXiv2019

Continuous-Time Markov Decision Processes with Controlled Observations

In this paper, we study a continuous-time discounted jump Markov decision process with both controlled actions and observations. The observation is only available for a discrete set of time instances. At each time of observation, one has to select an optimal timing for the next observation and a control trajectory for the time interval between two observation points. We provide a theoretical framework that the decision maker can utilize to find the optimal observation epochs and the optimal actions jointly. Two cases are investigated. One is gated queueing systems in which we explicitly characterize the optimal action and the optimal observation where the optimal observation is shown to be independent of the state. Another is the inventory control problem with Poisson arrival process in which we obtain numerically the optimal action and observation. The results show that it is optimal to observe more frequently at a region of states where the optimal action adapts constantly.

math.OC↗

arXiv2019

A Formal Approach for Efficient Navigation Management of Hybrid Electric Vehicles on Long Trips

Plug-in Hybrid Electric Vehicles (PHEVs) are gaining popularity due to their economic efficiency as well as their contribution to green management. PHEVs allow the driver to use electric power exclusively for driving and then switch to gasoline as needed. The more gasoline a vehicle uses, the higher cost is required for the trip. However, a PHEV cannot last for a long period on stored electricity without being recharged. Thus, it needs frequent recharging compared to traditional gasoline-powered vehicles. Moreover, the battery recharging time is usually long, which leads to longer delays on a trip. Therefore, it is necessary to provide a flexible navigation management scheme along with an efficient recharging schedule, which allows the driver to choose an optimal route based on the fuel-cost and time-to-destination constraints. In this paper, we present a formal model to solve this PHEV navigation management problem. The model is solved to provide a driver with a comprehensive routing plan including the potential recharging and refueling points that satisfy the given requirements, particularly the maximum fuel cost and the maximum trip time. In addition, we propose a price-based navigation control technique to achieve better load balance for the traffic system. Evaluation results show that the proposed formal models can be solved efficiently even with large road networks.

cs.LO↗

arXiv2019

Strategic Learning for Active, Adaptive, and Autonomous Cyber Defense

The increasing instances of advanced attacks call for a new defense paradigm that is active, autonomous, and adaptive, named as the \texttt{`3A'} defense paradigm. This chapter introduces three defense schemes that actively interact with attackers to increase the attack cost and gather threat information, i.e., defensive deception for detection and counter-deception, feedback-driven Moving Target Defense (MTD), and adaptive honeypot engagement. Due to the cyber deception, external noise, and the absent knowledge of the other players' behaviors and goals, these schemes possess three progressive levels of information restrictions, i.e., from the parameter uncertainty, the payoff uncertainty, to the environmental uncertainty. To estimate the unknown and reduce uncertainty, we adopt three different strategic learning schemes that fit the associated information restrictions. All three learning schemes share the same feedback structure of sensation, estimation, and actions so that the most rewarding policies get reinforced and converge to the optimal ones in autonomous and adaptive fashions. This work aims to shed lights on proactive defense strategies, lay a solid foundation for strategic learning under incomplete information, and quantify the tradeoff between the security and costs.

cs.CR↗

arXiv2019

A Dynamic Game Approach to Strategic Design of Secure and Resilient Infrastructure Network

Infrastructure networks are vulnerable to both cyber and physical attacks. Building a secure and resilient networked system is essential for providing reliable and dependable services. To this end, we establish a two-player three-stage game framework to capture the dynamics in the infrastructure protection and recovery phases. Specifically, the goal of the infrastructure network designer is to keep the network connected before and after the attack, while the adversary aims to disconnect the network by compromising a set of links. With costs for creating and removing links, the two players aim to maximize their utilities while minimizing the costs. In this paper, we use the concept of subgame perfect equilibrium (SPE) to characterize the optimal strategies of the network defender and attacker. We derive the SPE explicitly in terms of system parameters. We further investigate the resilience planning of the defender and the strategic timing of attack of the adversary. Finally, we use case studies of UAV-enabled communication networks for disaster recovery to corroborate the obtained analytical results.

eess.SY↗

arXiv2019

A Games-in-Games Approach to Mosaic Command and Control Design of Dynamic Network-of-Networks for Secure and Resilient Multi-Domain Operations

This paper presents a games-in-games approach to provide design guidelines for mosaic command and control that enables the secure and resilient multi-domain operations. Under the mosaic design, pieces or agents in the network are equipped with flexible interoperability and the capability of self-adaptability, self-healing, and resiliency so that they can reconfigure their responses to achieve the global mission in spite of failures of nodes and links in the adversarial environment. The proposed games-in-games approach provides a system-of-systems science for mosaic distributed design of large-scale systems. Specifically, the framework integrates three layers of design for each agent including strategic layer, tactical layer, and mission layer. Each layer in the established model corresponds to a game of a different scale that enables the integration of threat models and achieve self-mitigation and resilience capabilities. The solution concept of the developed multi-layer multi-scale mosaic design is characterized by Gestalt Nash equilibrium (GNE) which considers the interactions between agents across different layers. The developed approach is applicable to modern battlefield networks which are composed of heterogeneous assets that access highly diverse and dynamic information sources over multiple domains. By leveraging mosaic design principles, we can achieve the desired operational goals of deployed networks in a case study and ensure connectivity among entities for the exchange of information to accomplish the mission.

eess.SY↗

arXiv2019

A Game-Theoretic Taxonomy and Survey of Defensive Deception for Cybersecurity and Privacy

Cyberattacks on both databases and critical infrastructure have threatened public and private sectors. Ubiquitous tracking and wearable computing have infringed upon privacy. Advocates and engineers have recently proposed using defensive deception as a means to leverage the information asymmetry typically enjoyed by attackers as a tool for defenders. The term deception, however, has been employed broadly and with a variety of meanings. In this paper, we survey 24 articles from 2008-2018 that use game theory to model defensive deception for cybersecurity and privacy. Then we propose a taxonomy that defines six types of deception: perturbation, moving target defense, obfuscation, mixing, honey-x, and attacker engagement. These types are delineated by their information structures, agents, actions, and duration: precisely concepts captured by game theory. Our aims are to rigorously define types of defensive deception, to capture a snapshot of the state of the literature, to provide a menu of models which can be used for applied research, and to identify promising areas for future work. Our taxonomy provides a systematic foundation for understanding different types of defensive deception commonly encountered in cybersecurity and privacy.

cs.CR↗