arXiv subjects

P. R. Kumar

Publications and source records attributed to P. R. Kumar.

At least 73 records · Page 4Linked to original sources

arXiv2016

Belief Space Planning Simplified: Trajectory-Optimized LQG (T-LQG) (Extended Report)

Planning under motion and observation uncertainties requires solution of a stochastic control problem in the space of feedback policies. In this paper, we reduce the general (n^2+n)-dimensional belief space planning problem to an (n)-dimensional problem by obtaining a Linear Quadratic Gaussian (LQG) design with the best nominal performance. Then, by taking the underlying trajectory of the LQG controller as the decision variable, we pose a coupled design of trajectory, estimator, and controller design through a Non-Linear Program (NLP) that can be solved by a general NLP solver. We prove that under a first-order approximation and a careful usage of the separation principle, our approximations are valid. We give an analysis on the existing major belief space planning methods and show that our algorithm has the lowest computational burden. Finally, we extend our solution to contain general state and control constraints. Our simulation results support our design.

cs.RO↗

arXiv2016

Decentralized Control via Dynamic Stochastic Prices: The Independent System Operator Problem

A smart grid connects wind or solar or storage farms, fossil fuel plants, industrialor commercial loads, or load serving entities, modeled as stochastic dynamical systems. In each time period, they consume or supply electrical energy, with the constraint that total generation equals consumption. Each agent's utility is either the benefit accrued from consumption, or negative of generation cost. The Independent System Operator has to maximize their sum, the social welfare, without agents revealing their dynamic models or utilities. It has to announce prices after interacting with agents via bids. If agents observe and know the laws of uncertainties affecting other agents, then there is an iterative price and bid interaction that leads to the maximum social welfare attainable if agents pooled their information. In the important case where agents are LQG systems not even knowing of the existence of other agents, the bid and price iteration is dramatically simple, exchanging time vectors of future prices and consumptions or generations at each time step. State dependent bidding is not needed. This solution of the decentralized stochastic control problem may be of economic importance in power systems, and of broader interest in general equilibrium theory of economics for stochastic dynamic agents.

math.OC↗

arXiv2016

Dynamic Watermarking: Active Defense of Networked Cyber-Physical Systems

The coming decades may see the large scale deployment of networked cyber-physical systems to address global needs in areas such as energy, water, healthcare, and transportation. However, as recent events have shown, such systems are vulnerable to cyber attacks. Being safety critical, their disruption or misbehavior can cause economic losses or injuries and loss of life. It is therefore important to secure such networked cyber-physical systems against attacks. In the absence of credible security guarantees, there will be resistance to the proliferation of cyber-physical systems, which are much needed to meet global needs in critical infrastructures and services. This paper addresses the problem of secure control of networked cyber-physical systems. This problem is different from the problem of securing the communication network, since cyber-physical systems at their very essence need sensors and actuators that interface with the physical plant, and malicious agents may tamper with sensors or actuators, as recent attacks have shown. We consider physical plants that are being controlled by multiple actuators and sensors communicating over a network, where some sensors could be "malicious," meaning that they may not report the measurements that they observe. We address a general technique by which the actuators can detect the actions of malicious sensors in the system, and disable closed-loop control based on their information. This technique, called "watermarking," employs the technique of actuators injecting private excitation into the system which will reveal malicious tampering with signals. We show how such an active defense can be used to secure networked systems of sensors and actuators.

eess.SY↗

arXiv2016

Throughput Optimal Decentralized Scheduling of Multi-Hop Networks with End-to-End Deadline Constraints: Unreliable Links

We consider unreliable multi-hop networks serving multiple flows in which packets not delivered to their destination nodes by their deadlines are dropped. We address the design of policies for routing and scheduling packets that optimize any specified weighted average of the throughputs of the flows. We provide a new approach which directly yields an optimal distributed scheduling policy that attains any desired maximal timely-throughput vector under average-power constraints on the nodes. It pursues a novel intrinsically stochastic decomposition of the Lagrangian of the constrained network-wide MDP rather than of the fluid model. All decisions regarding a packet's transmission scheduling, transmit power level, and routing, are completely distributed, based solely on the age of the packet, not requiring any knowledge of network state or queue lengths at any of the nodes. Global coordination is achieved through a tractably computable "price" for transmission energy. This price is different from that used to derive the backpressure policy where price corresponds to queue lengths. A quantifiably near-optimal policy is provided if nodes have peak-power constraints.

cs.NI↗

arXiv2016

Feedback Motion Planning Under Non-Gaussian Uncertainty and Non-Convex State Constraints

Planning under process and measurement uncertainties is a challenging problem. In its most general form it can be modeled as a Partially Observed Markov Decision Process (POMDP) problem. However POMDPs are generally difficult to solve when the underlying spaces are continuous, particularly when beliefs are non-Gaussian, and the difficulty is further exacerbated when there are also non-convex constraints on states. Existing algorithms to address such challenging POMDPs are expensive in terms of computation and memory. In this paper, we provide a feedback policy in non-Gaussian belief space via solving a convex program for common non-linear observation models. The solution involves a Receding Horizon Control strategy using particle filters for the non-Gaussian belief representation. We develop a way of capturing non-convex constraints in the state space and adapt the optimization to incorporate such constraints, as well. A key advantage of this method is that it does not introduce additional variables in the optimization problem and is therefore more scalable than existing constrained problems in belief space. We demonstrate the performance of the method on different scenarios.

cs.RO↗

arXiv2015

The ISO Problem: Decentralized Stochastic Control via Bidding Schemes

We consider a smart-grid connecting several agents, modeled as stochastic dynamical systems, who may be electricity consumers/producers. At each discrete time instant, which may represent a 15 minute interval, each agent may consume/generate some quantity of electrical energy. The Independent System Operator (ISO) is given the task of assigning consumptions/generations to the agents so as to maximize the sum of the utilities accrued to the agents, subject to the constraint that energy generation equals consumption at each time. This task of coordinating generation and demand has to be accomplished by the ISO without the agents revealing their system states, dynamics, or utility/cost functions. We show how and when a simple iterative procedure converges to the optimal solution. The ISO iteratively obtains electricity bids by the agents, and declares the tentative market clearing prices. In response to these prices, the agents submit new bids. On the demand side, the solution yields an optimal demand response for dynamic and stochastic loads. On the generation side, it provides the optimal utilization of stochastically varying renewables such as solar/wind, and generation with fossil fuel based generation with dynamic constraints such as ramping rates. Thereby we solve a decentralized stochastic control problem, without agents sharing any information about their system models, states or utility functions.

eess.SY↗

arXiv2015

Decentralized Throughput Maximizing Policies for Deadline-Constrained Wireless Networks

We consider multi-hop wireless networks serving multiple flows in which only packets that meet hard end-to-end deadline constraints are useful, i.e., if a packet is not delivered to its destination node by its deadline, it is dropped from the network. We design decentralized scheduling policies for such multi-hop networks that attain the maximum throughput of useful packets. The resulting policy is decentralized in the sense that in order to make a transmission decision, a node only needs to know the "time-till-deadline" of the packets that are currently present at that node, and not the state of the entire network. The key to obtaining an easy-to-implement and highly decentralized policy is to replace the hard constraint on the number of simultaneous packet transmissions that can take place on the outgoing links of a node, by a time-average constraint on the number of transmissions. The policy thus obtained is guaranteed to provide maximum throughput. Analysis can be extended to the case of time-varying channel conditions in a straightforward manner. Simulations showing significant improvement over existing policies for deadline based scheduling, such as Earliest Deadline First, and supporting the theory, are presented.

cs.NI↗

arXiv2015

Optimizing Quality of Experience of Dynamic Video Streaming over Fading Wireless Networks

We address the problem of video streaming packets from an Access Point (AP) to multiple clients over a shared wireless channel with fading. In such systems, each client maintains a buffer of packets from which to play the video, and an outage occurs in the streaming whenever the buffer is empty. Clients can switch to a lower-quality of video packet, or request packet transmission at a higher energy level in order to minimize the number of outages plus the number of outage periods and the number of low quality video packets streamed, while there is an average power constraint on the AP. We pose the problem of choosing the video quality and transmission power as a Constrained Markov Decision Process (CMDP). We show that the problem involving $N$ clients decomposes into $N$ MDPs, each involving only a single client, and furthermore that the optimal policy has a threshold structure, in which the decision to choose the video-quality and power-level of transmission depends solely on the buffer-level.

cs.NI↗

arXiv2015

A Theory for the Operation of the Independent System Operator in a Smart Grid with Stochastic Renewables, Demand Response and Storage

In this paper, we address a key issue of designing architectures and algorithms which generate optimal demand response in a decentralized manner for a smart-grid consisting of several stochastic renewables and dynamic loads. By optimal demand response, we refer to the demand response which maximizes the utility of the agents connected to the smart-grid. By decentralized we refer to the desirable case where neither the independent system operator (ISO) needs to know the dynamics/utilities of the agents, nor do the agents need to have a knowledge of the dynamics/utilities of other agents connected to the grid. The communication between the ISO and agents is restricted to the ISO announcing a pricing policy and the agents responding with their energy generation/consumption bids in response to the pricing policy. We provide a complete solution for both the deterministic and stochastic cases. It features a price iteration scheme that results in optimality of social welfare. We also provide an optimal solution for the case where there is a common randomness affecting and observed by all agents. This solution can be computationally complex, and we pose approximations. For the more general partially observed randomness case, we exhibit a relaxation that significantly reduces complexity. We also provide an approximation strategy that leads to a model predictive control (MPC) approach. Simulation results comparing the resulting optimal demand response with the existing architectures employed by the ISO illustrate the benefit in social welfare utility realized by our scheme. To the best of the authors' knowledge, this is the first work of its kind to explicitly mark out the optimal response of dynamic demand.

eess.SY↗

arXiv2015

A High Reliability Asymptotic Approach for Packet Inter-Delivery Time Optimization in Cyber-Physical Systems

In cyber-physical systems such as automobiles, measurement data from sensor nodes should be delivered to other consumer nodes such as actuators in a regular fashion. But, in practical systems over unreliable media such as wireless, it is a significant challenge to guarantee small enough inter-delivery times for different clients with heterogeneous channel conditions and inter-delivery requirements. In this paper, we design scheduling policies aiming at satisfying the inter-delivery requirements of such clients. We formulate the problem as a risk-sensitive Markov Decision Process (MDP). Although the resulting problem involves an infinite state space, we first prove that there is an equivalent MDP involving only a finite number of states. Then we prove the existence of a stationary optimal policy and establish an algorithm to compute it in a finite number of steps. However, the bane of this and many similar problems is the resulting complexity, and, in an attempt to make fundamental progress, we further propose a new high reliability asymptotic approach. In essence, this approach considers the scenario when the channel failure probabilities for different clients are of the same order, and asymptotically approach zero. We thus proceed to determine the asymptotically optimal policy: in a two-client scenario, we show that the asymptotically optimal policy is a "modified least time-to-go" policy, which is intuitively appealing and easily implementable; in the general multi-client scenario, we are led to an SN policy, and we develop an algorithm of low computational complexity to obtain it. Simulation results show that the resulting policies perform well even in the pre-asymptotic regime with moderate failure probabilities.

cs.NI↗

arXiv2015

Optimal Energy-Efficient Regular Delivery of Packets in Cyber-Physical Systems

In cyber-physical systems such as in-vehicle wireless sensor networks, a large number of sensor nodes continually generate measurements that should be received by other nodes such as actuators in a regular fashion. Meanwhile, energy-efficiency is also important in wireless sensor networks. Motivated by these, we develop scheduling policies which are energy efficient and simultaneously maintain "regular" deliveries of packets. A tradeoff parameter is introduced to balance these two conflicting objectives. We employ a Markov Decision Process (MDP) model where the state of each client is the time-since-last-delivery of its packet, and reduce it into an equivalent finite-state MDP problem. Although this equivalent problem can be solved by standard dynamic programming techniques, it suffers from a high-computational complexity. Thus we further pose the problem as a restless multi-armed bandit problem and employ the low-complexity Whittle Index policy. It is shown that this problem is indexable and the Whittle indexes are derived. Also, we prove the Whittle Index policy is asymptotically optimal and validate its optimality via extensive simulations.

eess.SY↗

arXiv2015

Index Policies for Optimal Mean-Variance Trade-Off of Inter-delivery Times in Real-Time Sensor Networks

A problem of much current practical interest is the replacement of the wiring infrastructure connecting approximately 200 sensor and actuator nodes in automobiles by an access point. This is motivated by the considerable savings in automobile weight, simplification of manufacturability, and future upgradability. A key issue is how to schedule the nodes on the shared access point so as to provide regular packet delivery. In this and other similar applications, the mean of the inter-delivery times of packets, i.e., throughput, is not sufficient to guarantee service-regularity. The time-averaged variance of the inter-delivery times of packets is also an important metric. So motivated, we consider a wireless network where an Access Point schedules real-time generated packets to nodes over a fading wireless channel. We are interested in designing simple policies which achieve optimal mean-variance tradeoff in interdelivery times of packets by minimizing the sum of time-averaged means and variances over all clients. Our goal is to explore the full range of the Pareto frontier of all weighted linear combinations of mean and variance so that one can fully exploit the design possibilities. We transform this problem into a Markov decision process and show that the problem of choosing which node's packet to transmit in each slot can be formulated as a bandit problem. We establish that this problem is indexable and explicitly derive the Whittle indices. The resulting Index policy is optimal in certain cases. We also provide upper and lower bounds on the cost for any policy. Extensive simulations show that Index policies perform better than previously proposed policies.

cs.NI↗

arXiv2013

Model-based clock synchronization protocol for wireless sensor networks

In a wireless sensor network, nodes communicate time-stamped packets in order to synchronize their clocks, i.e., estimate each other's time display. We introduce and analyze a parametrized stochastic model for clocks and use it to calculate, at any given time, the Maximum-Likelihood (ML) estimate of the relative skew/offset between two communicating nodes. For network synchronization, event-based Kalman-Bucy filtering gives rise to a centralized scheme, and we propose an efficient distributed suboptimal algorithm. We study the performance both analytically and experimentally and provide provable guarantees. We summarize our findings into defining a new distributed model-based clock synchronization protocol (MBCSP), and present a comparative simulation study of its accuracy versus prior art to showcase improvements.

cs.NI↗

arXiv2013

Pathwise Performance of Debt Based Policies for Wireless Networks with Hard Delay Constraints

Hou et al have introduced a framework to serve clients over wireless channels when there are hard deadline constraints along with a minimum delivery ratio for each client's flow. Policies based on "debt," called maximum debt first policies (MDF) were introduced, and shown to be throughput optimal. By "throughput optimality" it is meant that if there exists a policy that fulfils a set of clients with a given vector of delivery ratios and a vector of channel reliabilities, then the MDF policy will also fulfill them. The debt of a user is the difference between the number of packets that should have been delivered so as to meet the delivery ratio and the number of packets that have been delivered for that client. The maximum debt first (MDF) prioritizes the clients in decreasing order of debts at the beginning of every period. Note that a throughput optimal policy only guarantees that \begin{small} $\liminf_{T \to \infty} \frac{1}{T}\sum_{t=1}^{T} \mathbbm{1}\{\{client $n$'s packet is delivered in frame $t$} \} \geq q_{i}$ \end{small}, where the right hand side is the required delivery ratio for client $i$. Thus, it only guarantees that the debts of each user are $o(T)$, and can be otherwise arbitrarily large. This raises the interesting question about what is the growth rate of the debts under the MDF policy. We show the optimality of MDF policy in the case when the channel reliabilities of all users are same, and obtain performance bounds for the general case. For the performance bound we obtain the almost sure bounds on $\limsup_{t\to\infty}\frac{d_{i}(t)}{ϕ(t)}$ for all $i$, where $ϕ(t) = \sqrt{2t\log\log t}$.

cs.NI↗

arXiv2013

A System-Theoretic Clean Slate Approach to Provably Secure Ad Hoc Wireless Networking

Traditionally, wireless network protocols have been designed for performance. Subsequently, as attacks have been identified, patches have been developed. This has resulted in an "arms race" development process of discovering vulnerabilities and then patching them. The fundamental difficulty with this approach is that other vulnerabilities may still exist. No provable security or performance guarantees can ever be provided. We develop a system-theoretic approach to security that provides a complete protocol suite with provable guarantees, as well as proof of min-max optimality with respect to any given utility function of source-destination rates. Our approach is based on a model capturing the essential features of an adhoc wireless network that has been infiltrated with hostile nodes. We consider any collection of nodes, some good and some bad, possessing specified capabilities vis-a-vis cryptography, wireless communication and clocks. The good nodes do not know the bad nodes. The bad nodes can collaborate perfectly, and are capable of any disruptive acts ranging from simply jamming to non-cooperation with the protocols in any manner they please. The protocol suite caters to the complete life-cycle, all the way from birth of nodes, through all phases of ad hoc network formation, leading to an optimized network carrying data reliably. It provably achieves the min-max of the utility function, where the max is over all protocol suites published and followed by the good nodes, while the min is over all Byzantine behaviors of the bad nodes. Under the protocol suite, the bad nodes do not benefit from any actions other than jamming or cooperating. This approach supersedes much previous work that deals with several types of attacks.

cs.NI↗

arXiv2012

Bounded epsilon-Reach Set Computation of a Class of Deterministic and Transversal Linear Hybrid Automata

We define a special class of hybrid automata, called Deterministic and Transversal Linear Hybrid Automata (DTLHA), whose continuous dynamics in each location are linear time-invariant (LTI) with a constant input, and for which every discrete transition up to a given bounded time is deterministic and, importantly, transversal. For such a DTLHA starting from an initial state, we show that it is possible to compute an approximation of the reach set of a DTLHA over a finite time interval that is arbitrarily close to the exact reach set, called a bounded epsilon-reach set, through sampling and polyhedral over-approximation of sampled states. We propose an algorithm and an attendant architecture for the overall bounded epsilon-reach set computation process.

eess.SY↗

arXiv2012

Near-optimal quantization and linear network coding for relay networks

We introduce a discrete network corresponding to any Gaussian wireless network that is obtained by simply quantizing the received signals and restricting the transmitted signals to a finite precision. Since signals in the discrete network are obtained from those of a Gaussian network, the Gaussian network can be operated on the quantization-based digital interface defined by the discrete network. We prove that this digital interface is near-optimal for Gaussian relay networks and the capacities of the Gaussian and the discrete networks are within a bounded gap of O(M^2) bits, where M is the number of nodes. We prove that any near-optimal coding strategy for the discrete network can be naturally transformed into a near-optimal coding strategy for the Gaussian network merely by quantization. We exploit this by designing a linear coding strategy for the case of layered discrete relay networks. The linear coding strategy is near-optimal for Gaussian and discrete networks and achieves rates within O(M^2) bits of the capacity, independent of channel gains or SNR. The linear code is robust and the relays need not know the channel gains. The transmit and receive signals at all relays are simply quantized to binary tuples of the same length $n$ . The linear network code requires all the relay nodes to collect the received binary tuples into a long binary vector and apply a linear transformation on the long vector. The resulting binary vector is split into smaller binary tuples for transmission by the relays. The quantization requirements of the linear network code are completely defined by the parameter $n$, which also determines the resolution of the analog-to-digital and digital-to-analog convertors for operating the network within a bounded gap of the network's capacity. The linear network code explicitly connects network coding for wireline networks with codes for Gaussian networks.

cs.IT↗

arXiv2011

Optimal Function Computation in Directed and Undirected Graphs

We consider the problem of information aggregation in sensor networks, where one is interested in computing a function of the sensor measurements. We allow for block processing and study in-network function computation in directed graphs and undirected graphs. We study how the structure of the function affects the encoding strategies, and the effect of interactive information exchange. We begin by considering a directed graph G = (V, E) on the sensor nodes, where the goal is to determine the optimal encoders on each edge which achieve function computation at the collector node. Our goal is to characterize the rate region in R^{|E|}, i.e., the set of points for which there exist feasible encoders with given rates which achieve zero-error computation for asymptotically large block length. We determine the solution for directed trees, specifying the optimal encoder and decoder for each edge. For general directed acyclic graphs, we provide an outer bound on the rate region by finding the disambiguation requirements for each cut, and describe examples where this outer bound is tight. Next, we address the scenario where nodes are connected in an undirected tree network, and every node wishes to compute a given symmetric Boolean function of the sensor data. Undirected edges permit interactive computation, and we therefore study the effect of interaction on the aggregation and communication strategies. We focus on sum-threshold functions, and determine the minimum worst-case total number of bits to be exchanged on each edge. The optimal strategy involves recursive in-network aggregation which is reminiscent of message passing. In the case of general graphs, we present a cutset lower bound, and an achievable scheme based on aggregation along trees. For complete graphs, we prove that the complexity of this scheme is no more than twice that of the optimal scheme.

cs.IT↗