arXiv subjects

Ashutosh Nayyar

Publications and source records attributed to Ashutosh Nayyar.

At least 55 records · Page 3Linked to original sources

arXiv2018

Optimal Local and Remote Controllers with Unreliable Uplink Channels

We consider a networked control system consisting of a remote controller and a collection of linear plants, each associated with a local controller. Each local controller directly observes the state of its co-located plant and can inform the remote controller of the plant's state through an unreliable uplink channel. We assume that the downlink channels from the remote controller to local controllers are perfect. The objective of the local controllers and the remote controller is to cooperatively minimize a quadratic performance cost. We provide a dynamic program for this decentralized control problem using the common information approach. Although our problem is not a partially nested problem, we obtain explicit optimal strategies for all controllers. In the optimal strategies, all controllers compute common estimates of the states of the plants based on the common information obtained from the communication network. The remote controller's action is linear in the common state estimates, and the action of each local controller is linear in both the actual state of its co-located plant and the common state estimates. We illustrate our results with numerical experiments using randomly generated models.

eess.SY↗

arXiv2018

Decentralized Control of Stochastically Switched Linear System with Unreliable Communication

We consider a networked control system (NCS) consisting of two plants, a global plant and a local plant, and two controllers, a global controller and a local controller. The global (resp. local) plant follows discrete-time stochastically switched linear dynamics with a continuous global (resp. local) state and a discrete global (resp. local) mode. We assume that the state and mode of the global plant are observed by both controllers while the state and mode of the local plant are only observed by the local controller. The local controller can inform the global controller of the local plant's state and mode through an unreliable TCP-like communication channel where successful transmissions are acknowledged. The objective of the controllers is to cooperatively minimize a modes-dependent quadratic cost over a finite time horizon. Following the method developed in [1] and [2], we construct a dynamic program based on common information and a decomposition of strategies, and use it to obtain explicit optimal strategies for the controllers. In the optimal strategies, both controllers compute a common estimate of the local plant's state. The global controller's action is linear in the state of the global plant and the common estimated state, and the local controller's action is linear in the actual states of both plants and the common estimated state. Furthermore, the gain matrices for the global controller depend on the global mode and its observation about the local mode, while the gain matrices for the local controller depend on the actual modes of both plants and the global controller's observation about the local mode.

eess.SY↗

arXiv2018

Optimal Auction Design for Flexible Consumers

We study the problem of designing revenue-maximizing auctions for allocating multiple goods to flexible consumers. In our model, each consumer is interested in a subset of goods known as its flexibility set and wants to consume one good from this set. A consumer's flexibility set and its utility from consuming a good from its flexibility set are its private information. We focus on the case of nested flexibility sets --- each consumer's flexibility set can be one of $k$ nested sets. We provide several examples where such nested flexibility sets may arise. We characterize the allocation rule for an incentive compatible, individually rational and revenue-maximizing auction as the solution to an integer program. The corresponding payment rule is described by an integral equation. We then leverage the nestedness of flexibility sets to simplify the optimal auction and provide a complete characterization of allocations and payments in terms of simple thresholds.

cs.GT↗

arXiv2017

Optimal Mechanism Design with Flexible Consumers and Costly Supply

The problem of designing a profit-maximizing, Bayesian incentive compatible and individually rational mechanism with flexible consumers and costly heterogeneous supply is considered. In our setup, each consumer is associated with a flexibility set that describes the subset of goods the consumer is equally interested in. Each consumer wants to consume one good from its flexibility set. The flexibility set of a consumer and the utility it gets from consuming a good from its flexibility set are its private information. We adopt the flexibility model of [1] and focus on the case of nested flexibility sets -- each consumer's flexibility set can be one of k nested sets. Examples of settings with this inherent nested structure are provided. On the supply side, we assume that the seller has an initial stock of free supply but it can purchase more goods for each of the nested sets at fixed exogenous prices. We characterize the allocation and purchase rules for a profit-maximizing, Bayesian incentive compatible and individually rational mechanism as the solution to an integer program. The optimal payment function is pinned down by the optimal allocation rule in the form of an integral equation. We show that the nestedness of flexibility sets can be exploited to obtain a simple description of the optimal allocations, purchases and payments in terms of thresholds that can be computed through a straightforward iterative procedure.

cs.GT↗

arXiv2017

Learning Unknown Markov Decision Processes: A Thompson Sampling Approach

We consider the problem of learning an unknown Markov Decision Process (MDP) that is weakly communicating in the infinite horizon setting. We propose a Thompson Sampling-based reinforcement learning algorithm with dynamic episodes (TSDE). At the beginning of each episode, the algorithm generates a sample from the posterior distribution over the unknown model parameters. It then follows the optimal stationary policy for the sampled model for the rest of the episode. The duration of each episode is dynamically determined by two stopping criteria. The first stopping criterion controls the growth rate of episode length. The second stopping criterion happens when the number of visits to any state-action pair is doubled. We establish $\tilde O(HS\sqrt{AT})$ bounds on expected regret under a Bayesian setting, where $S$ and $A$ are the sizes of the state and action spaces, $T$ is time, and $H$ is the bound of the span. This regret bound matches the best available bound for weakly communicating MDPs. Numerical results show it to perform better than existing algorithms for infinite horizon MDPs.

cs.LG↗

arXiv2016

Dynamic Teams and Decentralized Control Problems with Substitutable Actions

This paper considers two problems -- a dynamic team problem and a decentralized control problem. The problems we consider do not belong to the known classes of "simpler" dynamic team/decentralized control problems such as partially nested or quadratically invariant problems. However, we show that our problems admit simple solutions under an assumption referred to as the substitutability assumption. Intuitively, substitutability in a team (resp. decentralized control) problem means that the effects of one team member's (resp. controller's) action on the cost function and the information (resp. state dynamics) can be achieved by an action of another member (resp. controller). For the non-partially-nested LQG dynamic team problem, it is shown that under certain conditions linear strategies are optimal. For the non-partially-nested decentralized LQG control problem, the state structure can be exploited to obtain optimal control strategies with recursively update-able sufficient statistics. These results suggest that substitutability can work as a counterpart of the information structure requirements that enable simplification of dynamic teams and decentralized control problems.

eess.SY↗

arXiv2016

Optimal Local and Remote Controllers with Unreliable Communication

We consider a decentralized optimal control problem for a linear plant controlled by two controllers, a local controller and a remote controller. The local controller directly observes the state of the plant and can inform the remote controller of the plant state through a packet-drop channel. We assume that the remote controller is able to send acknowledgments to the local controller to signal the successful receipt of transmitted packets. The objective of the two controllers is to cooperatively minimize a quadratic performance cost. We provide a dynamic program for this decentralized control problem using the common information approach. Although our problem is not a partially nested LQG problem, we obtain explicit optimal strategies for the two controllers. In the optimal strategies, both controllers compute a common estimate of the plant state based on the common information. The remote controller's action is linear in the common estimated state, and the local controller's action is linear in both the actual state and the common estimated state.

eess.SY↗

arXiv2016

Decentralized Control Problems with Substitutable Actions

We consider a decentralized system with multiple controllers and define substitutability of one controller by another in open-loop strategies. We explore the implications of this property on the optimization of closed-loop strategies. In particular, we focus on the decentralized LQG problem with substitutable actions. Even though the problem we formulate does not belong to the known classes of "simpler" decentralized problems such as partially nested or quadratically invariant problems, our results show that, under the substitutability assumption, linear strategies are optimal and we provide a complete state space characterization of optimal strategies. We also identify a family of information structures that all give the same optimal cost as the centralized information structure under the substitutability assumption. Our results suggest that open-loop substitutability can work as a counterpart of the information structure requirements that enable simplification of decentralized control problems.

eess.SY↗

arXiv2014

Rate-constrained Energy Services: Allocation Policies and Market Decisions

The integration of renewable generation poses operational and economic challenges for the electricity grid. For the core problem of power balance, the legacy paradigm of tailoring supply to follow random demand may be inappropriate under deep penetration of uncertain and intermittent renewable generation. In this situation, there is an emerging consensus that the alternative approach of controlling demand to follow random supply offers compelling economic benefits in terms of reduced regulation costs. This approach exploits the flexibility of demand side resources and requires sensing, actuation, and communication infrastructure; distributed control algorithms; and viable schemes to compensate participating loads. This paper considers rate-constrained energy services which are a specific paradigm for flexible demand. These services are characterized by a specified delivery window, the total amount of energy that must be supplied over this window, and the maximum rate at which this energy may be delivered. We consider a forward market where rate-constrained energy services are traded. We explore allocation policies and market decisions of a supplier in this market. The supplier owns a generation mix that includes some uncertain renewable generation and may also purchase energy in day-ahead and real-time markets to meet customer demand. The supplier must optimally select the portfolio of rate-constrained services to sell, the amount of day-ahead energy to buy, and the policies for making real-time energy purchases and allocations to customers to maximize its expected profit. We offer solutions to the supplier's decision and control problems to economically provide rate constrained energy services.

eess.SY↗

arXiv2014

Duration-differentiated Energy Services with a Continuum of Loads

As the proportion of total power supplied by renewable sources increases, it gets more costly to use reserve generation to compensate for the variability of renewables like solar and wind. Hence attention has been drawn to exploiting flexibility in demand as a substitute for reserve generation. Flexibility has different attributes. In this paper we consider loads requiring a constant power for a specified duration (within say one day), whose flexibility resides in the fact that power may be delivered at any time so long as the total duration of service equals the load's specified duration. We give conditions under which a variable power supply is adequate to meet these flexible loads, and describe how to allocate the power to the loads. We also characterize the additional power needed when the supply is inadequate. We study the problem of allocating the available power to loads to maximize welfare, and show that the welfare optimum can be sustained as a competitive equilibrium in a forward market in which electricity is sold as service contracts differentiated by the duration of service and power level. We compare this forward market with a spot market in their ability to capture the flexiblity inherent in duration-differentiated loads.

eess.SY↗

arXiv2014

Optimal Control for LQG Systems on Graphs---Part I: Structural Results

In this two-part paper, we identify a broad class of decentralized output-feedback LQG systems for which the optimal control strategies have a simple intuitive estimation structure and can be computed efficiently. Roughly, we consider the class of systems for which the coupling of dynamics among subsystems and the inter-controller communication is characterized by the same directed graph. Furthermore, this graph is assumed to be a multitree, that is, its transitive reduction can have at most one directed path connecting each pair of nodes. In this first part, we derive sufficient statistics that may be used to aggregate each controller's growing available information. Each controller must estimate the states of the subsystems that it affects (its descendants) as well as the subsystems that it observes (its ancestors). The optimal control action for a controller is a linear function of the estimate it computes as well as the estimates computed by all of its ancestors. Moreover, these state estimates may be updated recursively, much like a Kalman filter.

eess.SY↗

arXiv2014

Duration-Differentiated Services in Electricity

The integration of renewable sources poses challenges at the operational and economic levels of the power grid. In terms of keeping the balance between supply and demand, the usual scheme of supply following load may not be appropriate for large penetration levels of uncertain and intermittent renewable supply. In this paper, we focus on an alternative scheme in which the load follows the supply, exploiting the flexibility associated with the demand side. We consider a model of flexible loads that are to be serviced by zero-marginal cost renewable power together with conventional generation if necessary. Each load demands 1 kW for a specified number of time slots within an operational period. The flexibility of a load resides in the fact that the service may be delivered over any slots within the operational period. Loads therefore require flexible energy services that are differentiated by the demanded duration. We focus on two problems associated with durations-differentiated loads. The first problem deals with the operational decisions that a supplier has to make to serve a given set of duration differentiated loads. The second problem focuses on a market implementation for duration differentiated services. We give necessary and sufficient conditions under which the available power can service the loads, and we describe an algorithm that constructs an appropriate allocation. In the event the available supply is inadequate, we characterize the minimum amount of power that must be purchased to service the loads. Next we consider a forward market where consumers can purchase duration differentiated energy services. We first characterize social welfare maximizing allocations in this forward market and then show the existence of an efficient competitive equilibrium.

eess.SY↗

arXiv2014

Signaling in sensor networks for sequential detection

Sequential detection problems in sensor networks are considered. The true state of nature/true hypothesis is modeled as a binary random variable $H$ with known prior distribution. There are $N$ sensors making noisy observations about the hypothesis; $\mathcal{N} =\{1,2,\ldots,N\}$ denotes the set of sensors. Sensor $i$ can receive messages from a subset $\mathcal{P}^i \subset \mathcal{N}$ of sensors and send a message to a subset $\mathcal{C}^i \subset \mathcal{N}$. Each sensor is faced with a stopping problem. At each time $t$, based on the observations it has taken so far and the messages it may have received, sensor $i$ can decide to stop and communicate a binary decision to the sensors in $\mathcal{C}^i$, or it can continue taking observations and receiving messages. After sensor $i$'s binary decision has been sent, it becomes inactive. Sensors incur operational costs (cost of taking observations, communication costs etc.) while they are active. In addition, the system incurs a terminal cost that depends on the true hypothesis $H$, the sensors' binary decisions and their stopping times. The objective is to determine decision strategies for all sensors to minimize the total expected cost.

eess.SY↗

arXiv2014

Sufficient statistics for linear control strategies in decentralized systems with partial history sharing

In decentralized control systems with linear dynamics, quadratic cost, and Gaussian disturbance (also called decentralized LQG systems) linear control strategies are not always optimal. Nonetheless, linear control strategies are appealing due to analytic and implementation simplicity. In this paper, we investigate decentralized LQG systems with partial history sharing information structure and identify finite dimensional sufficient statistics for such systems. Unlike prior work on decentralized LQG systems, we do not assume partially nestedness or quadratic invariance. Our approach is based on the common information approach of Nayyar \emph{et al}, 2013 and exploits the linearity of the system dynamics and control strategies. To illustrate our methodology, we identify sufficient statistics for linear strategies in decentralized systems where controllers communicate over a strongly connected graph with finite delays, and for decentralized systems consisting of coupled subsystems with control sharing or one-sided one step delay sharing information structures.

math.OC↗

arXiv2014

Common Information based Markov Perfect Equilibria for Linear-Gaussian Games with Asymmetric Information

We consider a class of two-player dynamic stochastic nonzero-sum games where the state transition and observation equations are linear, and the primitive random variables are Gaussian. Each controller acquires possibly different dynamic information about the state process and the other controller's past actions and observations. This leads to a dynamic game of asymmetric information among the controllers. Building on our earlier work on finite games with asymmetric information, we devise an algorithm to compute a Nash equilibrium by using the common information among the controllers. We call such equilibria common information based Markov perfect equilibria of the game, which can be viewed as a refinement of Nash equilibrium in games with asymmetric information. If the players' cost functions are quadratic, then we show that under certain conditions a unique common information based Markov perfect equilibrium exists. Furthermore, this equilibrium can be computed by solving a sequence of linear equations. We also show through an example that there could be other Nash equilibria in a game of asymmetric information, not corresponding to common information based Markov perfect equilibria.

eess.SY↗

arXiv2013

Structural Results and Explicit Solution for Two-Player LQG Systems on a Finite Time Horizon

It is well-known that linear dynamical systems with Gaussian noise and quadratic cost (LQG) satisfy a separation principle. Finding the optimal controller amounts to solving separate dual problems; one for control and one for estimation. For the discrete-time finite-horizon case, each problem is a simple forward or backward recursion. In this paper, we consider a generalization of the LQG problem in which there are two controllers. Each controller is responsible for one of two system inputs, but has access to different subsets of the available measurements. Our paper has three main contributions. First, we prove a fundamental structural result: sufficient statistics for the controllers can be expressed as conditional means of the global state. Second, we give explicit state-space formulae for the optimal controller. These formulae are reminiscent of the classical LQG solution with dual forward and backward recursions, but with the important difference that they are intricately coupled. Lastly, we show how these recursions can be solved efficiently, with computational complexity comparable to that of the centralized problem.

eess.SY↗

arXiv2012

Nash Equilibria for Stochastic Games with Asymmetric Information-Part 1: Finite Games

A model of stochastic games where multiple controllers jointly control the evolution of the state of a dynamic system but have access to different information about the state and action processes is considered. The asymmetry of information among the controllers makes it difficult to compute or characterize Nash equilibria. Using common information among the controllers, the game with asymmetric information is shown to be equivalent to another game with symmetric information. Further, under certain conditions, a Markov state is identified for the equivalent symmetric information game and its Markov perfect equilibria are characterized. This characterization provides a backward induction algorithm to find Nash equilibria of the original game with asymmetric information in pure or behavioral strategies. Each step of this algorithm involves finding Bayesian Nash equilibria of a one-stage Bayesian game. The class of Nash equilibria of the original game that can be characterized in this backward manner are named common information based Markov perfect equilibria.

cs.GT↗

arXiv2012

Decentralized Stochastic Control with Partial History Sharing: A Common Information Approach

A general model of decentralized stochastic control called partial history sharing information structure is presented. In this model, at each step the controllers share part of their observation and control history with each other. This general model subsumes several existing models of information sharing as special cases. Based on the information commonly known to all the controllers, the decentralized problem is reformulated as an equivalent centralized problem from the perspective of a coordinator. The coordinator knows the common information and select prescriptions that map each controller's local information to its control actions. The optimal control problem at the coordinator is shown to be a partially observable Markov decision process (POMDP) which is solved using techniques from Markov decision theory. This approach provides (a) structural results for optimal strategies, and (b) a dynamic program for obtaining optimal strategies for all controllers in the original decentralized problem. Thus, this approach unifies the various ad-hoc approaches taken in the literature. In addition, the structural results on optimal control strategies obtained by the proposed approach cannot be obtained by the existing generic approach (the person-by-person approach) for obtaining structural results in decentralized problems; and the dynamic program obtained by the proposed approach is simpler than that obtained by the existing generic approach (the designer's approach) for obtaining dynamic programs in decentralized problems.

eess.SY↗