arXiv subjects

Aditya Mahajan

Publications and source records attributed to Aditya Mahajan.

At least 55 records · Page 3Linked to original sources

arXiv2018

Linear Quadratic Mean Field Teams: Optimal and Approximately Optimal Decentralized Solutions

We consider team optimal control of decentralized systems with linear dynamics, quadratic costs, and arbitrary disturbance that consist of multiple sub-populations with exchangeable agents (i.e., exchanging two agents within the same sub-population does not affect the dynamics or the cost). Such a system is equivalent to one where the dynamics and costs are coupled across agents through the mean-field (or empirical mean) of the states and actions (even when the primitive random variables are non-exchangeable). Two information structures are investigated. In the first, all agents observe their local state and the mean-field of all sub-populations, in the second, all agents observe their local state but the mean-field of only a subset of the sub-populations. Both information structures are non-classical and not partially nested. Nonetheless, it is shown that linear control strategies are optimal for the first and approximately optimal for the second, the approximation error is inversely proportional to the size of the sub-populations whose mean-fields are not observed. The corresponding gains are determined by the solution of K+1 decoupled standard Riccati equations, where K is the number of sub-populations. The dimensions of the Riccati equations do not depend on the size of the sub-populations, thus the solution complexity is independent of the number of agents. Generalizations to major-minor agents, tracking cost, weighted mean-field, and infinite horizon are provided. The results are illustrated using an example of demand response in smart grids.

eess.SY↗

arXiv2018

Remote estimation over a packet-drop channel with Markovian state

We investigate a remote estimation problem in which a transmitter observes a Markov source and chooses the power level to transmit it over a time-varying packet-drop channel. The channel is modeled as a channel with Markovian state where the packet drop probability depends on the channel state and the transmit power. A receiver observes the channel output and the channel state and estimates the source realization. The receiver also feeds back the channel state and an acknowledgment for successful reception to the transmitter. We consider two models for the source---finite state Markov chains and first-order autoregressive processes. For the first model, using ideas from team theory, we establish the structure of optimal transmission and estimation strategies and identify a dynamic program to determine optimal strategies with that structure. For the second model, we assume that the noise process has unimodal and symmetric distribution. Using ideas from majorization theory, we show that the optimal transmission strategy is symmetric and monotonic and the optimal estimation strategy is like Kalman filter. Consequently, when there are a finite number of power levels, the optimal transmission strategy may be described using thresholds that depend on the channel state. Finally, we propose a simulation based approach (Renewal Monte Carlo) to compute the optimal thresholds and optimal performance and elucidate the algorithm with an example.

eess.SY↗

arXiv2018

Renewal Monte Carlo: Renewal theory based reinforcement learning

In this paper, we present an online reinforcement learning algorithm, called Renewal Monte Carlo (RMC), for infinite horizon Markov decision processes with a designated start state. RMC is a Monte Carlo algorithm and retains the advantages of Monte Carlo methods including low bias, simplicity, and ease of implementation while, at the same time, circumvents their key drawbacks of high variance and delayed (end of episode) updates. The key ideas behind RMC are as follows. First, under any reasonable policy, the reward process is ergodic. So, by renewal theory, the performance of a policy is equal to the ratio of expected discounted reward to the expected discounted time over a regenerative cycle. Second, by carefully examining the expression for performance gradient, we propose a stochastic approximation algorithm that only requires estimates of the expected discounted reward and discounted time over a regenerative cycle and their gradients. We propose two unbiased estimators for evaluating performance gradients---a likelihood ratio based estimator and a simultaneous perturbation based estimator---and show that for both estimators, RMC converges to a locally optimal policy. We generalize the RMC algorithm to post-decision state models and also present a variant that converges faster to an approximately optimal policy. We conclude by presenting numerical experiments on a randomly generated MDP, event-triggered communication, and inventory management.

cs.LG↗

arXiv2017

Information-Theoretic Privacy for Smart Metering Systems with a Rechargeable Battery

Smart-metering systems report electricity usage of a user to the utility provider on almost real-time basis. This could leak private information about the user to the utility provider. In this work we investigate the use of a rechargeable battery in order to provide privacy to the user. We assume that the user load sequence is a first-order Markov process, the battery satisfies ideal charge conservation, and that privacy is measured using normalized mutual information (leakage rate) between the user load and the battery output. We consider battery charging policies in this setup that satisfy the feasibility constraints. We propose a series reductions on the original problem and ultimately recast it as a Markov Decision Process (MDP) that can be solved using a dynamic program. In the special case of i.i.d. demand, we explicitly characterize the optimal policy and show that the associated leakage rate can be expressed as a single-letter mutual information expression. In this case we show that the optimal charging policy admits an intuitive interpretation of preserving a certain invariance property of the state. Interestingly an alternative proof of optimality can be provided that does not rely on the MDP approach, but is based on purely information theoretic reductions.

cs.IT↗

arXiv2017

Sufficient conditions for the value function and optimal strategy to be even and quasi-convex

Sufficient conditions are identified under which the value function and the optimal strategy of a Markov decision process (MDP) are even and quasi-convex in the state. The key idea behind these conditions is the following. First, sufficient conditions for the value function and optimal strategy to be even are identified. Next, it is shown that if the value function and optimal strategy are even, then one can construct a "folded MDP" defined only on the non-negative values of the state space. Then, the standard sufficient conditions for the value function and optimal strategy to be monotone are "unfolded" to identify sufficient conditions for the value function and the optimal strategy to be quasi-convex. The results are illustrated by using an example of power allocation in remote estimation.

math.OC↗

arXiv2017

Structure of optimal strategies for remote estimation over Gilbert-Elliott channel with feedback

We investigate remote estimation over a Gilbert-Elliot channel with feedback. We assume that the channel state is observed by the receiver and fed back to the transmitter with one unit delay. In addition, the transmitter gets ACK/NACK feedback for successful/unsuccessful transmission. Using ideas from team theory, we establish the structure of optimal transmission and estimation strategies and identify a dynamic program to determine optimal strategies with that structure. We then consider first-order autoregressive sources where the noise process has unimodal and symmetric distribution. Using ideas from majorization theory, we show that the optimal transmission strategy has a threshold structure and the optimal estimation strategy is Kalman-like.

math.OC↗

arXiv2016

Fundamental limits of remote estimation of autoregressive Markov processes under communication constraints

The fundamental limits of remote estimation of Markov processes under communication constraints are presented. The remote estimation system consists of a sensor and an estimator. The sensor observes a discrete-time Markov process, which is a symmetric countable state Markov source or a Gauss-Markov process. At each time, the sensor either transmits the current state of the Markov process or does not transmit at all. Communication is noiseless but costly. The estimator estimates the Markov process based on the transmitted observations. In such a system, there is a trade-off between communication cost and estimation accuracy. Two fundamental limits of this trade-off are characterized for infinite horizon discounted cost and average cost setups. First, when each transmission is costly, we characterize the minimum achievable cost of communication plus estimation error. Second, when there is a constraint on the average number of transmissions, we characterize the minimum achievable estimation error. Transmission and estimation strategies that achieve these fundamental limits are also identified.

math.OC↗

arXiv2014

Distortion-transmission trade-off in real-time transmission of Markov sources

The problem of optimal real-time transmission of a Markov source under constraints on the expected number of transmissions is considered, both for the discounted and long term average cases. This setup is motivated by applications where transmission is sporadic and the cost of switching on the radio and transmitting is significantly more important than the size of the transmitted data packet. For this model, we characterize the distortion-transmission function, i.e., the minimum expected distortion that can be achieved when the expected number of transmissions is less than or equal to a particular value. In particular, we show that the distortion-transmission function is a piecewise linear, convex, and decreasing function. We also give an explicit characterization of each vertex of the piecewise linear function. To prove the results, the optimization problem is cast as a decentralized constrained stochastic control problem. We first consider the Lagrange relaxation of the constrained problem and identify the structure of optimal transmission and estimation strategies. In particular, we show that the optimal transmission is of a threshold type. Using these structural results, we obtain dynamic programs for the Lagrange relaxations. We identify the performance of an arbitrary threshold-type transmission strategy and use the idea of calibration from multi-armed bandits to determine the optimal transmission strategy for the Lagrange relaxation. Finally, we show that the optimal strategy for the constrained setup is a randomized strategy that randomizes between two deterministic strategies that differ only at one state. By evaluating the performance of these strategies, we determine the shape of the distortion-transmission function. These results are illustrated using an example of transmitting a birth-death Markov source.

cs.IT↗

arXiv2014

Sufficient statistics for linear control strategies in decentralized systems with partial history sharing

In decentralized control systems with linear dynamics, quadratic cost, and Gaussian disturbance (also called decentralized LQG systems) linear control strategies are not always optimal. Nonetheless, linear control strategies are appealing due to analytic and implementation simplicity. In this paper, we investigate decentralized LQG systems with partial history sharing information structure and identify finite dimensional sufficient statistics for such systems. Unlike prior work on decentralized LQG systems, we do not assume partially nestedness or quadratic invariance. Our approach is based on the common information approach of Nayyar \emph{et al}, 2013 and exploits the linearity of the system dynamics and control strategies. To illustrate our methodology, we identify sufficient statistics for linear strategies in decentralized systems where controllers communicate over a strongly connected graph with finite delays, and for decentralized systems consisting of coupled subsystems with control sharing or one-sided one step delay sharing information structures.

math.OC↗

arXiv2013

Decentralized stochastic control

Decentralized stochastic control refers to the multi-stage optimization of a dynamical system by multiple controllers that have access to different information. Decentralization of information gives rise to new conceptual challenges that require new solution approaches. In this expository paper, we use the notion of an \emph{information-state} to explain the two commonly used solution approaches to decentralized control: the person-by-person approach and the common-information approach.

math.OC↗

arXiv2013

Forensic Analysis of Instant Messenger Applications on Android Devices

This paper focuses on conducting forensic data analysis of 2 widely used IMs applications on Android phones WhatsApp and Viber. The tests and analysis were performed with the aim of determining what data and information can be found on the devices internal memory for instant messengers eg chat messaging logs and history send & received image or video files etc. The experiments and results show that heavy amount of potential evidences and valuable data can be found on Android phones by forensic investigators.

cs.CY↗

arXiv2012

Decentralized Stochastic Control with Partial History Sharing: A Common Information Approach

A general model of decentralized stochastic control called partial history sharing information structure is presented. In this model, at each step the controllers share part of their observation and control history with each other. This general model subsumes several existing models of information sharing as special cases. Based on the information commonly known to all the controllers, the decentralized problem is reformulated as an equivalent centralized problem from the perspective of a coordinator. The coordinator knows the common information and select prescriptions that map each controller's local information to its control actions. The optimal control problem at the coordinator is shown to be a partially observable Markov decision process (POMDP) which is solved using techniques from Markov decision theory. This approach provides (a) structural results for optimal strategies, and (b) a dynamic program for obtaining optimal strategies for all controllers in the original decentralized problem. Thus, this approach unifies the various ad-hoc approaches taken in the literature. In addition, the structural results on optimal control strategies obtained by the proposed approach cannot be obtained by the existing generic approach (the person-by-person approach) for obtaining structural results in decentralized problems; and the dynamic program obtained by the proposed approach is simpler than that obtained by the existing generic approach (the designer's approach) for obtaining dynamic programs in decentralized problems.

eess.SY↗

arXiv2012

Optimal decentralized control of coupled subsystems with control sharing

Subsystems that are coupled due to dynamics and costs arise naturally in various communication applications. In many such applications the control actions are shared between different control stations giving rise to a \emph{control sharing} information structure. Previous studies of control-sharing have concentrated on the linear quadratic Gaussian setup and a solution approach tailored to continuous valued control actions. In this paper a three step solution approach for finite valued control actions is presented. In the first step, a person-by-person approach is used to identify redundant data or a sufficient statistic for local information at each control station. In the second step, the common-information based approach of Nayyar et al.\ (2011) is used to find a sufficient statistic for the common information shared between all control stations and to obtain a dynamic programming decomposition. In the third step, the specifics of the model are used to simplify the sufficient statistic and the dynamic program. As an example, an exact solution of a two-user multiple access broadcast system is presented.

eess.SY↗

arXiv2012

Age Based User Interface in Mobile Operating System

This paper proposes the creation of different interfaces in the mobile operating system for different age groups. The different age groups identified are kids, elderly people and all others. The motive behind creating different interfaces is to make the smartphones of today's world usable to all age groups.

cs.OS↗

arXiv2011

Opportunistic capacity and error exponent regions for compound channel with feedback

Variable length communication over a compound channel with feedback is considered. Traditionally, capacity of a compound channel without feedback is defined as the maximum rate that is determined before the start of communication such that communication is reliable. This traditional definition is pessimistic. In the presence of feedback, an opportunistic definition is given. Capacity is defined as the maximum rate that is determined at the end of communication such that communication is reliable. Thus, the transmission rate can adapt to the channel chosen by nature. Under this definition, feedback communication over a compound channel is conceptually similar to multi-terminal communication. Transmission rate is a vector rather than a scalar; channel capacity is a region rather than a scalar; error exponent is a region rather than a scalar. In this paper, variable length communication over a compound channel with feedback is formulated, its opportunistic capacity region is characterized, and lower bounds for its error exponent region are provided..

cs.IT↗

arXiv2010

Optimal sequential transmission over broadcast channel with nested feedback

We consider the optimal design of sequential transmission over broadcast channel with nested feedback. Nested feedback means that the channel output of the outer channel is also available at the decoder of the inner channel. We model the communication system as a decentralized team with three decision makers---the encoder and the two decoders. Structure of encoding and decoding strategies that minimize a total distortion measure over a finite horizon are determined. The results are applicable for real-time communication as well as for the information theoretic setup.

cs.IT↗

arXiv2010

Optimal Control Strategies in Delayed Sharing Information Structures

The $n$-step delayed sharing information structure is investigated. This information structure comprises of $K$ controllers that share their information with a delay of $n$ time steps. This information structure is a link between the classical information structure, where information is shared perfectly between the controllers, and a non-classical information structure, where there is no "lateral" sharing of information among the controllers. Structural results for optimal control strategies for systems with such information structures are presented. A sequential methodology for finding the optimal strategies is also derived. The solution approach provides an insight for identifying structural results and sequential decomposition for general decentralized stochastic control problems.

cs.OH↗

arXiv2007

On Real-Time Communication Systems with Noisy Feedback

We consider a real-time communication system with noisy feedback consisting of a Markov source, a forward and a backward discrete memoryless channels, and a receiver with finite memory. The objective is to design an optimal communication strategy (that is, encoding, decoding, and memory update strategies) to minimize the total expected distortion over a finite horizon. We present a sequential decomposition for the problem, which results in a set of nested optimality equations to determine optimal communication strategies. This provides a systematic methodology to determine globally optimal joint source-channel encoding and decoding strategies for real-time communication systems with noisy feedback.

cs.IT↗