arXiv subjects

Vijay Subramanian

Publications and source records attributed to Vijay Subramanian.

31 records · Page 2Linked to original sources

arXiv2021

Dynamic Games among Teams with Delayed Intra-Team Information Sharing

We analyze a class of stochastic dynamic games among teams with asymmetric information, where members of a team share their observations internally with a delay of $d$. Each team is associated with a controlled Markov Chain, whose dynamics are coupled through the players' actions. These games exhibit challenges in both theory and practice due to the presence of signaling and the increasing domain of information over time. We develop a general approach to characterize a subset of Nash Equilibria where the agents can use a compressed version of their information, instead of the full information, to choose their actions. We identify two subclasses of strategies: Sufficient Private Information Based (SPIB) strategies, which only compress private information, and Compressed Information Based (CIB) strategies, which compress both common and private information. We show that while SPIB-strategy-based equilibria always exist, the same is not true for CIB-strategy-based equilibria. We develop a backward inductive sequential procedure, whose solution (if it exists) provides a CIB strategy-based equilibrium. We identify some instances where we can guarantee the existence of a solution to the above procedure. Our results highlight the tension among compression of information, existence of (compression based) equilibria, and backward inductive sequential computation of such equilibria in stochastic dynamic games with asymmetric information.

cs.MA↗

arXiv2021

Restart perturbations for reversible Markov chains: trichotomy and pre-cutoff equivalence

Given a reversible Markov chain $P_n$ on $n$ states, and another chain $\tilde{P}_n$ obtained by perturbing each row of $P_n$ by at most $α_n$ in total variation, we study the total variation distance between the two stationary distributions, $\| π_n - \tildeπ_n \|$. We show that for chains with cutoff, $\| π_n - \tildeπ_n \|$ converges to $0$, $e^{-c}$, and $1$, respectively, if the product of $α_n$ and the mixing time of $P_n$ converges to $0$, $c$, and $\infty$, respectively. This echoes recent results for specific random walks that exhibit cutoff, suggesting that cutoff is the key property underlying such results. Moreover, we show $\| π_n - \tildeπ_n \|$ is maximized by restart perturbations, for which $\tilde{P}_n$ "restarts" $P_n$ at a random state with probability $α_n$ at each step. Finally, we show that pre-cutoff is (almost) equivalent to a notion of "sensitivity to restart perturbations," suggesting that chains with sharper convergence to stationarity are inherently less robust.

math.PR↗

arXiv2020

Empirical Policy Evaluation with Supergraphs

We devise and analyze algorithms for the empirical policy evaluation problem in reinforcement learning. Our algorithms explore backward from high-cost states to find high-value ones, in contrast to forward approaches that work forward from all states. While several papers have demonstrated the utility of backward exploration empirically, we conduct rigorous analyses which show that our algorithms can reduce average-case sample complexity from $O(S \log S)$ to as low as $O(\log S)$.

cs.LG↗

arXiv2019

On the role of clustering in Personalized PageRank estimation

Personalized PageRank (PPR) is a measure of the importance of a node from the perspective of another (we call these nodes the $\textit{target}$ and the $\textit{source}$, respectively). PPR has been used in many applications, such as offering a Twitter user (the source) recommendations of who to follow (targets deemed important by PPR); additionally, PPR has been used in graph-theoretic problems such as community detection. However, computing PPR is infeasible for large networks like Twitter, so efficient estimation algorithms are necessary. In this work, we analyze the relationship between PPR estimation complexity and clustering. First, we devise algorithms to estimate PPR for many source/target pairs. In particular, we propose an enhanced version of the existing single pair estimator $\texttt{Bidirectional-PPR}$ that is more useful as a primitive for many pair estimation. We then show that the common underlying graph can be leveraged to efficiently and jointly estimate PPR for many pairs, rather than treating each pair separately using the primitive algorithm. Next, we show the complexity of our joint estimation scheme relates closely to the degree of clustering among the sources and targets at hand, indicating that estimating PPR for many pairs is easier when clustering occurs. Finally, we consider estimating PPR when several machines are available for parallel computation, devising a method that leverages our clustering findings, specifically the quantities computed $\textit{in situ}$, to assign tasks to machines in a manner that reduces computation time. This demonstrates that the relationship between complexity and clustering has important consequences in a practical distributed setting.

cs.SI↗

arXiv2018

Social Learning with Questions

This work studies sequential social learning (also known as Bayesian observational learning), and how private communication can enable agents to avoid herding to the wrong action/state. Starting from the seminal BHW (Bikhchandani, Hirshleifer, and Welch, 1992) model where asymptotic learning does not occur, we allow agents to ask private and finite questions to a bounded subset of their predecessors. While retaining the publicly observed history of the agents and their Bayes rationality from the BHW model, we further assume that both the ability to ask questions and the questions themselves are common knowledge. Then interpreting asking questions as partitioning information sets, we study whether asymptotic learning can be achieved with finite capacity questions. Restricting our attention to the network where every agent is only allowed to query her immediate predecessor, an explicit construction shows that a 1-bit question from each agent is enough to enable asymptotic learning.

cs.GT↗

arXiv2018

Incentive design for learning in user-recommendation systems with time-varying states

We consider the problem of how strategic users with asymmetric information can learn an underlying time varying state in a user-recommendation system. Users who observe private signals about the state, sequentially make a decision about buying a product whose value varies with time in an ergodic manner. We formulate the team problem as an instance of decentralized stochastic control problem and characterize its optimal policies. With strategic users, we design incentives such that users reveal their true private signals, so that the gap between the strategic and team objective is small and the overall expected incentive payments are also small.

cs.GT↗

arXiv2018

Personalized PageRank dimensionality and algorithmic implications

Many systems, including the Internet, social networks, and the power grid, can be represented as graphs. When analyzing graphs, it is often useful to compute scores describing the relative importance or distance between nodes. One example is Personalized PageRank (PPR), which assigns to each node $v$ a vector whose $i$-th entry describes the importance of the $i$-th node from the perspective of $v$. PPR has proven useful in many applications, such as recommending who users should follow on social networks (if this $i$-th entry is large, $v$ may be interested in following the $i$-th user). Unfortunately, computing $n$ such PPR vectors (where $n$ is the number of nodes) is infeasible for many graphs of interest. In this work, we argue that the situation is not so dire. Our main result shows that the dimensionality of the set of PPR vectors scales sublinearly in $n$ with high probability, for a certain class of random graphs and for a notion of dimensionality similar to rank. Put differently, we argue that the effective dimension of this set is much less than $n$, despite the fact that the matrix containing these vectors has rank $n$. Furthermore, we show this dimensionality measure relates closely to the complexity of a PPR estimation scheme that was proposed (but not analyzed) by Jeh and Widom. This allows us to argue that accurately estimating all $n$ PPR vectors amounts to computing a vanishing fraction of the $n^2$ vector elements (when the technical assumptions of our main result are satisfied). Finally, we demonstrate empirically that similar conclusions hold when considering real-world networks, despite the assumptions of our theory not holding.

cs.SI↗

arXiv2018

Accurate Learning or Fast Mixing? Dynamic Adaptability of Caching Algorithms

Typical analysis of content caching algorithms using the metric of steady state hit probability under a stationary request process does not account for performance loss under a variable request arrival process. In this work, we consider adaptability of caching algorithms from two perspectives: (a) the accuracy of learning a fixed popularity distribution, and (b) the speed of learning items' popularity. In order to attain this goal, we compute the distance between the stationary distributions of several popular algorithms with that of a genie-aided algorithm that has knowledge of the true popularity ranking, which we use as a measure of learning accuracy. We then characterize the mixing time of each algorithm, i.e., the time needed to attain the stationary distribution, which we use as a measure of learning efficiency. We merge both measures above to obtain the "learning error" representing both how quickly and how accurately an algorithm learns the optimal caching distribution, and use this to determine the trade-off between these two objectives of many popular caching algorithms. Informed by the results of our analysis, we propose a novel hybrid algorithm, Adaptive-LRU (A-LRU), that learns both faster and better the changes in the popularity. We show numerically that it also outperforms all other candidate algorithms when confronted with either a dynamically changing synthetic request process or using real world traces.

cs.NI↗

arXiv2018

Mean Field Games in Nudge Systems for Societal Networks

We consider the general problem of resource sharing in societal networks, consisting of interconnected communication, transportation, energy and other networks important to the functioning of society. Participants in such network need to take decisions daily, both on the quantity of resources to use as well as the periods of usage. With this in mind, we discuss the problem of incentivizing users to behave in such a way that society as a whole benefits. In order to perceive societal level impact, such incentives may take the form of rewarding users with lottery tickets based on good behavior, and periodically conducting a lottery to translate these tickets into real rewards. We will pose the user decision problem as a mean field game (MFG), and the incentives question as one of trying to select a good mean field equilibrium (MFE). In such a framework, each agent (a participant in the societal network) takes a decision based on an assumed distribution of actions of his/her competitors, and the incentives provided by the social planner. The system is said to be at MFE if the agent's action is a sample drawn from the assumed distribution. We will show the existence of such an MFE under different settings, and also illustrate how to choose an attractive equilibrium using as an example demand-response in energy networks.

cs.GT↗

arXiv2017

Small-Scale Markets for Bilateral Resource Trading in the Sharing Economy

We consider a general small-scale market for agent-to-agent resource sharing, in which each agent could either be a server (seller) or a client (buyer) in each time period. In every time period, a server has a certain amount of resources that any client could consume, and randomly gets matched with a client. Our target is to maximize the resource utilization in such an agent-to-agent market, where the agents are strategic. During each transaction, the server gets money and the client gets resources. Hence, trade ratio maximization implies efficiency maximization of our system. We model the proposed market system through a Mean Field Game approach and prove the existence of the Mean Field Equilibrium, which can achieve an almost 100% trade ratio. Finally, we carry out a simulation study motivated by an agent-to-agent computing market, and a case study on a proposed photovoltaic market, and show the designed market benefits both individuals and the system as a whole.

cs.GT↗

arXiv2017

The Value of Sharing Intermittent Spectrum

Recent initiatives by regulatory agencies to increase spectrum resources available for broadband access include rules for sharing spectrum with high-priority incumbents. We study a model in which wireless Service Providers (SPs) charge for access to their own exclusive-use (licensed) band along with access to an additional shared band. The total, or delivered price in each band is the announced price plus a congestion cost, which depends on the load, or total users normalized by the bandwidth. The shared band is intermittently available with some probability, due to incumbent activity, and when unavailable, any traffic carried on that band must be shifted to licensed bands. The SPs then compete for quantity of users. We show that the value of the shared band depends on the relative sizes of the SPs: large SPs with more bandwidth are better able to absorb the variability caused by intermittency than smaller SPs. However, as the amount of shared spectrum increases, the large SPs may not make use of it. In that scenario shared spectrum creates more value than splitting it among the SPs for exclusive use. We also show that fixing the average amount of available shared bandwidth, increasing the reliability of the band is preferable to increasing the bandwidth.

cs.GT↗

arXiv2016

Approximately Envy-Free Spectrum Allocation with Complementarities

With spectrum auctions as our prime motivation, in this paper we analyze combinatorial auctions where agents' valuations exhibit complementarities. Assuming that the agents only value bundles of size at most $k$ and also assuming that we can assess prices, we present a mechanism that is efficient, approximately envy-free, asymptotically strategy-proof and that has polynomial-time complexity. Modifying an iterative rounding procedure from assignment problems, we use the primal and dual optimal solutions to the linear programming relaxation of the auction problem to construct a lottery for the allocations and to assess the prices to bundles. The allocations in the lottery over-allocate goods by at most $k-1$ units, and the dual prices are shown to be (approximately) envy-free irrespective of the allocation chosen. We conclude with a detailed numerical investigation of a specific spectrum allocation problem.

cs.GT↗

arXiv2016

Incentivizing Sharing in Realtime D2D Streaming Networks: A Mean Field Game Perspective

We consider the problem of streaming live content to a cluster of co-located wireless devices that have both an expensive unicast base-station-to-device (B2D) interface, as well as an inexpensive broadcast device-to-device (D2D) interface, which can be used simultaneously. Our setting is a streaming system that uses a block-by-block random linear coding approach to achieve a target percentage of on-time deliveries with minimal B2D usage. Our goal is to design an incentive framework that would promote such cooperation across devices, while ensuring good quality of service. Based on ideas drawn from truth-telling auctions, we design a mechanism that achieves this goal via appropriate transfers (monetary payments or rebates) in a setting with a large number of devices, and with peer arrivals and departures. Here, we show that a Mean Field Game can be used to accurately approximate our system. Furthermore, the complexity of calculating the best responses under this regime is low. We implement the proposed system on an Android testbed, and illustrate its efficient performance using real world experiments.

cs.GT↗