arXiv subjects

Washim Uddin Mondal

Publications and source records attributed to Washim Uddin Mondal.

25 records · Page 2Linked to original sources

arXiv2023

Cooperating Graph Neural Networks with Deep Reinforcement Learning for Vaccine Prioritization

This study explores the vaccine prioritization strategy to reduce the overall burden of the pandemic when the supply is limited. Existing methods conduct macro-level or simplified micro-level vaccine distribution by assuming the homogeneous behavior within subgroup populations and lacking mobility dynamics integration. Directly applying these models for micro-level vaccine allocation leads to sub-optimal solutions due to the lack of behavioral-related details. To address the issue, we first incorporate the mobility heterogeneity in disease dynamics modeling and mimic the disease evolution process using a Trans-vaccine-SEIR model. Then we develop a novel deep reinforcement learning to seek the optimal vaccine allocation strategy for the high-degree spatial-temporal disease evolution system. The graph neural network is used to effectively capture the structural properties of the mobility contact network and extract the dynamic disease features. In our evaluation, the proposed framework reduces 7% - 10% of infections and deaths than the baseline strategies. Extensive evaluation shows that the proposed framework is robust to seek the optimal vaccine allocation with diverse mobility patterns in the micro-level disease evolution system. In particular, we find the optimal vaccine allocation strategy in the transit usage restriction scenario is significantly more effective than restricting cross-zone mobility for the top 10% age-based and income-based zones. These results provide valuable insights for areas with limited vaccines and low logistic efficacy.

q-bio.PE↗

arXiv2022

On the Near-Optimality of Local Policies in Large Cooperative Multi-Agent Reinforcement Learning

We show that in a cooperative $N$-agent network, one can design locally executable policies for the agents such that the resulting discounted sum of average rewards (value) well approximates the optimal value computed over all (including non-local) policies. Specifically, we prove that, if $|\mathcal{X}|, |\mathcal{U}|$ denote the size of state, and action spaces of individual agents, then for sufficiently small discount factor, the approximation error is given by $\mathcal{O}(e)$ where $e\triangleq \frac{1}{\sqrt{N}}\left[\sqrt{|\mathcal{X}|}+\sqrt{|\mathcal{U}|}\right]$. Moreover, in a special case where the reward and state transition functions are independent of the action distribution of the population, the error improves to $\mathcal{O}(e)$ where $e\triangleq \frac{1}{\sqrt{N}}\sqrt{|\mathcal{X}|}$. Finally, we also devise an algorithm to explicitly construct a local policy. With the help of our approximation results, we further establish that the constructed local policy is within $\mathcal{O}(\max\{e,ε\})$ distance of the optimal policy, and the sample complexity to achieve such a local policy is $\mathcal{O}(ε^{-3})$, for any $ε>0$.

cs.LG↗

arXiv2022

Deep Learning based Coverage and Rate Manifold Estimation in Cellular Networks

This article proposes Convolutional Neural Network-based Auto Encoder (CNN-AE) to predict location-dependent rate and coverage probability of a network from its topology. We train the CNN utilising BS location data of India, Brazil, Germany, and the USA and compare its performance with stochastic geometry (SG) based analytical models. In comparison to the best-fitted SG-based model, CNN-AE improves the coverage and rate prediction errors by a margin of as large as $40\%$ and $25\%$ respectively. As an application, we propose a low complexity, provably convergent algorithm that, using trained CNN-AE, can compute locations of new BSs that need to be deployed in a network in order to satisfy pre-defined spatially heterogeneous performance goals.

cs.NI↗

arXiv2022

Can Mean Field Control (MFC) Approximate Cooperative Multi Agent Reinforcement Learning (MARL) with Non-Uniform Interaction?

Mean-Field Control (MFC) is a powerful tool to solve Multi-Agent Reinforcement Learning (MARL) problems. Recent studies have shown that MFC can well-approximate MARL when the population size is large and the agents are exchangeable. Unfortunately, the presumption of exchangeability implies that all agents uniformly interact with one another which is not true in many practical scenarios. In this article, we relax the assumption of exchangeability and model the interaction between agents via an arbitrary doubly stochastic matrix. As a result, in our framework, the mean-field `seen' by different agents are different. We prove that, if the reward of each agent is an affine function of the mean-field seen by that agent, then one can approximate such a non-uniform MARL problem via its associated MFC problem within an error of $e=\mathcal{O}(\frac{1}{\sqrt{N}}[\sqrt{|\mathcal{X}|} + \sqrt{|\mathcal{U}|}])$ where $N$ is the population size and $|\mathcal{X}|$, $|\mathcal{U}|$ are the sizes of state and action spaces respectively. Finally, we develop a Natural Policy Gradient (NPG) algorithm that can provide a solution to the non-uniform MARL with an error $\mathcal{O}(\max\{e,ε\})$ and a sample complexity of $\mathcal{O}(ε^{-3})$ for any $ε>0$.

cs.LG↗

arXiv2022

On the Approximation of Cooperative Heterogeneous Multi-Agent Reinforcement Learning (MARL) using Mean Field Control (MFC)

Mean field control (MFC) is an effective way to mitigate the curse of dimensionality of cooperative multi-agent reinforcement learning (MARL) problems. This work considers a collection of $N_{\mathrm{pop}}$ heterogeneous agents that can be segregated into $K$ classes such that the $k$-th class contains $N_k$ homogeneous agents. We aim to prove approximation guarantees of the MARL problem for this heterogeneous system by its corresponding MFC problem. We consider three scenarios where the reward and transition dynamics of all agents are respectively taken to be functions of $(1)$ joint state and action distributions across all classes, $(2)$ individual distributions of each class, and $(3)$ marginal distributions of the entire population. We show that, in these cases, the $K$-class MARL problem can be approximated by MFC with errors given as $e_1=\mathcal{O}(\frac{\sqrt{|\mathcal{X}|}+\sqrt{|\mathcal{U}|}}{N_{\mathrm{pop}}}\sum_{k}\sqrt{N_k})$, $e_2=\mathcal{O}(\left[\sqrt{|\mathcal{X}|}+\sqrt{|\mathcal{U}|}\right]\sum_{k}\frac{1}{\sqrt{N_k}})$ and $e_3=\mathcal{O}\left(\left[\sqrt{|\mathcal{X}|}+\sqrt{|\mathcal{U}|}\right]\left[\frac{A}{N_{\mathrm{pop}}}\sum_{k\in[K]}\sqrt{N_k}+\frac{B}{\sqrt{N_{\mathrm{pop}}}}\right]\right)$, respectively, where $A, B$ are some constants and $|\mathcal{X}|,|\mathcal{U}|$ are the sizes of state and action spaces of each agent. Finally, we design a Natural Policy Gradient (NPG) based algorithm that, in the three cases stated above, can converge to an optimal MARL policy within $\mathcal{O}(e_j)$ error with a sample complexity of $\mathcal{O}(e_j^{-3})$, $j\in\{1,2,3\}$, respectively.

cs.LG↗

arXiv2021

Queuing Analysis of Opportunistic Cognitive Radio IoT Network with Imperfect Sensing

In this paper, we analyze a Cognitive Radio-based Internet-of-Things (CR-IoT) network comprising a Primary Network Provider (PNP) and an IoT operator. The PNP uses its licensed spectrum to serve its users. The IoT operator identifies the white-space in the licensed band at regular intervals and opportunistically exploits them to serve the IoT nodes under its coverage. IoT nodes are battery-operated devices that require periodical energy replenishment. We employ the Microwave Power Transfer (MPT) technique for its superior energy transfer efficiency over long-distance. The white-space detection process is not always perfect and the IoT operator may jeopardize the PNP's transmissions due to misdetection. To reduce the possibility of such interferences, some of the spectrum holes must remain unutilized, even when the IoT nodes have data to transmit. The IoT operator needs to decide what percentage of the white-space to keep unutilized and how to judiciously use the rest for data transmission and energy-replenishment to maintain an optimal balance between the average interference inflicted on PNP's users and the Quality-of-Service (QoS) experienced by IoT nodes. Due to the periodic nature of the spectrum-sensing process, Discrete Time Markov Chain (DTMC) method can realistically model this framework. In literature, activities of the PNP and IoT operator are assumed to be mutually exclusive, for ease of analysis. Our model incorporates possible overlaps between these activities, making the analysis more realistic. Using our model, the sustainability region of the CR-IoT network can be obtained. The accuracy of our analysis is demonstrated via extensive simulation.

cs.PF↗

arXiv2020

On Exact Distribution of Poisson-Voronoi Area in $K$-tier HetNets with Generalized Association Rule

This letter characterizes the exact distribution function of a typical Voronoi area in a $K$-tier Poisson network. The users obey a generalized association (GA) rule, which is a superset of nearest base station association and maximum received power based association (with arbitrary fading) rules that are commonly adopted in the literature. Combining the Robbins' theorem and the probability generating functional of a Poisson point process, we obtain the exact moments of a typical $k$-th tier Voronoi area, $k \in \{1,...,K\}$ under the GA rule. We apply this result in several special cases. For example, we prove that in multi-tier networks with the GA rule, the mean of $k$-th tier Voronoi area can exactly be expressed in a closed-form. We also obtain simplified expressions of its higher-order moments for both average and instantaneous received power based user association. In single-tier networks with exponential fading, the later association rule provides closed-form expression of the second-order moment of a typical Voronoi area. We numerically evaluate this exact expression and compare it with an approximated result.

cs.IT↗