arXiv subjects

Nouman Khan

Publications and source records attributed to Nouman Khan.

5 recordsLinked to original sources

arXiv2025

A Strong Duality Result for Constrained POMDPs with Multiple Cooperative Agents

The work studies the problem of decentralized constrained POMDPs in a team-setting where multiple nonstrategic agents have asymmetric information. Using an extension of Sion's Minimax theorem for functions with positive infinity and results on weak-convergence of measures, strong duality is established for the setting of infinite-horizon expected total discounted costs when the observations lie in a countable space, the actions are chosen from a finite space, the constraint costs are bounded, and the objective cost is bounded from below.

math.OC↗

arXiv2025

On Equivalence Between Decentralized Policy-Profile Mixtures and Behavioral Coordination Policies in Multi-Agent Systems

Constrained decentralized team problem formulations are good models for many cooperative multi-agent systems. Constraints necessitate randomization when solving for optimal solutions -- past results show that joint randomization in the team is in general necessary for (strong) Lagrangian duality to hold -- , but a better understanding of randomization still remains. For a partially observed multi-agent system with a Borel hidden state, countable observations, and finite actions, we prove the following: \textit{i}) independently randomized decentralized policy-profiles -- whether behavioral or pure -- induce the same occupation measures (on joint-history and joint-action pairs) as decentralized behavioral policy-profiles; and \textit{ii}) jointly randomized behavioral and pure decentralized policy-profiles induce the same occupation measures. Restricting to finite observations, we also prove that joint mixtures of decentralized policy-profiles (both pure and behavioral) and common information based behavioral coordination policies (also mixtures of them) induce the same occupation measures. This generalizes past work that shows equivalence between pure decentralized policy-profiles and pure coordination policies. These results can be used to develop further results on Lagrangian duality, the minimum number of randomizations needed in an optimal behavioral coordination policy, and learning based schemes that can find approximately optimal solutions.

math.OC↗

arXiv2023

Backward and Forward Inference in Interacting Independent-Cascade Processes: A Scalable and Convergent Message-Passing Approach

We study the problems of estimating the past and future evolutions of two diffusion processes that spread concurrently on a network. Specifically, given a known network $G=(V, \overrightarrow{E})$ and a (possibly noisy) snapshot $\mathcal{O}_n$ of its state taken at (a possibly unknown) time $W$, we wish to determine the posterior distributions of the initial state of the network and the infection times of its nodes. These distributions are useful in finding source nodes of epidemics and rumors -- $\textit{backward inference}$ -- , and estimating the spread of a fixed set of source nodes -- $\textit{forward inference}$. To model the interaction between the two processes, we study an extension of the independent-cascade (IC) model where, when a node gets infected with either process, its susceptibility to the other one changes. First, we derive the exact joint probability of the initial state of the network and the observation-snapshot $\mathcal{O}_n$. Then, using the machinery of factor-graphs, factor-graph transformations, and the generalized distributive-law, we derive a Belief-Propagation (BP) based algorithm that is scalable to large networks and can converge on graphs of arbitrary topology (at a likely expense in approximation accuracy).

cs.SI↗

arXiv2023

Cooperative Multi-Agent Constrained POMDPs: Strong Duality and Primal-Dual Reinforcement Learning with Approximate Information States

We study the problem of decentralized constrained POMDPs in a team-setting where the multiple non-strategic agents have asymmetric information. Strong duality is established for the setting of infinite-horizon expected total discounted costs when the observations lie in a countable space, the actions are chosen from a finite space, and the immediate cost functions are bounded. Following this, connections with the common-information and approximate information-state approaches are established. The approximate information-states are characterized independent of the Lagrange-multipliers vector so that adaptations of the multiplier (during learning) will not necessitate new representations. Finally, a primal-dual multi-agent reinforcement learning (MARL) framework based on centralized training distributed execution (CTDE) and three time-scale stochastic approximation is developed with the aid of recurrent and feedforward neural-networks as function-approximators.

math.OC↗

arXiv2022

Rarest-First with Probabilistic-Mode-Suppression

Recent studies suggested that the BitTorrent's rarest-first protocol, owing to its work-conserving nature, can become unstable in the presence of non-persistent users. Consequently, for any provably stable protocol, many peers, at some point, would have to be endogenously forced to hold off their file-download activity. In this work, we propose a tunable piece-selection policy that minimizes this (undesirable) requisite by combining the (work-conserving but not stabilizing) rarest-first protocol with only an appropriate share of the (non-work conserving and stabilizing) mode-suppression protocol. We refer to this policy as ``Rarest-First with Probabilistic Mode-Suppression'' or simply RFwPMS. We study RFwPMS using a stochastic abstraction of the BitTorrent network that is general enough to capture a multiple swarm setting of non-persistent users -- each swarm having its own altruistic preferences that may or may not overlap with those of other swarms. Using Lyapunov drift analysis, we show that for all kinds of inter-swarm behaviors and all arrival-rate configurations, RFwPMS is stable. Then, using the Kingman's moment bound technique, we further show that the expected steady-state sojourn time of RFwPMS is independent of the arrival-rate in the single-swarm case (under a mild additional assumption). Finally, our simulation-based performance evaluation confirms our theoretical findings and shows that the steady-state expected sojourn time is linear in the file-size (compared to our loose estimate of a polynomial with degree 6). Overall, an improved performance is observed in comparison to previously proposed stabilizing schemes like mode-suppression (MS).

eess.SY↗