arXiv subjects

Qing Zhao

Publications and source records attributed to Qing Zhao.

209 records · Page 12Linked to original sources

arXiv2010

Distributed Learning in Multi-Armed Bandit with Multiple Players

We formulate and study a decentralized multi-armed bandit (MAB) problem. There are M distributed players competing for N independent arms. Each arm, when played, offers i.i.d. reward according to a distribution with an unknown parameter. At each time, each player chooses one arm to play without exchanging observations or any information with other players. Players choosing the same arm collide, and, depending on the collision model, either no one receives reward or the colliding players share the reward in an arbitrary way. We show that the minimum system regret of the decentralized MAB grows with time at the same logarithmic order as in the centralized counterpart where players act collectively as a single entity by exchanging observations and making decisions jointly. A decentralized policy is constructed to achieve this optimal order while ensuring fairness among players and without assuming any pre-agreement or information exchange among players. Based on a Time Division Fair Sharing (TDFS) of the M best arms, the proposed policy is constructed and its order optimality is proven under a general reward model. Furthermore, the basic structure of the TDFS policy can be used with any order-optimal single-player policy to achieve order optimality in the decentralized setting. We also establish a lower bound on the system regret growth rate for a general class of decentralized polices, to which the proposed policy belongs. This problem finds potential applications in cognitive radio networks, multi-channel communication systems, multi-agent systems, web search and advertising, and social networks.

math.OC↗

arXiv2009

On the Connectivity and Multihop Delay of Ad Hoc Cognitive Radio Networks

We analyze the multihop delay of ad hoc cognitive radio networks, where the transmission delay of each hop consists of the propagation delay and the waiting time for the availability of the communication channel (i.e., the occurrence of a spectrum opportunity at this hop). Using theories and techniques from continuum percolation and ergodicity, we establish the scaling law of the minimum multihop delay with respect to the source-destination distance in cognitive radio networks. When the propagation delay is negligible, we show the starkly different scaling behavior of the minimum multihop delay in instantaneously connected networks as compared to networks that are only intermittently connected due to scarcity of spectrum opportunities. Specifically, if the network is instantaneously connected, the minimum multihop delay is asymptotically independent of the distance; if the network is only intermittently connected, the minimum multihop delay scales linearly with the distance. When the propagation delay is nonnegligible but small, we show that although the scaling order is always linear, the scaling rate for an instantaneously connected network can be orders of magnitude smaller than the one for an intermittently connected network.

cs.NI↗

arXiv2009

Hopf Structures on Minimal Hopf Quivers

In this paper we investigate pointed Hopf algebras via quiver methods. We classify all possible Hopf structures arising from minimal Hopf quivers, namely basic cycles and the linear chain. This provides full local structure information for general pointed Hopf algebras.

math.QA↗

arXiv2009

Connectivity of Heterogeneous Wireless Networks

We address the connectivity of large-scale ad hoc heterogeneous wireless networks, where secondary users exploit channels temporarily unused by primary users and the existence of a communication link between two secondary users depends on not only the distance between them but also the transmitting and receiving activities of nearby primary users. We introduce the concept of connectivity region defined as the set of density pairs -- the density of secondary users and the density of primary transmitters -- under which the secondary network is connected. Using theories and techniques from continuum percolation, we analytically characterize the connectivity region of the secondary network and reveal the tradeoff between proximity (the number of neighbors) and the occurrence of spectrum opportunities. Specifically, we establish three basic properties of the connectivity region -- contiguity, monotonicity of the boundary, and uniqueness of the infinite connected component, where the uniqueness implies the occurrence of a phase transition phenomenon in terms of the almost sure existence of either zero or one infinite connected component; we identify and analyze two critical densities which jointly specify the profile as well as an outer bound on the connectivity region; we study the impacts of secondary users' transmission power on the connectivity region and the conditional average degree of a secondary user, and demonstrate that matching the interference ranges of the primary and the secondary networks maximizes the tolerance of the secondary network to the primary traffic load. Furthermore, we establish a necessary condition and a sufficient condition for connectivity, which lead to an outer bound and an inner bound on the connectivity region.

cs.NI↗

arXiv2009

Power Control in Cognitive Radio Networks: How to Cross a Multi-Lane Highway

We consider power control in cognitive radio networks where secondary users identify and exploit instantaneous and local spectrum opportunities without causing unacceptable interference to primary users. We qualitatively characterize the impact of the transmission power of secondary users on the occurrence of spectrum opportunities and the reliability of opportunity detection. Based on a Poisson model of the primary network, we quantify these impacts by showing that (i) the probability of spectrum opportunity decreases exponentially with respect to the transmission power of secondary users, where the exponential decay constant is given by the traffic load of primary users; (ii) reliable opportunity detection is achieved in the two extreme regimes in terms of the ratio between the transmission power of secondary users and that of primary users. Such analytical characterizations allow us to study power control for optimal transport throughput under constraints on the interference to primary users. Furthermore, we reveal the difference between detecting primary signals and detecting spectrum opportunities, and demonstrate the complex relationship between physical layer spectrum sensing and MAC layer throughput. The dependency of this PHY-MAC interaction on the application type and the use of handshake signaling such as RTS/CTS is illustrated.

cs.NI↗

arXiv2009

Optimality of Myopic Sensing in Multi-Channel Opportunistic Access

We consider opportunistic communications over multiple channels where the state ("good" or "bad") of each channel evolves as independent and identically distributed Markov processes. A user, with limited sensing and access capability, chooses one channel to sense and subsequently access (based on the sensed channel state) in each time slot. A reward is obtained when the user senses and accesses a "good" channel. The objective is to design the optimal channel selection policy that maximizes the expected reward accrued over time. This problem can be generally cast as a Partially Observable Markov Decision Process (POMDP) or a restless multi-armed bandit process, to which optimal solutions are often intractable. We show in this paper that the myopic policy, with a simple and robust structure, achieves optimality under certain conditions. This result finds applications in opportunistic communications in fading environment, cognitive radio networks for spectrum overlay, and resource-constrained jamming and anti-jamming.

cs.NI↗

arXiv2008

Structure and Optimality of Myopic Policy in Opportunistic Access with Noisy Observations

A restless multi-armed bandit problem that arises in multichannel opportunistic communications is considered, where channels are modeled as independent and identical Gilbert-Elliot channels and channel state observations are subject to errors. A simple structure of the myopic policy is established under a certain condition on the false alarm probability of the channel state detector. It is shown that the myopic policy has a semi-universal structure that reduces channel selection to a simple round-robin procedure and obviates the need to know the underlying Markov transition probabilities. The optimality of the myopic policy is proved for the case of two channels and conjectured for the general case based on numerical examples.

cs.NI↗

arXiv2008

Indexability of Restless Bandit Problems and Optimality of Whittle's Index for Dynamic Multichannel Access

We consider a class of restless multi-armed bandit problems (RMBP) that arises in dynamic multichannel access, user/server scheduling, and optimal activation in multi-agent systems. For this class of RMBP, we establish the indexability and obtain Whittle's index in closed-form for both discounted and average reward criteria. These results lead to a direct implementation of Whittle's index policy with remarkably low complexity. When these Markov chains are stochastically identical, we show that Whittle's index policy is optimal under certain conditions. Furthermore, it has a semi-universal structure that obviates the need to know the Markov transition probabilities. The optimality and the semi-universal structure result from the equivalency between Whittle's index policy and the myopic policy established in this work. For non-identical channels, we develop efficient algorithms for computing a performance upper bound given by Lagrangian relaxation. The tightness of the upper bound and the near-optimal performance of Whittle's index policy are illustrated with simulation examples.

cs.IT↗

arXiv2008

On Myopic Sensing for Multi-Channel Opportunistic Access: Structure, Optimality, and Performance

We consider a multi-channel opportunistic communication system where the states of these channels evolve as independent and statistically identical Markov chains (the Gilbert-Elliot channel model). A user chooses one channel to sense and access in each slot and collects a reward determined by the state of the chosen channel. The problem is to design a sensing policy for channel selection to maximize the average reward, which can be formulated as a multi-arm restless bandit process. In this paper, we study the structure, optimality, and performance of the myopic sensing policy. We show that the myopic sensing policy has a simple robust structure that reduces channel selection to a round-robin procedure and obviates the need for knowing the channel transition probabilities. The optimality of this simple policy is established for the two-channel case and conjectured for the general case based on numerical results. The performance of the myopic sensing policy is analyzed, which, based on the optimality of myopic sensing, characterizes the maximum throughput of a multi-channel opportunistic communication system and its scaling behavior with respect to the number of channels. These results apply to cognitive radio networks, opportunistic transmission in fading environments, and resource-constrained jamming and anti-jamming.

cs.NI↗

arXiv2007

Joint Design and Separation Principle for Opportunistic Spectrum Access in the Presence of Sensing Errors

We address the design of opportunistic spectrum access (OSA) strategies that allow secondary users to independently search for and exploit instantaneous spectrum availability. Integrated in the joint design are three basic components: a spectrum sensor that identifies spectrum opportunities, a sensing strategy that determines which channels in the spectrum to sense, and an access strategy that decides whether to access based on imperfect sensing outcomes. We formulate the joint PHY-MAC design of OSA as a constrained partially observable Markov decision process (POMDP). Constrained POMDPs generally require randomized policies to achieve optimality, which are often intractable. By exploiting the rich structure of the underlying problem, we establish a separation principle for the joint design of OSA. This separation principle reveals the optimality of myopic policies for the design of the spectrum sensor and the access strategy, leading to closed-form optimal solutions. Furthermore, decoupling the design of the sensing strategy from that of the spectrum sensor and the access strategy, the separation principle reduces the constrained POMDP to an unconstrained one, which admits deterministic optimal policies. Numerical examples are provided to study the design tradeoffs, the interaction between the spectrum sensor and the sensing and access strategies, and the robustness of the ensuing design to model mismatch.

cs.NI↗

arXiv2006

Dynamic Spectrum Access: Signal Processing, Networking, and Regulatory Policy

In this article, we first provide a taxonomy of dynamic spectrum access. We then focus on opportunistic spectrum access, the overlay approach under the hierarchical access model of dynamic spectrum access. we aim to provide an overview of challenges and recent developments in both technological and regulatory aspects of opportunistic spectrum access.

cs.NI↗