Searcharxiv⌕ Search

arXiv subjects

Mihaela van der Schaar

Publications and source records attributed to Mihaela van der Schaar.

At least 361 records · Page 20Linked to original sources
arXiv2010

Adaptive MAC Protocols Using Memory for Networks with Critical Traffic

We consider wireless communication networks where network users are subject to critical events such as emergencies and crises. If a critical event occurs to a user, the user needs to send critical traffic as early as possible. However, most existing medium access control (MAC) protocols are not adequate to meet the urgent need for data transmission by users with critical traffic. In this paper, we devise a class of distributed MAC protocols that achieve coordination using the finite-length memory of users containing their own observations and traffic types. We formulate a protocol design problem and find optimal protocols that solve the problem. We show that the proposed protocols enable a user with critical traffic to transmit its critical traffic without interruption from other users after a short delay while allowing users to share the channel efficiently when there is no critical traffic. Moreover, the proposed protocols require short memory and can be implemented without explicit message passing.

cs.NI↗
arXiv2010

Structure-Aware Stochastic Control for Transmission Scheduling

In this paper, we consider the problem of real-time transmission scheduling over time-varying channels. We first formulate the transmission scheduling problem as a Markov decision process (MDP) and systematically unravel the structural properties (e.g. concavity in the state-value function and monotonicity in the optimal scheduling policy) exhibited by the optimal solutions. We then propose an online learning algorithm which preserves these structural properties and achieves -optimal solutions for an arbitrarily small . The advantages of the proposed online method are that: (i) it does not require a priori knowledge of the traffic arrival and channel statistics and (ii) it adaptively approximates the state-value functions using piece-wise linear functions and has low storage and computation complexity. We also extend the proposed low-complexity online learning solution to the prioritized data transmission. The simulation results demonstrate that the proposed method achieves significantly better utility (or delay)-energy trade-offs when comparing to existing state-of-art online optimization methods.

cs.LG↗
arXiv2010

A Game Theoretic Analysis of Incentives in Content Production and Sharing over Peer-to-Peer Networks

User-generated content can be distributed at a low cost using peer-to-peer (P2P) networks, but the free-rider problem hinders the utilization of P2P networks. In order to achieve an efficient use of P2P networks, we investigate fundamental issues on incentives in content production and sharing using game theory. We build a basic model to analyze non-cooperative outcomes without an incentive scheme and then use different game formulations derived from the basic model to examine five incentive schemes: cooperative, payment, repeated interaction, intervention, and enforced full sharing. The results of this paper show that 1) cooperative peers share all produced content while non-cooperative peers do not share at all without an incentive scheme; 2) a cooperative scheme allows peers to consume more content than non-cooperative outcomes do; 3) a cooperative outcome can be achieved among non-cooperative peers by introducing an incentive scheme based on payment, repeated interaction, or intervention; and 4) enforced full sharing has ambiguous welfare effects on peers. In addition to describing the solutions of different formulations, we discuss enforcement and informational requirements to implement each solution, aiming to offer a guideline for protocol designers when designing incentive schemes for P2P networks.

cs.NI↗
arXiv2010

Medium Access Control Protocols With Memory

Many existing medium access control (MAC) protocols utilize past information (e.g., the results of transmission attempts) to adjust the transmission parameters of users. This paper provides a general framework to express and evaluate distributed MAC protocols utilizing a finite length of memory for a given form of feedback information. We define protocols with memory in the context of a slotted random access network with saturated arrivals. We introduce two performance metrics, throughput and average delay, and formulate the problem of finding an optimal protocol. We first show that a TDMA outcome, which is the best outcome in the considered scenario, can be obtained after a transient period by a protocol with (N-1)-slot memory, where N is the total number of users. Next, we analyze the performance of protocols with 1-slot memory using a Markov chain and numerical methods. Protocols with 1-slot memory can achieve throughput arbitrarily close to 1 (i.e., 100% channel utilization) at the expense of large average delay, by correlating successful users in two consecutive slots. Finally, we apply our framework to wireless local area networks.

cs.NI↗
arXiv2009

Dynamic Conjectures in Random Access Networks Using Bio-inspired Learning

This paper considers a conjecture-based distributed learning approach that enables autonomous nodes to independently optimize their transmission probabilities in random access networks. We model the interaction among multiple self-interested nodes as a game. It is well-known that the Nash equilibria in this game result in zero throughput for all the nodes if they take myopic best-response, thereby leading to a network collapse. This paper enables nodes to behave as intelligent entities which can proactively gather information, form internal conjectures on how their competitors would react to their actions, and update their beliefs according to their local observations. In this way, nodes are capable to autonomously "learn" the behavior of their competitors, optimize their own actions, and eventually cultivate reciprocity in the random access network. To characterize the steady-state outcome, the conjectural equilibrium is introduced. Inspired by the biological phenomena of "derivative action" and "gradient dynamics", two distributed conjecture-based action update mechanisms are proposed to stabilize the random access network. The sufficient conditions that guarantee the proposed conjecture-based learning algorithms to converge are derived. Moreover, it is shown that all the achievable operating points in the throughput region are essentially stable conjectural equilibria corresponding to different conjectures. We investigate how the conjectural equilibrium can be selected in heterogeneous networks and how the proposed methods can be extended to ad-hoc networks. Simulations verify that the system performance significantly outperforms existing protocols, such as IEEE 802.11 DCF protocol and the PMAC protocol, in terms of throughput, fairness, convergence, and stability.

cs.GT↗
arXiv2009

Media-TCP: A Quality-Centric TCP-Friendly Congestion Control for Multimedia Transmission

In this paper, we propose a quality-centric congestion control for multimedia streaming over IP networks, which we refer to as media-TCP. Unlike existing congestion control schemes that adapt a user's sending rate merely to the network condition, our solution adapts the sending rate to both the network condition and the application characteristics by explicitly considering the distortion impacts, delay deadlines, and interdependencies of different video packet classes. Hence, our media-aware solution is able to provide differential services for transmitting various packet classes and thereby, further improves the multimedia streaming quality. We model this problem using a Finite-Horizon Markov Decision Process (FHMDP) and determine the optimal congestion control policy that maximizes the long-term multimedia quality, while adhering to the horizon- TCP-friendliness constraint, which ensures long-term fairness with existing TCP applications. We show that the FHMDP problem can be decomposed into multiple optimal stopping problems, which admit a low-complexity threshold-based solution. Moreover, unlike existing congestion control approaches, which focus on maintaining throughput-based fairness among users, the proposed media-TCP aims to achieve quality-based fairness among multimedia users. We also derive sufficient conditions for multiple multimedia users to achieve quality-based fairness using media-TCP congestion control. Our simulation results show that the proposed media-TCP achieves more than 3dB improvement in terms of PSNR over the conventional TCP congestion control approaches, with the largest improvements observed for real-time streaming applications requiring stringent playback delays.

cs.NI↗
arXiv2009

Linearly Coupled Communication Games

This paper discusses a special type of multi-user communication scenario, in which users' utilities are linearly impacted by their competitors' actions. First, we explicitly characterize the Nash equilibrium and Pareto boundary of the achievable utility region. Second, the price of anarchy incurred by the non-collaborative Nash strategy is quantified. Third, to improve the performance in the non-cooperative scenarios, we investigate the properties of an alternative solution concept named conjectural equilibrium, in which individual users compensate for their lack of information by forming internal beliefs about their competitors. The global convergence of the best response and Jacobi update dynamics that achieve various conjectural equilibria are analyzed. It is shown that the Pareto boundaries of the investigated linearly coupled games can be sustained as stable conjectural equilibria if the belief functions are properly initialized. The investigated models apply to a variety of realistic applications encountered in the multiple access design, including wireless random access and flow control.

cs.GT↗
arXiv2009

Online Reinforcement Learning for Dynamic Multimedia Systems

In our previous work, we proposed a systematic cross-layer framework for dynamic multimedia systems, which allows each layer to make autonomous and foresighted decisions that maximize the system's long-term performance, while meeting the application's real-time delay constraints. The proposed solution solved the cross-layer optimization offline, under the assumption that the multimedia system's probabilistic dynamics were known a priori. In practice, however, these dynamics are unknown a priori and therefore must be learned online. In this paper, we address this problem by allowing the multimedia system layers to learn, through repeated interactions with each other, to autonomously optimize the system's long-term performance at run-time. We propose two reinforcement learning algorithms for optimizing the system under different design constraints: the first algorithm solves the cross-layer optimization in a centralized manner, and the second solves it in a decentralized manner. We analyze both algorithms in terms of their required computation, memory, and inter-layer communication overheads. After noting that the proposed reinforcement learning algorithms learn too slowly, we introduce a complementary accelerated learning algorithm that exploits partial knowledge about the system's dynamics in order to dramatically improve the system's performance. In our experiments, we demonstrate that decentralized learning can perform as well as centralized learning, while enabling the layers to act autonomously. Additionally, we show that existing application-independent reinforcement learning algorithms, and existing myopic learning algorithms deployed in multimedia systems, perform significantly worse than our proposed application-aware and foresighted learning methods.

cs.LG↗
arXiv2009

Stackelberg Contention Games in Multiuser Networks

Interactions among selfish users sharing a common transmission channel can be modeled as a non-cooperative game using the game theory framework. When selfish users choose their transmission probabilities independently without any coordination mechanism, Nash equilibria usually result in a network collapse. We propose a methodology that transforms the non-cooperative game into a Stackelberg game. Stackelberg equilibria of the Stackelberg game can overcome the deficiency of the Nash equilibria of the original game. A particular type of Stackelberg intervention is constructed to show that any positive payoff profile feasible with independent transmission probabilities can be achieved as a Stackelberg equilibrium payoff profile. We discuss criteria to select an operating point of the network and informational requirements for the Stackelberg game. We relax the requirements and examine the effects of relaxation on performance.

cs.GT↗
arXiv2009

Structural Solutions for Cross-Layer Optimization of Wireless Multimedia Transmission

In this paper, we propose a systematic solution to the problem of cross-layer optimization for delay-sensitive media transmission over time-varying wireless channels as well as investigate the structures and properties of this solution, such that it can be easily implemented in various multimedia systems and applications. Specifically, we formulate this problem as a finite-horizon Markov decision process (MDP) by explicitly considering the users' heterogeneous multimedia traffic characteristics (e.g. delay deadlines, distortion impacts and dependencies etc.), time-varying network conditions as well as, importantly, their ability to adapt their cross-layer transmission strategies in response to these dynamics. Based on the heterogeneous characteristics of the media packets, we are able to express the transmission priorities between packets as a new type of directed acyclic graph (DAG). This DAG provides the necessary structure for determining the optimal cross-layer actions in each time slot: the root packet in the DAG will always be selected for transmission since it has the highest positive marginal utility; and the complexity of the proposed cross-layer solution is demonstrated to linearly increase w.r.t. the number of disconnected packet pairs in the DAG and exponentially increase w.r.t. the number of packets on which the current packets depend on. The simulation results demonstrate that the proposed solution significantly outperforms existing state-of-the-art cross-layer solutions. Moreover, we show that our solution provides the upper bound performance for the cross-layer optimization solutions with delayed feedback such as the well-known RaDiO framework.

cs.MM↗
arXiv2009

Mission-Aware Medium Access Control in Random Access Networks

We study mission-critical networking in wireless communication networks, where network users are subject to critical events such as emergencies and crises. If a critical event occurs to a user, the user needs to send necessary information for help as early as possible. However, most existing medium access control (MAC) protocols are not adequate to meet the urgent need for information transmission by users in a critical situation. In this paer, we propose a novel class of MAC protocols that utilize available past information as well as current information. Our proposed protocols are mission-aware since they prescribe different transmission decision rules to users in different situations. We show that the proposed protocols perform well not only when the system faces a critical situation but also when there is no critical situation. By utilizing past information, the proposed protocols coordinate transmissions by users to achieve high throughput in the normal phase of operation and to let a user in a critical situation make successful transmissions while it is in the critical situation. Moreover, the proposed protocols require short memory and no message exchanges.

cs.NI↗
arXiv2009

A Systematic Framework for Dynamically Optimizing Multi-User Wireless Video Transmission

In this paper, we formulate the collaborative multi-user wireless video transmission problem as a multi-user Markov decision process (MUMDP) by explicitly considering the users' heterogeneous video traffic characteristics, time-varying network conditions and the resulting dynamic coupling between the wireless users. These environment dynamics are often ignored in existing multi-user video transmission solutions. To comply with the decentralized nature of wireless networks, we propose to decompose the MUMDP into local MDPs using Lagrangian relaxation. Unlike in conventional multi-user video transmission solutions stemming from the network utility maximization framework, the proposed decomposition enables each wireless user to individually solve its own dynamic cross-layer optimization (i.e. the local MDP) and the network coordinator to update the Lagrangian multipliers (i.e. resource prices) based on not only current, but also future resource needs of all users, such that the long-term video quality of all users is maximized. However, solving the MUMDP requires statistical knowledge of the experienced environment dynamics, which is often unavailable before transmission time. To overcome this obstacle, we then propose a novel online learning algorithm, which allows the wireless users to update their policies in multiple states during one time slot. This is different from conventional learning solutions, which often update one state per time slot. The proposed learning algorithm can significantly improve the learning performance, thereby dramatically improving the video quality experienced by the wireless users over time. Our simulation results demonstrate the efficiency of the proposed MUMDP framework as compared to conventional multi-user video transmission solutions.

cs.MM↗
arXiv2008

A New Perspective on Multi-user Power Control Games in Interference Channels

This paper considers the problem of how to allocate power among competing users sharing a frequency-selective interference channel. We model the interaction between selfish users as a non-cooperative game. As opposed to the existing iterative water-filling algorithm that studies the myopic users, this paper studies how a foresighted user, who knows the channel state information and response strategies of its competing users, should optimize its transmission strategy. To characterize this multi-user interaction, the Stackelberg equilibrium is introduced, and the existence of this equilibrium for the investigated non-cooperative game is shown. We analyze this interaction in more detail using a simple two-user example, where the foresighted user determines its transmission strategy by solving as a bi-level program which allows him to account for the myopic user's response. It is analytically shown that a foresighted user can improve its performance, if it has the necessary information about its competitors. Since the optimal solution of Stackelberg equilibrium is computationally prohibitive, we propose a practical low-complexity approach based on Lagrangian duality theory. Numerical simulations verify the performance improvements. Possible ways to acquire the required information and to extend the formulation to more than two users are also discussed.

cs.GT↗
arXiv2008

Decomposition Principles and Online Learning in Cross-Layer Optimization for Delay-Sensitive Applications

In this paper, we propose a general cross-layer optimization framework in which we explicitly consider both the heterogeneous and dynamically changing characteristics of delay-sensitive applications and the underlying time-varying network conditions. We consider both the independently decodable data units (DUs, e.g. packets) and the interdependent DUs whose dependencies are captured by a directed acyclic graph (DAG). We first formulate the cross-layer design as a non-linear constrained optimization problem by assuming complete knowledge of the application characteristics and the underlying network conditions. The constrained cross-layer optimization is decomposed into several cross-layer optimization subproblems for each DU and two master problems. The proposed decomposition method determines the necessary message exchanges between layers for achieving the optimal cross-layer solution. However, the attributes (e.g. distortion impact, delay deadline etc) of future DUs as well as the network conditions are often unknown in the considered real-time applications. The impact of current cross-layer actions on the future DUs can be characterized by a state-value function in the Markov decision process (MDP) framework. Based on the dynamic programming solution to the MDP, we develop a low-complexity cross-layer optimization algorithm using online learning for each DU transmission. This online algorithm can be implemented in real-time in order to cope with unknown source characteristics, network dynamics and resource constraints. Our numerical results demonstrate the efficiency of the proposed online algorithm.

cs.MM↗
arXiv2008

Analytical Framework for Optimizing Weighted Average Download Time in Peer-to-Peer Networks

This paper proposes an analytical framework for peer-to-peer (P2P) networks and introduces schemes for building P2P networks to approach the minimum weighted average download time (WADT). In the considered P2P framework, the server, which has the information of all the download bandwidths and upload bandwidths of the peers, minimizes the weighted average download time by determining the optimal transmission rate from the server to the peers and from the peers to the other peers. This paper first defines the static P2P network, the hierarchical P2P network and the strictly hierarchical P2P network. Any static P2P network can be decomposed into an equivalent network of sub-peers that is strictly hierarchical. This paper shows that convex optimization can minimize the WADT for P2P networks by equivalently minimizing the WADT for strictly hierarchical networks of sub-peers. This paper then gives an upper bound for minimizing WADT by constructing a hierarchical P2P network, and lower bound by weakening the constraints of the convex problem. Both the upper bound and the lower bound are very tight. This paper also provides several suboptimal solutions for minimizing the WADT for strictly hierarchical networks, in which peer selection algorithms and chunk selection algorithm can be locally designed.

cs.NI↗
arXiv2008

Conjectural Equilibrium in Water-filling Games

This paper considers a non-cooperative game in which competing users sharing a frequency-selective interference channel selfishly optimize their power allocation in order to improve their achievable rates. Previously, it was shown that a user having the knowledge of its opponents' channel state information can make foresighted decisions and substantially improve its performance compared with the case in which it deploys the conventional iterative water-filling algorithm, which does not exploit such knowledge. This paper discusses how a foresighted user can acquire this knowledge by modeling its experienced interference as a function of its own power allocation. To characterize the outcome of the multi-user interaction, the conjectural equilibrium is introduced, and the existence of this equilibrium for the investigated water-filling game is proved. Interestingly, both the Nash equilibrium and the Stackelberg equilibrium are shown to be special cases of the generalization of conjectural equilibrium. We develop practical algorithms to form accurate beliefs and search desirable power allocation strategies. Numerical simulations indicate that a foresighted user without any a priori knowledge of its competitors' private information can effectively learn the required information, and induce the entire system to an operating point that improves both its own achievable rate as well as the rates of the other participants in the water-filling game.

cs.GT↗
arXiv2007

A New Theoretic Foundation for Cross-Layer Optimization

Cross-layer optimization solutions have been proposed in recent years to improve the performance of network users operating in a time-varying, error-prone wireless environment. However, these solutions often rely on ad-hoc optimization approaches, which ignore the different environmental dynamics experienced at various layers by a user and violate the layered network architecture of the protocol stack by requiring layers to provide access to their internal protocol parameters to other layers. This paper presents a new theoretic foundation for cross-layer optimization, which allows each layer to make autonomous decisions individually, while maximizing the utility of the wireless user by optimally determining what information needs to be exchanged among layers. Hence, this cross-layer framework does not change the current layered architecture. Specifically, because the wireless user interacts with the environment at various layers of the protocol stack, the cross-layer optimization problem is formulated as a layered Markov decision process (MDP) in which each layer adapts its own protocol parameters and exchanges information (messages) with other layers in order to cooperatively maximize the performance of the wireless user. The message exchange mechanism for determining the optimal cross-layer transmission strategies has been designed for both off-line optimization and on-line dynamic adaptation. We also show that many existing cross-layer optimization algorithms can be formulated as simplified, sub-optimal, versions of our layered MDP framework.

cs.NI↗