arXiv subjects

Apostolos Destounis

Publications and source records attributed to Apostolos Destounis.

14 recordsLinked to original sources

arXiv2025

Adaptive Design of mmWave Initial Access Codebooks using Reinforcement Learning

Initial access (IA) is the process by which user equipment (UE) establishes its first connection with a base station. In 5G systems, particularly at millimeter-wave frequencies, IA integrates beam management to support highly directional transmissions. The base station employs a codebook of beams for the transmission of Synchronization Signal Blocks (SSBs), which are periodically swept to detect and connect users. The design of this SSB codebook is critical for ensuring reliable, wide-area coverage. In current networks, SSB codebooks are meticulously engineered by domain experts. While these expert-defined codebooks provide a robust baseline, they lack flexibility in dynamic or heterogeneous environments where user distributions vary, limiting their overall effectiveness. This paper proposes a hybrid Reinforcement Learning (RL) framework for adaptive SSB codebook design. Building on top of expert knowledge, the RL agent leverages a pool of expert-designed SSB beams and learns to adaptively select or combine them based on real-time feedback. This enables the agent to dynamically tailor codebooks to the actual environment, without requiring explicit user location information, while always respecting practical beam constraints. Simulation results demonstrate that, on average, the proposed approach improves user connectivity by 10.8$\%$ compared to static expert configurations. These findings highlight the potential of combining expert knowledge with data-driven optimization to achieve more intelligent, flexible, and resilient beam management in next-generation wireless networks.

cs.NI↗

arXiv2021

Blind Optimal User Association in Small-Cell Networks

We learn optimal user association policies for traffic from different locations to Access Points(APs), in the presence of unknown dynamic traffic demand. We aim at minimizing a broad family of $α$-fair cost functions that express various objectives in load assignment in the wireless downlink, such as total load or total delay minimization. Finding an optimal user association policy in dynamic environments is challenging because traffic demand fluctuations over time are non-stationary and difficult to characterize statistically, which obstructs the computation of cost-efficient associations. Assuming arbitrary traffic patterns over time, we formulate the problem of online learning of optimal user association policies using the Online Convex Optimization (OCO) framework. We introduce a periodic benchmark for OCO problems that generalizes state-of-the-art benchmarks. We exploit inherent properties of the online user association problem and propose PerOnE, a simple online learning scheme that dynamically adapts the association policy to arbitrary traffic demand variations. We compare PerOnE against our periodic benchmark and prove that it enjoys the no-regret property, with additional sublinear dependence of the network size. To the best of our knowledge, this is the first work that introduces a periodic benchmark for OCO problems and a no-regret algorithm for the online user association problem. Our theoretical findings are validated through results on a real-trace dataset.

cs.NI↗

arXiv2020

Multi-Agent Deep Stochastic Policy Gradient for Event Based Dynamic Spectrum Access

We consider the dynamic spectrum access (DSA) problem where $K$ Internet of Things (IoT) devices compete for $T$ time slots constituting a frame. Devices collectively monitor $M$ events where each event could be monitored by multiple IoT devices. Each device, when at least one of its monitored events is active, picks an event and a time slot to transmit the corresponding active event information. In the case where multiple devices select the same time slot, a collision occurs and all transmitted packets are discarded. In order to capture the fact that devices observing the same event may transmit redundant information, we consider the maximization of the average sum event rate of the system instead of the classical frame throughput. We propose a multi-agent reinforcement learning approach based on a stochastic version of Multi-Agent Deep Deterministic Policy Gradient (MADDPG) to access the frame by exploiting device-level correlation and time correlation of events. Through numerical simulations, we show that the proposed approach is able to efficiently exploit the aforementioned correlations and outperforms benchmark solutions such as standard multiple access protocols and the widely used Independent Deep Q-Network (IDQN) algorithm.

cs.IT↗

arXiv2020

Adaptive Coded Caching for Fair Delivery over Fading Channels

The performance of existing coded caching schemes is sensitive to the worst channel quality, a problem which is exacerbated when communicating over fading channels. In this paper, we address this limitation in the following manner: in short-term, we allow transmissions to subsets of users with good channel quality, avoiding users with fades, while in long-term we ensure fairness among users. Our online scheme combines (i) the classical decentralized coded caching scheme \cite{maddah2013decentralized} with (ii) joint scheduling and power control for the fading broadcast channel, as well as (iii) congestion control for ensuring the optimal long-term average performance. We prove that our online delivery scheme maximizes the alpha-fair utility among all schemes restricted to decentralized placement. By tuning the value of alpha, the proposed scheme can achieve different operating points on the average delivery rate region and tune performance according to an operator's choice. We demonstrate via simulations that our scheme outperforms two baseline schemes: (a) standard coded caching with multicast transmission, limited by the worst channel user yet exploiting the global caching gain; (b) opportunistic scheduling with unicast transmissions exploiting the fading diversity but limited to local caching gain.

cs.IT↗

arXiv2020

Online Convex Optimization for Caching Networks

We study the problem of wireless edge caching when file popularity is unknown and possibly non-stationary. A bank of $J$ caches receives file requests and a utility is accrued for each request depending on the serving cache. The network decides dynamically which files to store at each cache and how to route them, in order to maximize total utility. The request sequence is assumed to be drawn from an arbitrary distribution, thus capturing time-variance, temporal, or spatial locality of requests. For this challenging setting, we propose the \emph{Bipartite Supergradient Caching Algorithm} (BSCA) which provably exhibits no regret ($R_T/T \to 0$). That is, as the time horizon $T$ increases, BSCA achieves the same performance with the cache configuration that we would have chosen knowing all future requests. The learning rate of the algorithm is characterized by its regret expression, found to be $R_T=O(\sqrt{JT})$, which is independent of the content catalog size. For the single-cache case, we prove that this is the lowest attainable bound. BSCA requires at each step $J$ projections on intersections of boxes and simplices, for which we propose a tailored algorithm. Our model is the first that draws a connection between the network caching problem and Online Convex Optimization, and we demonstrate its generality by discussing various practical extensions and presenting a trace-driven comparison with state-of-the-art competitors.

cs.NI↗

arXiv2019

Learning to Cooperate in D2D Caching Networks

We consider a wireless device-to-device (D2D) cooperative network where memory-endowed nodes store and exchange content. Each node generates random file requests following an unknown and possibly arbitrary spatio-temporal process, and a base station (BS) delivers any file that is not found at its neighbors' cache, at the expense of higher cost. We design an online learning algorithm which minimizes the aggregate delivery cost by assisting each node to decide which files to cache and which files to fetch from the BS and other devices. Our policy relies on the online gradient descent algorithm, is amenable to distributed execution, and achieves asymptotically optimal performance for any request pattern, without prior information.

cs.NI↗

arXiv2019

Complexity of URLLC Scheduling and Efficient Approximation Schemes

In this paper we address the problem of joint admission control and resource scheduling for \emph{Ultra Reliable Low Latency Communications} (URLLC). We examine two models: (i) the \emph{continuous}, where all allocated resource blocks contribute to the success probability, and (ii) a \emph{binary}, where only resource blocks with strong signal are "active" for each user, and user $k$ needs $d_k$ active resource blocks for a successful URLLC transmission. In situations of congestion, we are interested in finding a subset of users that can be scheduled simultaneously. We show that finding a feasible schedule for at least $m$ URLLC users is NP-complete in the (easier) binary SNR model, hence also in the continuous. Maximizing the reward obtained from a feasible set of URLLC users is NP-hard and inapproximable to within ${(\log_2d)^2}/{d}$ of the optimal, where $d\doteq \max_kd_k$. On the other hand, we prove that checking a candidate set of users for feasibility and finding the corresponding schedule (when feasible) can be done in polynomial time, which we exploit to design an efficient heuristic algorithm for the general continuous SNR model. We complement our theoretical contributions with a numerical evaluation of our proposed schemes.

cs.IT↗

arXiv2019

Learning to Cache With No Regrets

This paper introduces a novel caching analysis that, contrary to prior work, makes no modeling assumptions for the file request sequence. We cast the caching problem in the framework of Online Linear Optimization (OLO), and introduce a class of minimum regret caching policies, which minimize the losses with respect to the best static configuration in hindsight when the request model is unknown. These policies are very important since they are robust to popularity deviations in the sense that they learn to adjust their caching decisions when the popularity model changes. We first prove a novel lower bound for the regret of any caching policy, improving existing OLO bounds for our setting. Then we show that the Online Gradient Ascent (OGA) policy guarantees a regret that matches the lower bound, hence it is universally optimal. Finally, we shift our attention to a network of caches arranged to form a bipartite graph, and show that the Bipartite Subgradient Algorithm (BSA) has no regret

cs.NI↗

arXiv2019

Learn2MAC: Online Learning Multiple Access for URLLC Applications

This paper addresses a fundamental limitation of previous random access protocols, their lack of latency performance guarantees. We consider $K$ IoT transmitters competing for uplink resources and we design a fully distributed protocol for deciding how they access the medium. Specifically, each transmitter restricts decisions to a locally-generated dictionary of transmission patterns. At the beginning of a frame, pattern $i$ is chosen with probability $p^i$, and an online exponentiated gradient algorithm is used to adjust this probability distribution. The performance of the proposed scheme is showcased in simulations, where it is compared with a baseline random access protocol. Simulation results show that (a) the proposed scheme achieves good latent throughput performance and low energy consumption, while (b) it outperforms by a big margin random transmissions.

cs.NI↗

arXiv2019

Problem-Adapted Artificial Intelligence for Online Network Optimization

Future 5G wireless networks will rely on agile and automated network management, where the usage of diverse resources must be jointly optimized with surgical accuracy. A number of key wireless network functionalities (e.g., traffic steering, power control) give rise to hard optimization problems. What is more, high spatio-temporal traffic variability coupled with the need to satisfy strict per slice/service SLAs in modern networks, suggest that these problems must be constantly (re-)solved, to maintain close-to-optimal performance. To this end, we propose the framework of Online Network Optimization (ONO), which seeks to maintain both agile and efficient control over time, using an arsenal of data-driven, online learning, and AI-based techniques. Since the mathematical tools and the studied regimes vary widely among these methodologies, a theoretical comparison is often out of reach. Therefore, the important question `what is the right ONO technique?' remains open to date. In this paper, we discuss the pros and cons of each technique and present a direct quantitative comparison for a specific use case, using real data. Our results suggest that carefully combining the insights of problem modeling with state-of-the-art AI techniques provides significant advantages at reasonable complexity.

cs.AI↗

arXiv2018

Selective Fair Scheduling over Fading Channels

Imposing fairness in resource allocation incurs a loss of system throughput, known as the Price of Fairness ($PoF$). In wireless scheduling, $PoF$ increases when serving users with very poor channel quality because the scheduler wastes resources trying to be fair. This paper proposes a novel resource allocation framework to rigorously address this issue. We introduce selective fairness: being fair only to selected users, and improving $PoF$ by momentarily blocking the rest. We study the associated admission control problem of finding the user selection that minimizes $PoF$ subject to selective fairness, and show that this combinatorial problem can be solved efficiently if the feasibility set satisfies a condition; in our model it suffices that the wireless channels are stochastically dominated. Exploiting selective fairness, we design a stochastic framework where we minimize $PoF$ subject to an SLA, which ensures that an ergodic subscriber is served frequently enough. In this context, we propose an online policy that combines the drift-plus-penalty technique with Gradient-Based Scheduling experts, and we prove it achieves the optimal $PoF$. Simulations show that our intelligent blocking outperforms by 40$\%$ in throughput previous approaches which satisfy the SLA by blocking low-SNR users.

cs.NI↗

arXiv2017

Alpha Fair Coded Caching

The performance of existing \emph{coded caching} schemes is sensitive to worst channel quality, a problem which is exacerbated when communicating over fading channels. In this paper we address this limitation in the following manner: \emph{in short-term}, we allow transmissions to subsets of users with good channel quality, avoiding users with fades, while \emph{in long-term} we ensure fairness across the different users.Our online scheme combines (i) joint scheduling and power control for the broadcast channel with fading, and (ii) congestion control for ensuring the optimal long-term average performance. We restrict the caching operations to the decentralized scheme of \cite{maddah2013decentralized}, and subject to this restriction we prove that our scheme has near-optimal overall performance with respect to the convex alpha-fairness coded caching optimization. By tuning the coefficient alpha, the operator can differentiate user performance with respect to video delivery rates achievable by coded caching. We demonstrate via simulations our scheme's superiority over legacy coded caching and unicast opportunistic scheduling, which are identified as special cases of our general framework.

cs.IT↗

arXiv2016

Asymptotically optimal pilot allocation over Markovian fading channels

We investigate a pilot allocation problem in wireless networks over Markovian fading channels. In wireless systems, the Channel State Information (CSI) is collected at the Base Station (BS), in particular, this paper considers a pilot-aided channel estimation method (TDD mode). Typically, there are less available pilots than users, hence at each slot the scheduler needs to decide an allocation of pilots to users with the goal of maximizing the long-term average throughput. There is an inherent tradeoff in how the limited pilots are used: assign a pilot to a user with up-to-date CSI and good channel condition for exploitation, or assign a pilot to a user with outdated CSI for exploration. As we show, the arising pilot allocation problem is a restless bandit problem and thus its optimal solution is out of reach. In this paper, we propose an approximation that, through the Lagrangian relaxation approach, provides a low-complexity heuristic, the Whittle index policy. We prove this policy to be asymptotically optimal in the many users regime (when the number of users in the system and the available pilots for channel sensing grow large). We evaluate the performance of Whittle's index policy in various scenarios and illustrate its remarkably good performance.

math.OC↗

arXiv2016

Streaming Big Data meets Backpressure in Distributed Network Computation

We study network response to queries that require computation of remotely located data and seek to characterize the performance limits in terms of maximum sustainable query rate that can be satisfied. The available resources include (i) a communication network graph with links over which data is routed, (ii) computation nodes, over which computation load is balanced, and (iii) network nodes that need to schedule raw and processed data transmissions. Our aim is to design a universal methodology and distributed algorithm to adaptively allocate resources in order to support maximum query rate. The proposed algorithms extend in a nontrivial way the backpressure (BP) algorithm to take into account computations operated over query streams. They contribute to the fundamental understanding of network computation performance limits when the query rate is limited by both the communication bandwidth and the computation capacity, a classical setting that arises in streaming big data applications in network clouds and fogs.

cs.NI↗