arXiv subjects

Holger Karl

Publications and source records attributed to Holger Karl.

At least 19 recordsLinked to original sources

arXiv2026

DART: Aiming for Tail-Delay Control in Reconfigurable Networks

Many systems serve different job classes by switching among configurations. Often, reconfiguration takes a stochastic amount of time that depends on direction and can differ in distribution between configurations. Reaching a target configuration may require crossing intermediate ones. Under a high-percentile delay objective, rarely served or pass-through configurations accumulate the long-waiting jobs that dominate the sojourn-time tail. Controlling that tail couples three decisions: which target to choose, which path to take, and whether and how many jobs to serve at the configurations crossed on the way. We introduce DART, an opportunistic commitment policy that commits to a target configuration to avoid myopic detours while serving a traversed configuration only when its weighted delay or backlog justifies the pause. We show how these decisions shape the weighted sojourn-time tail. Across six stress topologies, each isolating a different tail-control challenge, DART achieves the lowest P99 sojourn time in every case, up to 23% below the strongest baseline. The advantage holds under heterogeneous, heavy-tailed, and asymmetric reconfiguration times.

cs.PF↗

arXiv2026

PNap: Lifecycle-aware Edge Multi-state sleep for Energy Efficient MEC

Multi-access Edge Computings (MECs) enables low-latency services by executing applications at the network edge. To fulfill low-latency requirements of mobile users, providers have to keep multiple edge servers running at multiple locations, even when, in low-load phases, their capacity is not needed. This significantly increases energy consumption. Multi-state sleep mechanisms mitigate this issue by allowing servers to enter progressively deeper sleep states, trading energy savings for longer wake-up delays. At the same time, service execution depends on non-instantaneous lifecycle operations that cannot be performed while servers are asleep, tightly coupling energy management with service continuity. This paper introduces PowerNap (PNap), a lifecycle-aware orchestration framework that jointly manages server sleep states and service lifecycle states. By leveraging traffic forecasting, PNap jointly minimizes the number of active edge servers and service disruptions. We compare PNap against baselines approaches and a state-of-the-art approach. Results validate PNap, showing how it can reduce energy consumption by up to 14.9% with respect to a state-of-the-art solution while matching its service availability results.

cs.NI↗

arXiv2026

RIPPLE: Lifecycle-aware Embedding of Service Function Chains in Multi-access Edge Computing

In Multi-access Edge Computing networks, services can be deployed on nearby edge clouds (EC) as service function chains (SFCs) to meet strict quality of service (QoS) requirements. As users move, frequent SFC reconfigurations are required, but these are non-trivial: SFCs can serve users only when all required virtual network functions (VNFs) are available, and VNFs undergo time-consuming lifecycle operations before becoming operational. We show that ignoring lifecycle dynamics oversimplifies deployment, jeopardizes QoS, and must be avoided in practical SFC management. To address this, forecasts of user connectivity can be leveraged to proactively deploy VNFs and reconfigure SFCs. But forecasts are inherently imperfect, requiring lifecycle and connectivity uncertainty to be jointly considered. We present RIPPLE, a lifecycle-aware SFC embedding approach to deploy VNFs at the right time and location, reducing service interruptions. We show that RIPPLE closes the gap with solutions that unrealistically assume instantaneous lifecycle, even under realistic lifecycle constraints.

cs.NI↗

arXiv2026

Dynamic Server Allocation Under Stochastic Switchover on Time-Varying Links

Dynamic resource allocation to parallel queues is a cornerstone of network scheduling, yet classical solutions often fail when accounting for the overhead of switching delays to queues with superior link conditions. In particular, system performance is further degraded when switching delays are stochastic and inhomogeneous. In this domain, the myopic, Max-Weight policy struggles, as it is agnostic to switching delays. This paper introduces ACI, a non-myopic, frame-based scheduling framework that directly amortizes these switching delays. We first use a Lyapunov drift analysis to prove that backlog-driven ACI is throughput-optimal with respect to a scaled capacity region; then validate ACI's effectiveness on multi-UAV networks with an FSO backhaul. Finally, we demonstrate how adapting its core urgency metric provides the flexibility to navigate the throughput-latency trade-off.

cs.NI↗

arXiv2025

Surfing the SWAVES: Lifecycle-aware Service Placement in MEC

In Multi-access Edge Computing (MEC) networks, users covered by a mobile network can exploit edge clouds (ECs), computational resources located at the network's edge, to execute virtual network functions (VNFs). ECs are particularly useful when deploying VNFs with strict delay and availability requirements. As users roam in the network and get handed over between cells, deployed VNFs must follow users to retain the benefits of edge computing. Yet, having VNFs ready at the closest EC can be challenging: (i) ECs are not usually powerful enough to store and run any combination of VNFs simultaneously; (ii) if a VNF is not available at the needed EC, a series of time-consuming operations has to be performed before the VNF becomes operational. These limitations can be addressed by proactively starting VNFs instances at (likely) future locations, balancing better latency properties against higher resource usage. Such proactive deployment does need forecasting of user movements, but these will be imperfect, creating yet another tradeoff. We present our approach to this service provisioning problem, SWAVES. When compared on the ratio of users' unsuccessful packets, SWAVES improves such metric by orders of magnitude with respect to other proposed heuristic.

cs.NI↗

arXiv2024

Demo: Testing AI-driven MAC Learning in Autonomic Networks

6G networks will be highly dynamic, re-configurable, and resilient. To enable and support such features, employing AI has been suggested. Integrating AIin networks will likely require distributed AI deployments with resilient connectivity, e.g., for communication between RL agents and environment. Such approaches need to be validated in realistic network environments. In this demo, we use ContainerNet to emulate AI-capable and autonomic networks that employ the routing protocol KIRA to provide resilient connectivity and service discovery. As an example AI application, we train and infer deep RL agents learning medium access control (MAC) policies for a wireless network environment in the emulated network.

cs.NI↗

arXiv2024

Learning Sub-Second Routing Optimization in Computer Networks requires Packet-Level Dynamics

Finding efficient routes for data packets is an essential task in computer networking. The optimal routes depend greatly on the current network topology, state and traffic demand, and they can change within milliseconds. Reinforcement Learning can help to learn network representations that provide routing decisions for possibly novel situations. So far, this has commonly been done using fluid network models. We investigate their suitability for millisecond-scale adaptations with a range of traffic mixes and find that packet-level network models are necessary to capture true dynamics, in particular in the presence of TCP traffic. To this end, we present $\textit{PackeRL}$, the first packet-level Reinforcement Learning environment for routing in generic network topologies. Our experiments confirm that learning-based strategies that have been trained in fluid environments do not generalize well to this more realistic, but more challenging setup. Hence, we also introduce two new algorithms for learning sub-second Routing Optimization. We present $\textit{M-Slim}$, a dynamic shortest-path algorithm that excels at high traffic volumes but is computationally hard to scale to large network topologies, and $\textit{FieldLines}$, a novel next-hop policy design that re-optimizes routing for any network topology within milliseconds without requiring any re-training. Both algorithms outperform current learning-based approaches as well as commonly used static baseline protocols in scenarios with high-traffic volumes. All findings are backed by extensive experiments in realistic network conditions in our fast and versatile training and evaluation framework.

cs.LG↗

arXiv2024

Multi-Objective Optimization Using Adaptive Distributed Reinforcement Learning

The Intelligent Transportation System (ITS) environment is known to be dynamic and distributed, where participants (vehicle users, operators, etc.) have multiple, changing and possibly conflicting objectives. Although Reinforcement Learning (RL) algorithms are commonly applied to optimize ITS applications such as resource management and offloading, most RL algorithms focus on single objectives. In many situations, converting a multi-objective problem into a single-objective one is impossible, intractable or insufficient, making such RL algorithms inapplicable. We propose a multi-objective, multi-agent reinforcement learning (MARL) algorithm with high learning efficiency and low computational requirements, which automatically triggers adaptive few-shot learning in a dynamic, distributed and noisy environment with sparse and delayed reward. We test our algorithm in an ITS environment with edge cloud computing. Empirical results show that the algorithm is quick to adapt to new environments and performs better in all individual and system metrics compared to the state-of-the-art benchmark. Our algorithm also addresses various practical concerns with its modularized and asynchronous online training method. In addition to the cloud simulation, we test our algorithm on a single-board computer and show that it can make inference in 6 milliseconds.

cs.LG↗

arXiv2023

Embedding Delay-Constrained VNF Forwarding Graphs into Reconfigurable WDM Optical Networks -- Extended Version

Operators of reconfigurable wavelength-division multiplexed (WDM) optical networks adapt the lightpath topology to balance load and reduce transmission delays. Such an adaption generally depends on a known or estimated traffic matrix. Network function virtualization (NFV) allows to implicitly change this traffic matrix. However, these two degrees of freedom have largely been considered separately, using resources suboptimally. Especially for delay-sensitive services, an optimal use of resources can be crucial. We aim to jointly optimize the embedding of virtualized network function (VNF) forwarding graphs with delay constraints and the lightpath topology of WDM optical networks. Unlike previous work, we consider all three types of delays: propagation, processing and forwarding-induced queuing delay. We model the latter two as M/M/1 queues. We formulate and analyze a mixed-integer nonlinear program (MINLP), reformulate it as a mixed-integer quadratic constrained program (MIQCP) and approximate it by a mixed-integer linear program (MILP). We evaluate our approach for small-scale examples of a multicast service.

cs.NI↗

arXiv2023

Stability and Convergence of Distributed Stochastic Approximations with large Unbounded Stochastic Information Delays

We generalize the Borkar-Meyn stability Theorem (BMT) to distributed stochastic approximations (SAs) with information delays that possess an arbitrary moment bound. To model the delays, we introduce Age of Information Processes (AoIPs): stochastic processes on the non-negative integers with a unit growth property. We show that AoIPs with an arbitrary moment bound cannot exceed any fraction of time infinitely often. In combination with a suitably chosen stepsize, this property turns out to be sufficient for the stability of distributed SAs. Compared to the BMT, our analysis requires crucial modifications and a new line of argument to handle the SA errors caused by AoI. In our analysis, we show that these SA errors satisfy a recursive inequality. To evaluate this recursion, we propose a new Gronwall-type inequality for time-varying lower limits of summations. As applications to our distributed BMT, we discuss distributed gradient-based optimization and a new approach to analyzing SAs with momentum.

math.OC↗

arXiv2022

3DPG: Distributed Deep Deterministic Policy Gradient Algorithms for Networked Multi-Agent Systems

We present Distributed Deep Deterministic Policy Gradient (3DPG), a multi-agent actor-critic (MAAC) algorithm for Markov games. Unlike previous MAAC algorithms, 3DPG is fully distributed during both training and deployment. 3DPG agents calculate local policy gradients based on the most recently available local data (states, actions) and local policies of other agents. During training, this information is exchanged using a potentially lossy and delaying communication network. The network therefore induces Age of Information (AoI) for data and policies. We prove the asymptotic convergence of 3DPG even in the presence of potentially unbounded Age of Information (AoI). This provides an important step towards practical online and distributed multi-agent learning since 3DPG does not assume information to be available deterministically. We analyze 3DPG in the presence of policy and data transfer under mild practical assumptions. Our analysis shows that 3DPG agents converge to a local Nash equilibrium of Markov games in terms of utility functions expressed as the expected value of the agents local approximate action-value functions (Q-functions). The expectations of the local Q-functions are with respect to limiting distributions over the global state-action space shaped by the agents' accumulated local experiences. Our results also shed light on the policies obtained by general MAAC algorithms. We show through a heuristic argument and numerical experiments that 3DPG improves convergence over previous MAAC algorithms that use old actions instead of old policies during training. Further, we show that 3DPG is robust to AoI; it learns competitive policies even with large AoI and low data availability.

cs.LG↗

arXiv2022

Multi-Agent Reinforcement Learning for Long-Term Network Resource Allocation through Auction: a V2X Application

We formulate offloading of computational tasks from a dynamic group of mobile agents (e.g., cars) as decentralized decision making among autonomous agents. We design an interaction mechanism that incentivizes such agents to align private and system goals by balancing between competition and cooperation. In the static case, the mechanism provably has Nash equilibria with optimal resource allocation. In a dynamic environment, this mechanism's requirement of complete information is impossible to achieve. For such environments, we propose a novel multi-agent online learning algorithm that learns with partial, delayed and noisy state information, thus greatly reducing information need. Our algorithm is also capable of learning from long-term and sparse reward signals with varying delay. Empirical results from the simulation of a V2X application confirm that through learning, agents with the learning algorithm significantly improve both system and individual performance, reducing up to 30% of offloading failure rate, communication overhead and load variation, increasing computation resource utilization and fairness. Results also confirm the algorithm's good convergence and generalization property in different environments.

cs.MA↗

arXiv2022

Learning to Bid Long-Term: Multi-Agent Reinforcement Learning with Long-Term and Sparse Reward in Repeated Auction Games

We propose a multi-agent distributed reinforcement learning algorithm that balances between potentially conflicting short-term reward and sparse, delayed long-term reward, and learns with partial information in a dynamic environment. We compare different long-term rewards to incentivize the algorithm to maximize individual payoff and overall social welfare. We test the algorithm in two simulated auction games, and demonstrate that 1) our algorithm outperforms two benchmark algorithms in a direct competition, with cost to social welfare, and 2) our algorithm's aggressive competitive behavior can be guided with the long-term reward signal to maximize both individual payoff and overall social welfare.

cs.LG↗

arXiv2022

Multi-Agent Distributed Reinforcement Learning for Making Decentralized Offloading Decisions

We formulate computation offloading as a decentralized decision-making problem with autonomous agents. We design an interaction mechanism that incentivizes agents to align private and system goals by balancing between competition and cooperation. The mechanism provably has Nash equilibria with optimal resource allocation in the static case. For a dynamic environment, we propose a novel multi-agent online learning algorithm that learns with partial, delayed and noisy state information, and a reward signal that reduces information need to a great extent. Empirical results confirm that through learning, agents significantly improve both system and individual performance, e.g., 40% offloading failure rate reduction, 32% communication overhead reduction, up to 38% computation resource savings in low contention, 18% utilization increase with reduced load variation in high contention, and improvement in fairness. Results also confirm the algorithm's good convergence and generalization property in significantly different environments.

cs.MA↗

arXiv2022

Distributed gradient-based optimization in the presence of dependent aperiodic communication

Iterative distributed optimization algorithms involve multiple agents that communicate with each other, over time, in order to minimize/maximize a global objective. In the presence of unreliable communication networks, the Age-of-Information (AoI), which measures the freshness of data received, may be large and hence hinder algorithmic convergence. In this paper, we study the convergence of general distributed gradient-based optimization algorithms in the presence of communication that neither happens periodically nor at stochastically independent points in time. We show that convergence is guaranteed provided the random variables associated with the AoI processes are stochastically dominated by a random variable with finite first moment. This improves on previous requirements of boundedness of more than the first moment. We then introduce stochastically strongly connected (SSC) networks, a new stochastic form of strong connectedness for time-varying networks. We show: If for any $p \ge0$ the processes that describe the success of communication between agents in a SSC network are $α$-mixing with $n^{p-1}α(n)$ summable, then the associated AoI processes are stochastically dominated by a random variable with finite $p$-th moment. In combination with our first contribution, this implies that distributed stochastic gradient descend converges in the presence of AoI, if $α(n)$ is summable.

math.OC↗

arXiv2021

Reinforcement Learning for Admission Control in Wireless Virtual Network Embedding

Using Service Function Chaining (SFC) in wireless networks became popular in many domains like networking and multimedia. It relies on allocating network resources to incoming SFCs requests, via a Virtual Network Embedding (VNE) algorithm, so that it optimizes the performance of the SFC. When the load of incoming requests -- competing for the limited network resources - increases, it becomes challenging to decide which requests should be admitted and which one should be rejected. In this work, we propose a deep Reinforcement learning (RL) solution that can learn the admission policy for different dependencies, such as the service lifetime and the priority of incoming requests. We compare the deep RL solution to a first-come-first-serve baseline that admits a request whenever there are available resources. We show that deep RL outperforms the baseline and provides higher acceptance rate with low rejections even when there are enough resources.

cs.NI↗

arXiv2021

V-Edge: Virtual Edge Computing as an Enabler for Novel Microservices and Cooperative Computing

As we move from 5G to 6G, edge computing is one of the concepts that needs revisiting. Its core idea is still intriguing: instead of sending all data and tasks from an end user's device to the cloud, possibly covering thousands of kilometers and introducing delays that are just owed to limited propagation speed, edge servers deployed in close proximity to the user, e.g., at some 5G gNB, serve as proxy for the cloud. Yet this promising idea is hampered by the limited availability of such edge servers. In this paper, we discuss a way forward, namely the virtual edge computing (V-Edge) concept. V-Edge bridges the gap between cloud, edge, and fog by virtualizing all available resources including the end users' devices and making these resources widely available using well-defined interfaces. V-Edge also acts as an enabler for novel microservices as well as cooperative computing solutions. We introduce the general V-Edge architecture and we characterize some of the key research challenges to overcome, in order to enable wide-spread and even more powerful edge services.

cs.NI↗

arXiv2021

Practical sufficient conditions for convergence of distributed optimisation algorithms over communication networks with interference

Information exchange over networks can be affected by various forms of delay. This causes challenges for using the network by a multi-agent system to solve a distributed optimisation problem. Distributed optimisation schemes, however, typically do not assume network models that are representative for real-world communication networks, since communication links are most of the time abstracted as lossless. Our objective is therefore to formulate a representative network model and provide practically verifiable network conditions that ensure convergence of distributed algorithms in the presence of interference and possibly unbounded delay. Our network is modelled by a sequence of directed-graphs, where to each network link we associate a process for the instantaneous signal-to-interference-plus-noise ratio. We then formulate practical conditions that can be verified locally and show that the age of information (AoI) associated with data communicated over the network is in $\mathcal{O}(\sqrt{n})$. Under these conditions we show that a penalty-based gradient descent algorithm can be used to solve a rich class of stochastic, constrained, distributed optimisation problems. The strength of our result lies in the bridge between practical verifiable network conditions and an abstract optimisation theory. We illustrate numerically that our algorithm converges in an extreme scenario where the average AoI diverges.

math.OC↗