arXiv subjects

Fiona Sloothaak

Publications and source records attributed to Fiona Sloothaak.

15 recordsLinked to original sources

arXiv2026

Learning Adaptive SED for heterogeneous load balancing

We study a two-server load balancing system with heterogeneous service rates that are a priori unknown to the dispatcher. The goal is to route customers according to the Shortest--Expected--Delay (SED) policy, but this requires knowledge of the service rates. Empirical policies that route based on estimates perform poorly: due to estimation error, the empirical policy disagrees with the oracle on an infinite region of the state space. We propose an online learning algorithm that converges to SED while learning the service rates. The algorithm carefully balances empirical SED routing with forced exploration phases that guarantee sufficient sampling of both servers. We prove that our algorithm achieves finite regret; this differs from classical Multi-Armed Bandit settings where regret typically grows logarithmically in time. Finally, numerical experiments demonstrate the performance of our algorithm and highlight the regimes in which forced exploration is especially beneficial.

cs.LG↗

arXiv2026

Heavy tails in dynamic flow networks: Universal explanation of their emergence

Overload-induced cascading failures can cause extreme disruptions in a wide range of networked systems, such as power grids, transportation networks, or financial systems. Empirical studies across domains report that the size of such disruptions often follows a Pareto- or heavy-tailed distribution. While many models reproduce this scaling behavior, they are either tailored to specific domains or based on simplified mechanisms that overlook key aspects of overload cascading behavior. Hence, a general understanding of the mechanisms driving scale-free behavior in these settings remains incomplete. In this paper, we develop a universal and analytically tractable model of overload cascading failures on flow networks, offering a new perspective on how Pareto-tailed disruptions emerge across networks. Our framework shows, under mild assumptions, that heavy-tailed disruptions can arise naturally from Pareto-tailed external inputs, and it establishes a transformation law linking the input and output tail exponents. We further identify broad conditions under which the resulting cascade cost exhibits a heavy-tailed distribution and show that the mechanism is robust across several domains, including power transmission, traffic networks, and processing systems. Our results provide a unified explanation for the emergence of scale-free failures in overload-driven systems and connect previously disparate, application-specific models under a unified framework.

physics.soc-ph↗

arXiv2025

Stability of skill-based queues with deterministic waiting time thresholds

We consider skill-based queues where service is First--Come--First--Served, but some compatibility lines are only available once a customer's waiting time exceeds a threshold. For this model, we establish necessary and sufficient stability conditions. We find that the thresholds do not affect stability; an intuitive result that has not been proved in the literature up to this point. While necessity follows from a coupling argument, the sufficiency proof is more challenging. It follows by formulating the First--in--Line waiting time process as a Piecewise Deterministic Markov Process (PDMP) and constructing a Lyapunov function that accounts for the threshold structure. We show that this Lyapunov function satisfies the boundary condition of the PDMP framework, and that its generator has negative drift when waiting times are large. Finally, we show that bounded waiting-time sets are petite, so that the Foster--Lyapunov criterion applies.

math.PR↗

arXiv2025

Asymptotically optimal reinforcement learning in Block Markov Decision Processes

The curse of dimensionality renders Reinforcement Learning (RL) impractical in many real-world settings with exponentially large state and action spaces. Yet, many environments exhibit exploitable structure that can accelerate learning. To formalize this idea, we study RL in Block Markov Decision Processes (BMDPs). BMDPs model problems with large observation spaces, but where transition dynamics are fully determined by latent states. Recent advances in clustering methods have enabled the efficient recovery of this latent structure. However, a regret analysis that exploits these techniques to determine their impact on learning performance remained open. We are now addressing this gap by providing a regret analysis that explicitly leverages clustering, demonstrating that accurate latent state estimation can indeed effectively speed up learning. Concretely, this paper analyzes a two-phase RL algorithm for BMDPs that first learns the latent structure through random exploration and then switches to an optimism-guided strategy adapted to the uncovered structure. This algorithm achieves a regret that is $O(\sqrt{T}+n)$ on a large class of BMDPs susceptible to clustering. Here, $T$ denotes the number of time steps, $n$ is the cardinality of the observation space, and the Landau notation $O(\cdot)$ holds up to constants and polylogarithmic factors. This improves the best prior bound, $O(\sqrt{T}+n^2)$, especially when $n$ is large. Moreover, we prove that no algorithm can achieve lower regret uniformly on this same class of BMDPs. This establishes that, on this class, the algorithm achieves asymptotic optimality.

cs.LG↗

arXiv2025

Demonstration of effective UCB-based routing in skill-based queues on real-world data

This paper is about optimally controlling skill-based queueing systems such as data centers, cloud computing networks, and service systems. By means of a case study using a real-world data set, we investigate the practical implementation of a recently developed reinforcement learning algorithm for optimal customer routing. Our experiments show that the algorithm efficiently learns and adapts to changing environments and outperforms static benchmark policies, indicating its potential for live implementation. We also augment the real-world applicability of this algorithm by introducing a new heuristic routing rule to reduce delays. Moreover, we show that the algorithm can optimize for multiple objectives: next to payoff maximization, secondary objectives such as server load fairness and customer waiting time reduction can be incorporated. Tuning parameters are used for balancing inherent performance trade--offs. Lastly, we investigate the sensitivity to estimation errors and parameter tuning, providing valuable insights for implementing adaptive routing algorithms in complex real-world queueing systems.

cs.LG↗

arXiv2025

Emergence of Scale-Free Traffic Jams in Highway Networks: A Probabilistic Approach

Traffic congestion continues to escalate with urbanization and socioeconomic development, necessitating advanced modeling to understand and mitigate its impacts. In large-scale networks, traffic congestion can be studied using cascade models, where congestion not only impacts isolated segments, but also propagates through the network in a domino-like fashion. One metric for understanding these impacts is congestion cost, which is typically defined as the additional travel time caused by traffic jams. Recent data suggests that congestion cost exhibits a universal scale-free-tailed behavior. However, the mechanism driving this phenomenon is not yet well understood. To address this gap, we propose a stochastic cascade model of traffic congestion. We show that traffic congestion cost is driven by the scale-free distribution of traffic intensities. This arises from the catastrophe principle, implying that severe congestion is likely caused by disproportionately large traffic originating from a single location. We also show that the scale-free nature of congestion cost is robust to various congestion propagation rules, explaining the universal scaling observed in empirical data. These findings provide a new perspective in understanding the fundamental drivers of traffic congestion and offer a unifying framework for studying congestion phenomena across diverse traffic networks.

physics.soc-ph↗

arXiv2024

Learning payoffs while routing in skill-based queues

Motivated by applications in service systems, we consider queueing systems where each customer must be handled by a server with the right skill set. We focus on optimizing the routing of customers to servers in order to maximize the total payoff of customer--server matches. In addition, customer--server dependent payoff parameters are assumed to be unknown a priori. We construct a machine learning algorithm that adaptively learns the payoff parameters while maximizing the total payoff and prove that it achieves polylogarithmic regret. Moreover, we show that the algorithm is asymptotically optimal up to logarithmic terms by deriving a regret lower bound. The algorithm leverages the basic feasible solutions of a static linear program as the action space. The regret analysis overcomes the complex interplay between queueing and learning by analyzing the convergence of the queue length process to its stationary behavior. We also demonstrate the performance of the algorithm numerically, and have included an experiment with time-varying parameters highlighting the potential of the algorithm in non-static environments.

cs.LG↗

arXiv2024

Scale-free cascading failures: Generalized approach for all simple, connected graphs

Cascading failures, wherein the failure of one component triggers subsequent failures in complex interconnected systems, pose a significant risk of disruptions and emerge across various domains. Understanding and mitigating the risk of such failures is crucial to minimize their impact and ensure the resilience of these systems. In multiple applications, the failure processes exhibit scale-free behavior in terms of their total failure sizes. Various models have been developed to explain the origin of this scale-free behavior. A recent study proposed a novel hypothesis, suggesting that scale-free failure sizes might be inherited from scale-free input characteristics in power networks. However, the scope of this study excluded certain network topologies. Here, motivated by power networks, we strengthen this hypothesis by generalizing to a broader range of graph topologies where this behavior is manifested. Our approach yields a universal theorem applicable to all simple, connected graphs, revealing that when a cascade leads to network disconnections, the total failure size exhibits a scale-free tail inherited from the input characteristics. We do so by characterizing cascade sequences of failures in the asymptotic regime.

math.PR↗

arXiv2023

Failure behavior in a connected configuration model under a critical loading mechanism

We study a cascading edge failure mechanism on a connected random graph with a prescribed degree sequence, sampled using the configuration model. This mechanism prescribes that every edge failure puts an additional strain on the remaining network, possibly triggering more failures. We show that under a critical loading mechanism that depends on the global structure of the network, the number of edge failure exhibits scale-free behavior (up to a certain threshold). Our result is a consequence of the failure mechanism and the graph topology. More specifically, the critical loading mechanism leads to scale-free failure sizes for any network where no disconnections take place. The disintegration of the configuration model ensures that the dominant contribution to the failure size comes from edge failures in the giant component, for which we show that the scale-free property prevails. We prove this rigorously for sublinear thresholds, and we explain intuitively why the analysis follows through for linear thresholds. Moreover, our result holds for other graph structures as well, which we validate with simulation experiments.

math.PR↗

arXiv2023

After-sales services during an asset's lifetime: collaborative planning of system upgrades

We consider a physical asset consisting of complex systems, where the systems may require upgrades during the lifetime of the asset. In practice, the asset owner and system supplier can make the upgrade decisions together, requiring a decision support model that can be jointly used to optimize the total benefit for both parties. Motivated by a real-life use case including an asset owner and a system supplier, we build a continuous-time model to optimize the upgrade decisions of a system during the fixed lifetime of the asset. In our model, we capture the key critical factors that drive the upgrade decisions: increasing functionality requirements due to evolving technology, age-dependent maintenance costs, a predetermined overhaul plan of the asset, and the lifetime of the asset. A system upgrade is less costly if it is executed jointly with an asset overhaul. We first analyze the case with no additional cost of upgrading outside an overhaul. We analytically characterize the structure of the optimal upgrade policy under various realistic assumptions that lead to different types of cost functions. We then use these results as a building block to characterize the optimal policy for a generalized cost function. When there is a penalty for upgrading outside an overhaul moment, we propose a dynamic programming approach that efficiently determines the optimal upgrade policy by using our analytical results. We also prove that as this penalty increases, the optimal policy can only change to one where the number of upgrades not jointly executed with overhauls is reduced. However, the optimal number of upgrades is a non-increasing function of this penalty. Also surprisingly, more overhauls may lead to a smaller number of upgrades under the optimal policy.

math.OC↗

arXiv2021

Complete resource pooling of a load balancing policy for a network of battery swapping stations

To reduce carbon emission in the transportation sector, there is currently a steady move taking place to an electrified transportation system. This brings about various issues for which a promising solution involves the construction and operation of a battery swapping infrastructure rather than in-vehicle charging of batteries. In this paper, we study a closed Markovian queueing network that allows for spare batteries under a dynamic arrival policy. We propose a provisioning rule for the capacity levels and show that these lead to near-optimal resource utilization, while guaranteeing good quality-of-service levels for Electric Vehicle (EV) users. Key in the derivations is to prove a state-space collapse result, which in turn implies that performance levels are as good as if there would have been a single station with an aggregated number of resources, thus achieving complete resource pooling.

math.PR↗

arXiv2020

Emergence of scale-free blackout sizes in power grids

We model power grids as graphs with heavy-tailed sinks, which represent demand from cities, and study cascading failures on such graphs. Our analysis links the scale-free nature of blackout sizes to the scale-free nature of city sizes, contrasting previous studies suggesting that this nature is governed by self-organized criticality. Our results are based on a new mathematical framework combining the physics of power flow with rare event analysis for heavy-tailed distributions, and are validated using various synthetic networks and the German transmission grid.

physics.soc-ph↗

arXiv2017

The impact of a network split on cascading failure processes

Cascading failure models are typically used to capture the phenomenon where failures possibly trigger further failures in succession, causing knock-on effects. In many networks this ultimately leads to a disintegrated network where the failure propagation continues independently across the various components. In order to gain insight in the impact of network splitting on cascading failure processes, we extend a well-established cascading failure model for which the number of failures obeys a power-law distribution. We assume that a single line failure immediately splits the network in two components, and examine its effect on the power-law exponent. The results provide valuable qualitative insights that are crucial first steps towards understanding more complex network splitting scenarios.

physics.soc-ph↗

arXiv2017

First-passage time asymptotics over moving boundaries for random walk bridges

We study the asymptotic tail probability of the first-passage time over a moving boundary for a random walk conditioned to return to zero, where the increments of the random walk have finite variance. Typically, the asymptotic tail behavior may be described through a regularly varying function with exponent -1/2, where the impact of the boundary is captured by the slowly varying function. Yet, the moving boundary may have a stronger effect when the tail is considered at a time close to the return point of the random walk bridge. In the latter case, a phase transition appears in the asymptotics, of which the precise nature depends on the order of distance between zero and the moving boundary.

math.PR↗

arXiv2016

The restricted Erlang-R Queue: Finite-size effects in service systems with returning customers

Motivated by health care systems with repeated services that have both personnel (nurse and physician) and space (beds) constraints, we study a restricted version of the Erlang-R model. The space restriction policies we consider are blocking or holding in a pre-entrant queue. We develop many-server approximations for the system performance measures when either policy applies, and explore the connection between them. We show that capacity allocation of both resources should be determined simultaneously, and derive the methodology to determine it explicitly. We show that the system dynamics is captured by the fraction of needy time in the network, and that returning customers should be accounted for both in steady-state and time-varying conditions. We demonstrate the application of our policies in two case studies of resource allocation in hospitals.

math.PR↗