arXiv subjects

Ana Bušić

Publications and source records attributed to Ana Bušić.

At least 19 recordsLinked to original sources

arXiv2026

Stability in stochastic hypergraph matching I: necessary and sufficient criteria

Stochastic matching on hypergraphs is an important topic for its versatility in capturing real-life systems, from living donor transplant to ride-hailing. Nevertheless, finding necessary and sufficient criteria for stability is a long-standing problem. One of the key difficulties is the fact that greedy policies, whilst maximally stable for stochastic matching on graphs, no longer achieve maximal stability region on hypergraphs. So far, no alternative families of policies with similar properties have been known. In this work, we introduce online assignment policies, in which each item is assigned to a matching hyperedge type upon arrival. We show that this is a good generalisation to greedy policies, by proving that they are maximally stable. Their natural amenability to analysis allow us to derive several necessary and sufficient criteria for stability, which generalise the known criteria for graphs. Furthermore, the constructive proof gives a maximally stable arrival-rate agnostic policy.

cs.DM↗

arXiv2026

Stability in stochastic hypergraph matching II: weights, batch arrivals, and continuous time

Many real-life systems can be found as examples of stochastic matching on hypergraphs, such as production lines or assemble-to-order systems. Two common features are the number of items required may vary between matchings, and there may intermediary items which exist as a combination of other items and not of external arrivals. Both of these phenomena can be modelled by considering the weighted variant of stochastic matching. In this work, we formalise the notion of stochastic weighted matching on hypergraphs. We also allow batch arrivals, meaning multiple items of multiple classes may arrive at the same time, and in particular, the arrivals can be correlated between classes. Unlike the classical setting where items arrive at discrete time $t \in \mathbb{N}$, we allow arrival processes to take place in continuous time $t \in \mathbb{R}_{\geq 0}$. We then extend the results from Nguyen and Bušić (2026) to overcome the intricacies brought up by this new setting. This allows us to derive necessary and sufficient criteria as direct generalisations of those in the unweighted setting, which depend only on the per-class arrival rates. As such, the correlation between classes bear no differences. The constructive proofs also give a maximally stable, periodic-review, size-based, arrival-rate agnostic policy.

math.PR↗

arXiv2026

Stability in stochastic hypergraph matching III: general reneging

In many real-life matching problems, waiting agents might abandon before being matched, such as patients deceasing before receiving organs, passengers/drivers cancelling ride requests, or raw materials/intermediary products degrading in production lines. This poses the need for incorporating reneging in stochastic matching models. In this work, we consider matching models on hypergraphs with batch arrivals and general-weight matchings. Since our model allows fractional weights, we may not be able to talk about individual items, and thus reneging is not required to be independent between items of the same class. For the simplicity sake's, we assume items arrive at discrete time. We show that stability depends on the exact nature of reneging, in stark contrast with the non-reneging case where it depends on the arrivals only through the arrival rates. To our best knowledge, this is the first such sensitivity result in stochastic matching. We uncover a new stabilising mechanism which exists neither in the non-reneging case nor in the graph case, which explains why incorporating reneging in stochastic matching is not straightforward. Together with balancing mechanism as hinted in the online assignment framework for the non-reneging case, it gives a criterion necessary and sufficient for stability. Finally, whilst verifying stability is a hard problem, we give a family of MaxWeight-type policies parameterised by $\varepsilon > 0$, which are maximally stabilising for all $\varepsilon$ sufficiently small. Unfortunately there is no effective bound for $\varepsilon$, but we show how to adjust its value during implementation.

math.PR↗

arXiv2026

Proportional dispatch and fairness in wind farm power tracking

Controlling the power output of a wind farm to track a target signal enables contributing to the power grid stability. This can be achieved by dividing the target signal into individual turbine power setpoints, which are then tracked by the corresponding turbine controllers. In this work, we address the problem of finding power allocations that fairly spread the power reserves (i.e. the unused fraction of available powers) among turbines, thereby improving robustness to uncertainties and varying wind conditions. Specifically, we investigate the fairness properties of proportional dispatch, the most widely used power allocation strategy. We show that, because of wake interactions within the wind farm, proportional dispatch must be applied iteratively to achieve fair distribution of power reserves. We study the convergence of the iterative proportional dispatch (IPD) process to equalized reserves, and then illustrate it using both steady-state and dynamic wind farm simulators. The numerical results show that IPD closely approximates max-min fairness, a related fairness criterion, while requiring significantly less computational effort than black-box optimization. Finally, we show that IPD also reduces the complexity of the problem of fair power dispatch combined with yaw wake steering optimization.

eess.SY↗

arXiv2026

Online Stochastic Matching: A Polytope Perspective

Stochastic dynamic matching problems have recently gained attention in the stochastic-modeling community due to their diverse applications, such as supply-chain management and kidney exchange programs. In this paper, we study a matching problem where items of different classes arrive according to independent Poisson processes. Unmatched items are stored in a queue, and compatibility between items is represented by a simple graph, where items can be matched if their classes are connected. We analyze matching policies in terms of stability, delay, and long-term matching rate optimization. Our approach relies on the conservation equation, which ensures a balance between arrivals and departures in any stable system. Our main contributions are as follows. We establish a link between the existence of stable policies, the dimensionality of the solution set of the conservation equation, and the compatibility graph's structure. We describe the convex polytope formed by non-negative solutions to the conservation equation, and we design policies that can achieve or closely approximate the vertices of this polytope. When a vertex can only be approximated, we quantify the resulting trade-off between regret and delay: our policies achieve arbitrarily small regret at the cost of increasing delay, and we prove that this trade-off is unavoidable. Lastly, we discuss potential extensions of our results beyond the main assumptions of this paper.

cs.NI↗

arXiv2026

Structural Equivalence and Learning Dynamics in Delayed MARL

We formally establish the equivalence between Observation Delay (OD) and Action Delay (AD) in cooperative partially observable multi-agent systems using observation-action histories. We show that both systems generate identical admissible joint-policy sets, and their induced state-action-observation trajectories are identical in distribution, leading to identical optimal solutions in Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs). This formally generalizes existing infinite-horizon single-agent results to any-horizon partially observable cooperative multi-agent problems with decentralized policy execution, and allows any mixed-delay configuration to be reduced to a pure OD system. We further prove that in Transition-Independent MDPs (TI-MDPs), the observation-action history reduces to a tractable minimal local augmented state. However, we show through numerical experiments that although the optimal solution spaces are structurally isomorphic, the practical learning dynamics are fundamentally different. First, using the minimal local augmented state, the equivalence no longer holds when transitions are not independent. Second, operational constraints and causal credit-assignment errors in Temporal Difference (TD) algorithms induce different learning behaviors across regimes. Finally, leveraging this structural equivalence to bypass these learning challenges, we demonstrate successful multi-agent zero-shot policy transfer from OD to AD, paving the way for unified, efficient solution methods in complex delayed systems.

cs.LG↗

arXiv2025

Mean Field Control of Thermostatically Controlled Loads as Piecewise Deterministic Markov Processes

This paper presents a mean-field control approach for Piecewise Deterministic Markov Processes (PDMPs), specifically designed for controlling a large number of agents. By modeling the interactions of a large number of agents through an aggregate cost function, the proposed method mitigates the high dimensionality of the problem by focusing on a representative agent. The contribution of this work is the application of a PDMP-based mean-field control framework to the coordination of a large population of Thermostatically Controlled Loads (TCLs). Adapting this framework to TCLs requires incorporating a quality-of-service constraint ensuring that each agent's temperature remains within a specified comfort range. To achieve this, an additional jump intensity is introduced so that agents are very likely to switch between heating and cooling modes when they reach the boundaries of their temperature range. This extension to TCLs is demonstrated through Water Heaters (WHs) control, with a decentralized algorithm based on a dual formulation and stochastic gradient descent. The numerical results obtained illustrate this approach on two examples (signal tracking and taking into account energy price).

math.OC↗

arXiv2025

Moment Constrained Optimal Transport for Thermostatically Controlled Loads

Controlling large populations of thermostatically controlled loads (TCLs), such as water heaters, poses significant challenges due to the need to balance global constraints (e.g., grid stability) with individual requirements (e.g., physical limits and quality of service). In this work, we introduce a novel framework based on Moment Constrained Optimal Transport (MCOT) for distributed control of TCLs. By formulating the control problem as an optimal transport problem with moment constraints, our approach integrates global consumption constraints and physical feasibility conditions into the control design. This problem with high (or infinite) dimensionality can be reduced to a much lower finite-dimensional problem. The structure of this problem allows for computing the gradient with Monte Carlo methods by generating trajectories of TCLs. Contrary to all previous work, in our MCOT framework, it is possible to choose the sampling law, which considerably speeds up the calculations. This algorithm mitigates the need for extensive state-space discretization and significantly reduces computational complexity compared to existing methods. Numerical experiments in a water heater case study demonstrate that our MCOT-based method effectively coordinates TCLs under various constraints. We further extend our approach to an online setting, illustrating its practical applicability on simulated data.

math.OC↗

arXiv2025

How Irrationality Shapes Nash Equilibria: A Prospect-Theoretic Perspective

Noncooperative games with uncertain payoffs have been classically studied under the expected-utility theory framework, which relies on the strong assumption that agents behave rationally. However, simple experiments on human decision makers found them to be not fully rational, due to their subjective risk perception. Prospect theory was proposed as an empirically-grounded model to incorporate irrational behaviours into game-theoretic models. But, how prospect theory shapes the set of Nash equilibria when considering irrational agents, is still poorly understood. To this end, we study how prospect theoretic transformations may generate new equilibria while eliminating existing ones. Focusing on aggregative games, we show that capturing users' irrationality can preserve symmetric equilibria while causing the vanishing of asymmetric equilibria. Further, there exist value functions which map uncountable sets of equilibria in the expected-utility maximization framework to finite sets. This last result may shape some equilibrium selection theories for human-in-the-loop systems where computing a single equilibrium is insufficient and comparison of equilibria is needed.

cs.GT↗

arXiv2025

WFCRL: A Multi-Agent Reinforcement Learning Benchmark for Wind Farm Control

The wind farm control problem is challenging, since conventional model-based control strategies require tractable models of complex aerodynamical interactions between the turbines and suffer from the curse of dimension when the number of turbines increases. Recently, model-free and multi-agent reinforcement learning approaches have been used to address this challenge. In this article, we introduce WFCRL (Wind Farm Control with Reinforcement Learning), the first open suite of multi-agent reinforcement learning environments for the wind farm control problem. WFCRL frames a cooperative Multi-Agent Reinforcement Learning (MARL) problem: each turbine is an agent and can learn to adjust its yaw, pitch or torque to maximize the common objective (e.g. the total power production of the farm). WFCRL also offers turbine load observations that will allow to optimize the farm performance while limiting turbine structural damages. Interfaces with two state-of-the-art farm simulators are implemented in WFCRL: a static simulator (FLORIS) and a dynamic simulator (FAST.Farm). For each simulator, $10$ wind layouts are provided, including $5$ real wind farms. Two state-of-the-art online MARL algorithms are implemented to illustrate the scaling challenges. As learning online on FAST.Farm is highly time-consuming, WFCRL offers the possibility of designing transfer learning strategies from FLORIS to FAST.Farm.

cs.LG↗

arXiv2024

Reinforcement Learning and Regret Bounds for Admission Control

The expected regret of any reinforcement learning algorithm is lower bounded by $Ω\left(\sqrt{DXAT}\right)$ for undiscounted returns, where $D$ is the diameter of the Markov decision process, $X$ the size of the state space, $A$ the size of the action space and $T$ the number of time steps. However, this lower bound is general. A smaller regret can be obtained by taking into account some specific knowledge of the problem structure. In this article, we consider an admission control problem to an $M/M/c/S$ queue with $m$ job classes and class-dependent rewards and holding costs. Queuing systems often have a diameter that is exponential in the buffer size $S$, making the previous lower bound prohibitive for any practical use. We propose an algorithm inspired by UCRL2, and use the structure of the problem to upper bound the expected total regret by $O(S\log T + \sqrt{mT \log T})$ in the finite server case. In the infinite server case, we prove that the dependence of the regret on $S$ disappears.

cs.LG↗

arXiv2024

Reinforcement learning based demand charge minimization using energy storage

Utilities have introduced demand charges to encourage customers to reduce their demand peaks, since a high peak may cause very high costs for both the utility and the consumer. We herein study the bill minimization problem for customers equipped with an energy storage device and a self-owned renewable energy production. A model-free reinforcement learning algorithm is carefully designed to reduce both the energy charge and the demand charge of the consumer. The proposed algorithm does not need forecasting models for the energy demand and the renewable energy production. The resulting controller can be used online, and progressively improved with newly gathered data. The algorithm is validated on real data from an office building of IFPEN Solaize site. Numerical results show that our algorithm can reduce electricity bills with both daily and monthly demand charges.

math.OC↗

arXiv2023

Kullback-Leibler-Quadratic Optimal Control

This paper presents approaches to mean-field control, motivated by distributed control of multi-agent systems. Control solutions are based on a convex optimization problem, whose domain is a convex set of probability mass functions (pmfs). The main contributions follow: 1. Kullback-Leibler-Quadratic (KLQ) optimal control is a special case, in which the objective function is composed of a control cost in the form of Kullback-Leibler divergence between a candidate pmf and the nominal, plus a quadratic cost on the sequence of marginals. Theory in this paper extends prior work on deterministic control systems, establishing that the optimal solution is an exponential tilting of the nominal pmf. Transform techniques are introduced to reduce complexity of the KLQ solution, motivated by the need to consider time horizons that are much longer than the inter-sampling times required for reliable control. 2. Infinite-horizon KLQ leads to a state feedback control solution with attractive properties. It can be expressed as either state feedback, in which the state is the sequence of marginal pmfs, or an open loop solution is obtained that is more easily computed. 3. Numerical experiments are surveyed in an application of distributed control of residential loads to provide grid services, similar to utility-scale battery storage. The results show that KLQ optimal control enables the aggregate power consumption of a collection of flexible loads to track a time-varying reference signal, while simultaneously ensuring each individual load satisfies its own quality of service constraints. Keywords: Mean field games, distributed control, Markov decision processes, Demand Dispatch.

math.OC↗

arXiv2022

Can locational disparity of prosumer energy optimization due to inverter rules be limited?

To mitigate issues related to the growth of variable smart loads and distributed generation, distribution system operators (DSO) now make it binding for prosumers with inverters to operate under pre-set rules. In particular, the maximum active and reactive power set points for prosumers are based on local voltage measurements to ensure that inverter output does not cause voltage violations. However, such actions, as observed in this work, restrict the range available for local energy management, with more adverse losses on arbitrage profits for prosumers located farther away from the substation. The goal of the paper is three-fold: (a) to develop an optimal local energy optimization algorithm for activation of load flexibility and inverter-interfaced solar PV and energy storage under time-varying electricity prices; (b) to quantify the locational impact on prosumer arbitrage gains due to inverter injection rules prevalent in different energy markets; (c) to propose a computationally efficient hybrid inverter control policy which provides voltage regulation while substantially reducing locational disparity. Using numerical simulations on three identical prosumers located at different parts of a radial feeder, we show that our control policy is able to minimize locational disparity in arbitrage gains between customers at the beginning and end of the feeder to 1.4%, while PV curtailment is reduced by 91.7% compared to the base case with restrictive volt-Var and volt-watt policy.

eess.SY↗

arXiv2021

A unified framework for coordination of thermostatically controlled loads

A collection of thermostatically controlled loads (TCLs) -- such as air conditioners and water heaters -- can vary their power consumption within limits to help the balancing authority of a power grid maintain demand supply balance. Doing so requires loads to coordinate their on/off decisions so that the aggregate power consumption profile tracks a grid-supplied reference. At the same time, each consumer's quality of service (QoS) must be maintained. While there is a large body of work on TCL coordination, there are several limitations. One is that they do not provide guarantees on the reference tracking performance and QoS maintenance. A second limitation of past work is that they do not provide a means to compute a suitable reference signal for power demand of a collection of TCLs. In this work we provide a framework that addresses these weaknesses. The framework enables coordination of an arbitrary number of TCLs that: (i) is computationally efficient, (ii) is implementable at the TCLs with local feedback and low communication, and (iii) enables reference tracking by the collection while ensuring that temperature and cycling constraints are satisfied at every TCL at all times. The framework is based on a Markov model obtained by discretizing a pair of Fokker-Planck equations derived in earlier work by Malhame and Chong [21]. We then use this model to design randomized policies for TCLs. The balancing authority broadcasts the same policy to all TCLs, and each TCL implements this policy which requires only local measurement to make on/off decisions. Simulation results are provided to support these claims.

eess.SY↗

arXiv2020

Control oriented modeling of TCLs

Thermostatically controlled loads (TCLs) have the potential to be a valuable resource for the Balancing Authority (BA) of the future. Examples of TCLs include household appliances such as air conditioners, water heaters, and refrigerators. Since the rated power of each TCL is on the order of kilowatts, to provide meaningful service for the BA, it is necessary to control large collections of TCLs. To perform design of a distributed coordination/control algorithm, the BA requires a control oriented model that describes the relevant dynamics of an ensemble. Works focusing on solely modeling the ensemble date back to the 1980's, while works focusing on control oriented modeling are more recent. In this work, we contribute to the control oriented modeling literature. We leverage techniques from computational fluid dynamics (CFD) to discretize a pair of Fokker-Planck equations derived in earlier work [1]. The discretized equations are shown to admit a certain factorization, which makes the developed model useful for control design. In particular, the effects of weather and control are shown to independently effect the system dynamics.

eess.SY↗

arXiv2020

Flexibility can hurt dynamic matching system performance

We study the performance of general dynamic matching models. This model is defined by a connected graph, where nodes represent the class of items and the edges the compatibilities between items. Items of different classes arrive one by one to the system according to a given probability distribution. Upon arrival, an item is matched with a compatible item according to the First Come First Served discipline and leave the system immediately, whereas it is enqueued with other items of the same class, if any. We show that such a model may exhibit a non intuitive behavior: increasing the services ability by adding new edges in the matching graph may lead to a larger average population. This is similar to a Braess paradox. We first consider a quasicomplete graph with four nodes and we provide values of the probability distribution of the arrivals such that when we add an edge the mean number of items is larger. Then, we consider an arbitrary matching graph and we show sufficient conditions for the existence or non-existence of this paradox. We conclude that the analog to the Braess paradox in matching models is given when specific independent sets are in saturation, i.e., the system is close to the stability condition.

cs.GT↗

arXiv2020

Optimal Control of Dynamic Bipartite Matching Models

A dynamic bipartite matching model is given by a bipartite matching graph which determines the possible matchings between the various types of supply and demand items. Both supply and demand items arrive to the system according to a stochastic process. Matched pairs leave the system and the others wait in the queues, which induces a holding cost. We model this problem as a Markov Decision Process and study the discounted cost and the average cost problem. We fully characterize the optimal matching policy for complete matching graphs and for the N -shaped matching graph. In the former case, the optimal policy consists of matching everything and, in the latter case, it prioritizes the matchings in the extreme edges and is of threshold type for the diagonal edge. In addition, for the average cost problem, we compute the optimal threshold value. For more general graphs, we need to consider some assumptions on the cost of the nodes. For complete graphs minus one edge, we provide conditions on the cost of the nodes such that the optimal policy of the N-shaped matching graph extends to this case. For acyclic graphs, we show that, when the cost of the extreme edges is large, the optimal matching policy prioritizes the matchings in the extreme edges. We also study the W-shaped matching graph and, using simulations, we show that there are cases where it is not optimal to prioritize to matchings in the extreme edges.

cs.DM↗