Towards Hierarchical Multi-Agent Decision-Making for Uncertainty-Aware EV Charging

Abstract

Recent advances in bidirectional EV charging and discharging systems have spurred interest in workplace applications. However, real-world deployments face various dynamic factors, such as fluctuating electricity prices and uncertain EV departure times, that hinder effective energy management. To address these issues and minimize building electricity costs while meeting EV charging requirements, we design a hierarchical multi-agent structure in which a high-level agent coordinates overall charge or discharge decisions based on real-time pricing, while multiple low-level agents manage individual power level accordingly. For uncertain EV departure times, we propose a novel uncertainty-aware critic augmentation mechanism for low-level agents that improves the evaluation of power-level decisions and ensures robust control under such uncertainty. Building upon these two key designs, we introduce HUCA, a real-time charging control framework that coordinates energy supply among the building and EVs. Experiments on real-world electricity datasets show that HUCA significantly reduces electricity costs and maintains competitive performance in meeting EV charging requirements under both simulated certain and uncertain departure scenarios. The results further highlight the importance of hierarchical control and the proposed critic augmentation under the uncertain departure scenario. A case study illustrates HUCA's capability to allocate energy between the building and EVs in real time, underscoring its potential for practical use.