Searcharxiv⌕ Search

arXiv subjects

David Krame Kadurha

Publications and source records attributed to David Krame Kadurha.

3 recordsLinked to original sources

A Rollout-Based Approach to Real-Time Topology and Dispatch Control in the AI4RealNet L2RPN Benchmark

Operating transmission networks under uncertain outages requires coordinating discrete network topology actions with continuous adjustments to generation and storage, while respecting operational constraints and real-time computational limits. We present a rollout-based controller that combines discrete and continuous decisions under the uncertain contingency policy of the L2RPN AI4RealNet 2026 challenge. Power-flow approximations and convex optimization identify the most promising actions, followed by short-horizon simulations to predict their effect on congestion, renewable curtailment, and redispatching costs. A further policy exploits the benchmark's specific attack protocol: keeping selected lines disconnected prevents some adversarial outages from being initiated. On the benchmark's $118$-bus network, the controller completed $189$ of $208$ final-test scenarios (score $64.39$) and achieved the second-best ratio of score to computation time among the five top-ranked entries, behind one that scored lower. Our results show that combining structured optimization with simulation-based action selection is effective even without a learned decision rule, and fast enough for real-time control. Code and reproduction material: https://github.com/DavidKrame/RAD_agent__AI4RealNet_competition.

eess.SY↗

Bellman operator convergence enhancements in reinforcement learning algorithms

This paper reviews the topological groundwork for the study of reinforcement learning (RL) by focusing on the structure of state, action, and policy spaces. We begin by recalling key mathematical concepts such as complete metric spaces, which form the foundation for expressing RL problems. By leveraging the Banach contraction principle, we illustrate how the Banach fixed-point theorem explains the convergence of RL algorithms and how Bellman operators, expressed as operators on Banach spaces, ensure this convergence. The work serves as a bridge between theoretical mathematics and practical algorithm design, offering new approaches to enhance the efficiency of RL. In particular, we investigate alternative formulations of Bellman operators and demonstrate their impact on improving convergence rates and performance in standard RL environments such as MountainCar, CartPole, and Acrobot. Our findings highlight how a deeper mathematical understanding of RL can lead to more effective algorithms for decision-making problems.

cs.LG↗

Topological Foundations of Reinforcement Learning

The goal of this work is to serve as a foundation for deep studies of the topology of state, action, and policy spaces in reinforcement learning. By studying these spaces from a mathematical perspective, we expect to gain more insight into how to build better algorithms to solve decision problems. Therefore, we focus on presenting the connection between the Banach fixed point theorem and the convergence of reinforcement learning algorithms, and we illustrate how the insights gained from this can practically help in designing more efficient algorithms. Before doing so, however, we first introduce relevant concepts such as metric spaces, normed spaces and Banach spaces for better understanding, before expressing the entire reinforcement learning problem in terms of Markov decision processes. This allows us to properly introduce the Banach contraction principle in a language suitable for reinforcement learning, and to write the Bellman equations in terms of operators on Banach spaces to show why reinforcement learning algorithms converge. Finally, we show how the insights gained from the mathematical study of convergence are helpful in reasoning about the best ways to make reinforcement learning algorithms more efficient.

cs.LG↗