Daniel Koutas | Searcharxiv

arXiv2026

Branching out: Prognostics-Based Replacement Policies for Series Systems

We propose a hybrid planning method for deriving prognostics-based predictive maintenance policies. The method accounts for the available decision options, the information on the future state of the system provided by a prognostic model, and the costs of the underlying renewal-reward process. It results in policies defined by only a few parameters, which can be determined based on theoretical considerations or by optimization from run-to-failure data. We demonstrate the potential of the method in two separate predictive maintenance decision settings: preventive replacement and preventive ordering. Numerical investigations show that the derived policies rival the performance of optimized benchmark policies, while being significantly more efficient and robust against overfitting.

math.OC↗

arXiv2025

Leaf It to Renewal: Improved Predictive Maintenance Policies via Renewal Theory and Decision Trees

We propose a hybrid planning method for deriving prognostics-based predictive maintenance policies. The method accounts for the available decision options, the information on the future state of the system provided by a prognostic model, and the costs of the underlying renewal-reward process. It results in policies defined by only a few parameters, which can be determined based on theoretical considerations or by optimization from run-to-failure data. We demonstrate the potential of the method in two separate predictive maintenance decision settings: preventive replacement and preventive ordering. Numerical investigations show that the derived policies rival the performance of optimized benchmark policies, while being significantly more efficient and robust against overfitting.

math.OC↗

arXiv2025

Convex Is Back: Solving Belief MDPs With Convexity-Informed Deep Reinforcement Learning

We present a novel method for Deep Reinforcement Learning (DRL), incorporating the convex property of the value function over the belief space in Partially Observable Markov Decision Processes (POMDPs). We introduce hard- and soft-enforced convexity as two different approaches, and compare their performance against standard DRL on two well-known POMDP environments, namely the Tiger and FieldVisionRockSample problems. Our findings show that including the convexity feature can substantially increase performance of the agents, as well as increase robustness over the hyperparameter space, especially when testing on out-of-distribution domains. The source code for this work can be found at https://github.com/Dakout/Convex_DRL.

cs.LG↗

arXiv2023

An investigation of belief-free DRL and MCTS for inspection and maintenance planning

We propose a novel Deep Reinforcement Learning (DRL) architecture for sequential decision processes under uncertainty, as encountered in inspection and maintenance (I&M) planning. Unlike other DRL algorithms for (I&M) planning, the proposed +RQN architecture dispenses with computing the belief state and directly handles erroneous observations instead. We apply the algorithm to a basic I&M planning problem for a one-component system subject to deterioration. In addition, we investigate the performance of Monte Carlo tree search for the I&M problem and compare it to the +RQN. The comparison includes a statistical analysis of the two methods' resulting policies, as well as their visualization in the belief space.

cs.AI↗