arXiv subjects

Constantin Cronrath

Publications and source records attributed to Constantin Cronrath.

3 recordsLinked to original sources

arXiv2026

Model-Based Data-Efficient and Robust Reinforcement Learning

A data-efficient learning-based control design method is proposed in this paper. It is based on learning a system dynamics model that is then leveraged in a two-level procedure. On the higher level, a simple but powerful optimization procedure is performed such that, for example, energy consumption in a vehicle can be reduced when hard state and action constraints are also introduced. Load disturbances and model errors are compensated for by a feedback controller on the lower level. In that regard, we briefly examine the robustness of both model-free and model-based learning approaches, and it is shown that the model-free approach greatly suffers from the inclusion of unmodeled dynamics. In evaluating the proposed method, it is assumed that a path is given, while the velocity and acceleration can be modified such that energy is saved, while still keeping speed limits and completion time. Compared with two well-known actor-critic reinforcement learning strategies, the suggested learning-based approach saves more energy and reduces the number of evaluated time steps by a factor of 100 or more.

eess.SY↗

arXiv2023

Reinforcement Learning for Safety Testing: Lessons from A Mobile Robot Case Study

Safety-critical robot systems need thorough testing to expose design flaws and software bugs which could endanger humans. Testing in simulation is becoming increasingly popular, as it can be applied early in the development process and does not endanger any real-world operators. However, not all safety-critical flaws become immediately observable in simulation. Some may only become observable under certain critical conditions. If these conditions are not covered, safety flaws may remain undetected. Creating critical tests is therefore crucial. In recent years, there has been a trend towards using Reinforcement Learning (RL) for this purpose. Guided by domain-specific reward functions, RL algorithms are used to learn critical test strategies. This paper presents a case study in which the collision avoidance behavior of a mobile robot is subjected to RL-based testing. The study confirms prior research which shows that RL can be an effective testing tool. However, the study also highlights certain challenges associated with RL-based testing, namely (i) a possible lack of diversity in test conditions and (ii) the phenomenon of reward hacking where the RL agent behaves in undesired ways due to a misalignment of reward and test specification. The challenges are illustrated with data and examples from the experiments, and possible mitigation strategies are discussed.

cs.RO↗

arXiv2022

Hazard Analysis of Collaborative Automation Systems: A Two-layer Approach based on Supervisory Control and Simulation

Safety critical systems are typically subjected to hazard analysis before commissioning to identify and analyse potentially hazardous system states that may arise during operation. Currently, hazard analysis is mainly based on human reasoning, past experiences, and simple tools such as checklists and spreadsheets. Increasing system complexity makes such approaches decreasingly suitable. Furthermore, testing-based hazard analysis is often not suitable due to high costs or dangers of physical faults. A remedy for this are model-based hazard analysis methods, which either rely on formal models or on simulation models, each with their own benefits and drawbacks. This paper proposes a two-layer approach that combines the benefits of exhaustive analysis using formal methods with detailed analysis using simulation. Unsafe behaviours that lead to unsafe states are first synthesised from a formal model of the system using Supervisory Control Theory. The result is then input to the simulation where detailed analyses using domain-specific risk metrics are performed. Though the presented approach is generally applicable, this paper demonstrates the benefits of the approach on an industrial human-robot collaboration system.

cs.RO↗