arXiv subjects

Hendrik W. Jordaan

Publications and source records attributed to Hendrik W. Jordaan.

3 recordsLinked to original sources

arXiv2023

Bypassing the Simulation-to-reality Gap: Online Reinforcement Learning using a Supervisor

Deep reinforcement learning (DRL) is a promising method to learn control policies for robots only from demonstration and experience. To cover the whole dynamic behaviour of the robot, DRL training is an active exploration process typically performed in simulation environments. Although this simulation training is cheap and fast, applying DRL algorithms to real-world settings is difficult. If agents are trained until they perform safely in simulation, transferring them to physical systems is difficult due to the sim-to-real gap caused by the difference between the simulation dynamics and the physical robot. In this paper, we present a method of online training a DRL agent to drive autonomously on a physical vehicle by using a model-based safety supervisor. Our solution uses a supervisory system to check if the action selected by the agent is safe or unsafe and ensure that a safe action is always implemented on the vehicle. With this, we can bypass the sim-to-real problem while training the DRL algorithm safely, quickly, and efficiently. We compare our method with conventional learning in simulation and on a physical vehicle. We provide a variety of real-world experiments where we train online a small-scale vehicle to drive autonomously with no prior simulation training. The evaluation results show that our method trains agents with improved sample efficiency while never crashing, and the trained agents demonstrate better driving performance than those trained in simulation.

cs.RO↗

arXiv2021

Learning the Subsystem of Local Planning for Autonomous Racing

The problem of autonomous racing is to navigate through a race course as quickly as possible while not colliding with any obstacles. We approach the autonomous racing problem with the added constraint of not maintaining an updated obstacle map of the environment. Several current approaches to this problem use end-to-end learning systems where an agent replaces the entire navigation pipeline. This paper presents a hierarchical planning architecture that combines a high level planner and path following system with a reinforcement learning agent that learns that subsystem of obstacle avoidance. The novel "modification planner" uses the path follower to track the global plan and the deep reinforcement learning agent to modify the references generated by the path follower to avoid obstacles. Importantly, our architecture does not require an updated obstacle map and only 10 laser range finders to avoid obstacles. The modification planner is evaluated in the context of F1/10th autonomous racing and compared to a end-to-end learning baseline, the Follow the Gap Method and an optimisation based planner. The results show that the modification planner can achieve faster average times compared to the baseline end-to-end planner and a 94% success rate which is similar to the baseline.

cs.RO↗

arXiv2021

Reward Signal Design for Autonomous Racing

Reinforcement learning (RL) has shown to be a valuable tool in training neural networks for autonomous motion planning. The application of RL to a specific problem is dependent on a reward signal to quantify how good or bad a certain action is. This paper addresses the problem of reward signal design for robotic control in the context of local planning for autonomous racing. We aim to design reward signals that are able to perform well in multiple, competing, continuous metrics. Three different methodologies of position-based, velocity-based, and action-based rewards are considered and evaluated in the context of F1/10th racing. A novel method of rewarding the agent on its state relative to an optimal trajectory is presented. Agents are trained and tested in simulation and the behaviors generated by the reward signals are compared to each other on the basis of average lap time and completion rate. The results indicate that a reward based on the distance and velocity relative to a minimum curvature trajectory produces the fastest lap times.

cs.RO↗