Using Continual Learning for Real-Time Detection of Vulnerable Road Users in Complex Traffic Scenarios

Abstract

Pedestrians and bicyclists are among the vulnerable road users (VRUs) that are inherently exposed to intricate traffic scenarios, which puts them at increased risk of sustaining injuries or facing fatal outcomes. This study presents an intelligent adaptive system that uses the YOLOv8-Dynamic (YOLOv8-D) algorithm that detects vulnerable road users and adapts in real time to prevent accidents before they occur. We select YOLOv8x as the detector by comparing it with other state-of-the-art object detection models, including Faster-RCNN, YOLOv5, YOLOv7, and variants. Compared to YOLOv5x, YOLOv8x shows improvements of 12.14% in F1 score and 45.61% in mean Average Precision (mAP). Against YOLOv7x, the improvements are 21.26% in F1 score and 128.44% in mAP. Our algorithm integrates continual learning ability in the architecture of the YOLOv8 detector to adjust to evolving road conditions flexibly, ensuring adaptability across multiple dataset domains and facilitating continuous enhancement of detection and tracking accuracy for VRUs, embracing the dynamic nature of real-world environments. In our proposed framework, we optimized the gradient descent mechanism of YOLOv8 model and train our optimized algorithm on two statistically different datasets in terms of image viewpoint and number of classes to achieve a 21.08% improvement in F1 score and a 31.86% improvement in mAP as compared to a custom YOLOv8 framework trained on a new dataset, thus overcoming the issue of catastrophic forgetting, which occurs when deep models are trained on statistically different types of datasets.