Learning-based Ecological Adaptive Cruise Control of Autonomous Electric Vehicles: A Comparison of ADP, DQN and DDPG Approaches

Abstract

This paper presents model-based and model-free learning methods for economic and ecological adaptive cruise control (Eco-ACC) of connected and autonomous electric vehicles. For model-based optimal control of Eco-ACC, we considered longitudinal vehicle dynamics and a quasi-steady-state powertrain model including the physical limits of a commercial electric vehicle. We used adaptive dynamic programming (ADP), in which the value function was trained using data obtained from IPG CarMaker simulations. For real-time implementation, forward multi-step look-ahead prediction and optimization were executed in a receding horizon scheme to maximize the energy efficiency of the electric machine while avoiding rear-end collisions and satisfying the powertrain, speed, and distance-gap constraints. For model-free optimal control of Eco-ACC, we applied two reinforcement learning methods, Deep Q-Network (DQN) and Deep Deterministic Policy Gradient (DDPG), in which deep neural networks were trained in IPG CarMaker simulations. For performance demonstrations, the HWFET, US06, and WLTP Class 3b driving cycles were used to simulate the front vehicle, and the energy consumptions of the host vehicle and front vehicle were compared. In high-fidelity IPG CarMaker simulations, the proposed learning-based Eco-ACC methods demonstrated approximately 3-5% and 10-14% efficiency improvements in highway and city-highway driving scenarios, respectively, compared with the front vehicle. A video of the CarMaker simulation is available at https://youtu.be/DIXzJxMVig8.