Transformer-Guided Deep Reinforcement Learning for Optimal Takeoff Trajectory Design of an eVTOL Drone

Abstract

The rapid advancement of electric vertical takeoff and landing (eVTOL) aircraft offers a promising opportunity to alleviate urban traffic congestion but is still limited by excessive power demands, especially during the takeoff phase. Thus, developing optimal takeoff trajectories for minimum energy consumption becomes essential for broader eVTOL aircraft applications. Conventional optimal control methods (such as dynamic programming and linear quadratic regulator) provide highly efficient and well-established solutions but are prohibited by problem dimensionality and complexity. Deep reinforcement learning (DRL) emerges as a special type of artificial intelligence tackling complex, nonlinear systems; however, the training difficulty is a key bottleneck that hinders DRL applications. To address these challenges, we propose the transformer-guided DRL to alleviate the training difficulty by exploring a realistic state space at each time step using a transformer. The proposed transformer-guided DRL was demonstrated on an optimal takeoff trajectory design of an eVTOL drone for minimal energy consumption while meeting takeoff conditions (i.e., minimum vertical displacement and minimum horizontal velocity) by varying control variables (i.e., power and wing angle to the vertical). Results presented that the transformer-guided DRL agent learned to take off with $4.57\times10^6$ time steps, representing $25\%$ of the $19.79\times10^6$ time steps needed by a vanilla DRL agent. In addition, the transformer-guided DRL achieved $97.2\%$ accuracy on the optimal energy consumption compared against the simulation-based optimal reference, while the vanilla DRL achieved $96.1\%$ accuracy. Therefore, the proposed transformer-guided DRL outperformed vanilla DRL in terms of both training efficiency and optimal design verification.