High-Speed Three-Dimensional Aerial Vehicle Evasion Based on a Multi-Stage Dueling Deep Q-Network

settingsOrder Article Reprints
Open AccessArticle
High-Speed Three-Dimensional Aerial Vehicle Evasion Based on a Multi-Stage Dueling Deep Q-Network
by Yefeng Yang 1,2,*,†ORCID,Tao Huang 1,2,†ORCID,Xinxin Wang 1,†,Chih-Yung Wen 2,†ORCID andXianlin Huang 1,†
1
Center for Control Theory and Guidance Technology, Harbin Institute of Technology, Harbin 150001, China
2
Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Hong Kong, China
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Aerospace 2022, 9(11), 673; https://doi.org/10.3390/aerospace9110673
Received: 23 September 2022 / Revised: 24 October 2022 / Accepted: 28 October 2022 / Published: 31 October 2022
(This article belongs to the Section Aeronautics)
Download Browse Figures Versions Notes
Abstract
This paper proposes a multi-stage dueling deep Q-network (MS-DDQN) algorithm to address the high-speed aerial vehicle evasion problem. High-speed aerial vehicle pursuit and evasion are an ongoing game attracting significant research attention in the field of autonomous aerial vehicle decision making. However, traditional maneuvering methods are usually not applicable in high-speed scenarios. Independent of the aerial vehicle model, the implemented MS-DDQN-based method searches for an approximate optimal maneuvering policy by iteratively interacting with the environment. Furthermore, the multi-stage learning mechanism was introduced to improve the training data quality. Simulation experiments were conducted to compare the proposed method with several typical evasion maneuvering policies and to reveal the effectiveness and robustness of the proposed MS-DDQN algorithm.