Data-Driven Noncooperative Game for UAVs At Intersection Passages Based on Q-Learning

This paper investigates the uncrewed aerial vehicles (UAVs) control problem at intersection passages for narrow channels formed by obstacles. A set of non-cooperative value functions is designed to iteratively determine the optimal passage policies for UAV swarms in different directions outside the channels. To tackle the challenge of solving the high-dimensional game algebraic Riccati equation (GARE), a decoupling approach is introduced, which leads the development of a multi-stage decoupled Q-learning algorithm for non-cooperative games (MDQNG). Rigorous mathematical proofs are provided to validate the decouplability of system dynamics and the GARE equation. Additionally, a comparative simulation between the MDQNG algorithm and deep Q-network (DQN) is conducted, demonstrating the effectiveness and superiority of the MDQNG algorithm in solving optimal control policies.