Joint Multi-Agent Reinforcement Learning and Message-Passing for Resilient Multi-UAV Networks

This paper introduces a novel resilient algorithm designed for distributed unmanned aerial vehicles (UAVs) in dynamic and unreliable network environments. Initially, the UAVs should be trained via multi-agent reinforcement learning (MARL) for autonomous mission-critical operations and are fundamentally grounded by centralized training and decentralized execution (CTDE) using a centralized MARL server. In this situation, it is crucial to consider the case where several UAVs cannot receive CTDE-based MARL learning parameters for resilient operations in unreliable network conditions. To tackle this issue, a communication graph is used where its edges are established when two UAVs/nodes are communicable. Then, the edge-connected UAVs can share their training data if one of the UAVs cannot be connected to the CTDE-based MARL server under unreliable network conditions. Additionally, the edge cost considers power efficiency. Based on this given communication graph, message-passing is used for electing the UAVs that can provide their MARL learning parameters to their edge-connected peers. Lastly, performance evaluations demonstrate the superiority of our proposed algorithm in terms of power efficiency and resilient UAV task management, outperforming existing benchmark algorithms.