Physically informed multi-agent deep reinforcement learning for distributed voltage control in distribution networks

The increasing integration of photovoltaic (PV) generation in distribution networks poses significant voltage control challenges due to its inherent stochasticity and intermittency. Conventional control methods are either computationally inefficient or excessively sensitive to parameter tuning and measurement inaccuracies. To overcome these challenges, this paper proposes a distributed voltage control scheme for distribution networks based on the physically informed multi-agent deep reinforcement learning. For the enhancement of global situational awareness and coordinated control performance, the graph convolutional network (GCN) is integrated into the multi-agent twin delayed deep deterministic policy gradient (MATD3) algorithm, enabling the agents to dynamically perceive and exploit grid topology and spatial dependencies. Then, a modified voltage penalization mechanism and an experience fusion strategy are adopted to balance the multiple control objectives and accelerate training convergence. Furthermore, the implementation of the centralized training decentralized execution (CTDE) paradigm reduces inter-agent communication requirements while enhancing system robustness against limited information exchange, thereby facilitating reliable distributed decision-making. Case studies are carried out to validate the proposed scheme, showing rapid convergence, minimized voltage violations, and reduced power losses. Moreover, the proposed scheme exhibits strong robustness against measurement data loss, highlighting its practical applicability in the complex environments of distribution networks. © 2017 Elsevier Inc. All rights reserved.