Stability-guaranteed redundant transmission scheduling over Gilbert-Elliot channels for remote state estimation: A deep reinforcement learning approach

Due to the vulnerability of wireless network control systems (WNCSs), data loss on unreliable channels severely degrades the remote state estimation performance in autonomous driving and connected vehicles. While redundant channels enhance successful transmission probability, they may lead to excessive consumption of limited vehicular network resources. To address this trade-off, this paper investigates optimal scheduling of redundant transmission in remote state estimation over dynamic vehicular links, modeled as Gilbert-Elliott channels. To minimize the long-term average estimation error in the remote estimation process, the redundant transmission scheduling problem is formulated as a Markov decision process (MDP), and deep reinforcement learning (DRL) is employed to solve the infinite-horizon reward optimization problem. To verify the stability of the established MDP model, a sufficient condition guaranteeing the boundedness of the average estimation error is derived. Furthermore, the existence of the optimal deterministic and stationary policy is established based on Bellman's optimality equation. Furthermore, considering the limitations of conventional reinforcement learning methods in processing high-dimensional vehicular communication data, the dueling double deep Q-network (D3QN) algorithm is developed to mitigate learning instability and value overestimation, enabling rapid convergence to the approximate optimal scheduling policy. In conclusion, the validity of the proposed method is corroborated through comprehensive comparisons with the periodic policy, greedy holding times policy, and the deep Q-network algorithm.