Reinforcement learning for bridge maintenance decision-making with two-dimensional state space

Adopting an apt decision-making approach for bridge maintenance can significantly mitigate structural deterioration, thereby prolonging its service life. Traditional maintenance decision-making methods typically rely on the bridge’s condition to make maintenance decisions, neglecting the impact of service time under nonlinear deterioration. To address this issue, this study proposes a bridge maintenance decision-making method incorporating Reinforcement Learning (RL) with the two-dimensional state space (RL-TDSS). The service time is treated as a variable and incorporated into the RL state space to establish a two-dimensional state space. Subsequently, the reward for each maintenance action is computed using the bridge state-maintenance cost reward function to update the Q-values. The optimal maintenance strategy is derived from the final Q-values. Comparison is employed between the RL-TDSS method with the condition-based maintenance (CBM), dynamic programming (DP) maintenance, and traditional RL methods. The results indicate that the RL-TDSS maintains each bridge component remaining in states 0, 1 and 2 exceeded 99.83%. Compared to RL, DP and CBM with a two-year maintenance interval, the RL-TDSS represents improvements of 13.96%, 13.72% and 32.79%, respectively. Additionally, the maintenance cost coefficients of the RL-TDSS achieved reductions of 13.96%, 13.72% and 32.79%, respectively, compared to RL, DP and CBM methods.