Balanced Exploration and Attention-Inspired Decision Making for Autonomous Driving

Autonomous driving has attracted great interest due to its potential capability in enhancing safety and improving traffic efficiency. Both model-based and learning-based methods are widely used in autonomous driving. Out of which, model-based methods rely on existing events in the dataset but are poor in learning extended situations. As a comparison, the deep Q-learning network (DQN) has a strong capability in learning within interactive driving. However, existing DQN faces challenges in convergence in terms of speed and accuracy, especially in interactive environments. Furthermore, the poor convergence causes high risks of collisions and slow driving speed. Therefore, this paper presents a modified DQN to achieve a lower number of collisions, higher average driving speed, and faster convergence during interactive driving. The modified DQN is developed by introducing a risk-attention mechanism, a balanced reward function, and a collision-supervised mechanism (RBDQN-CS). The proposed risk-attention mechanism enhances the DQN to pay attention to high-frequent interactions. The proposed balanced reward function specifies the weight of the control strategy to handle the interactions with surrounding human driven vehicles. The collision-supervised mechanism detects the collision risks and prevents the collision occurrence during lane-changing. Simulation results demonstrate that RBDQN-CS outperforms DQN and other popular baseline DRL algorithms.