Autonomous penetration testing using reinforcement learning: A review and perspectives

Penetration testing (pentesting) assesses cybersecurity through controlled, authorized attacks, but traditional manual methods demand considerable human and time resources. Reinforcement learning (RL), with its agent-environment interaction paradigm, offers a promising approach for autonomous pentesting. Despite remarkable advancements in this field, there is a lack of comprehensive reviews and perspectives on RL-based autonomous pentesting. To address this gap, this paper presents a systematic review of RL-based autonomous pentesting research. We outline the key challenges faced when applying RL in autonomous pentesting and categorize the existing literature into two main areas: attack path planning and autonomous pentesting frameworks, based on the research objectives and hypotheses. Additionally, we offer an in-depth analysis of the latest advancements and limitations in this field, while proposing a perspective on future research directions in the field of RL-based autonomous pentesting. We hope that our work will provide valuable insights for researchers, contributing to the advancement of autonomous pentesting and its practical application in the complex and diverse scenarios of the real world.