Ensuring the exercise of human agency in AI-based military systems: concerns across the lifecycle

Abstract Over the past years, the number of governance initiatives on applications of artificial intelligence (AI) in the military domain has expanded. As actors across the governance landscape turn towards implementing these initiatives, principles will need to be spelled out in practical terms, marking a decisive phase in the governance process. This includes the exercise of human agency in the context of using AI-based systems in the military domain. This paper considers what the notion of exercising human agency means across the lifecycle of AI systems. A lifecycle framework acknowledges that ensuring a qualitatively high exercise of human agency in AI-based systems cannot rely exclusively on the tail-end of the targeting decision-making process. Rather, it needs to be built into the lifecycle of AI-based systems from before the potential development of such systems all the way to post-use review. Each of the lifecycle stages raises manifold questions and challenges that various stakeholders need to address in their efforts to sustain and strengthen human agency. The paper highlights twelve key technical, ethical, legal, and strategic concerns across different stages of the lifecycle. These sets of concerns illustrate the value of developing more fine-grained thinking around applied lifecycle models. We conclude that ensuring the exercise of human agency in the use of AI-based systems in military contexts will require careful and reflective decision-making around questions and challenges among the stakeholders involved.