Optimal policy learning with observational data in multi-action scenarios: estimation, risk preference, and potential failures

Abstract This paper addresses optimal policy learning (OPL) with observational data in multi-action (or multi-arm) scenarios where a finite set of decision options is available. It is structured into three parts, each discussing estimation, risk preference, and potential failures, respectively. The first part provides a brief overview of key approaches to estimating the reward (or value) function and optimal policy within this analytical framework. It delineates the identification assumptions and statistical properties related to offline optimal policy learning estimators. The second part delves into the analysis of decision risk, revealing that optimal choices can be influenced by the decision-maker’s attitude toward risks, particularly concerning the trade-off between reward conditional mean and conditional variance. An application of the proposed model to real data is presented, illustrating how the average regret of a policy with multi-valued treatment depends on the decision-maker’s risk attitude. The third part discusses the limitations of optimal data-driven decision-making, highlighting conditions under which decision-making may falter. This aspect is linked to the failure of two fundamental assumptions essential for identifying the optimal choice: (i) overlapping and (ii) unconfoundedness. The paper provides practical guidance on how to apply multi-arm OPL across different application contexts.