Programmed to please: the moral and epistemic harms of AI sycophancy

Abstract AI sycophancy is the tendency of large language models to prioritize user approval over truth. While there has been recent technical research characterizing the phenomenon, it remains undertheorized within AI ethics. This article offers a conceptual analysis of AI sycophancy. We maintain that it is a distinctively intractable problem in AI ethics, rooted in reinforcement learning from human feedback (RLHF) and exacerbated by economic and philosophical constraints. We analyze AI sycophancy through the lens of Aristotelian virtue ethics, arguing that it is an artificial vice that generates moral and epistemic harms for individuals and liberal-democratic institutions. Drawing on Aristotle’s distinction between the obsequious sycophant and the flattering sycophant, we contend that AI sycophancy is best understood as the former, and that the companies that profit from it may be characterized in terms of the latter. We then explain how sycophancy prevents the possibility of true Aristotelian friendship with AI (even if the AI were conscious) and examine how multimodal AI systems may amplify these sycophantic tendencies in increasingly difficult-to-detect ways. We conclude by outlining policy and design interventions, as well as alternative reinforcement learning approaches that might cultivate artificial virtue rather than vice.