Intuitions of compromise: Utilitarianism, egalitarianism, contractualism.

What is the best compromise in a situation where different people value different things? The most common method for answering this question is to look at all the options, add up the utility per person associated with each, and pick the option with the largest sum. This approach seems like the obvious, theory-neutral solution, though, in fact, it relies on substantive utilitarian commitments. A second, quite common approach, involves integrating egalitarian concerns, and picking an option that trades off maximizing utility with equalizing outcomes. But there is an important, though often-ignored, third approach as well: a contractualist approach, which models what rational actors would agree to. In this paper, we test the proposals of these three competing preference aggregation algorithms in social decision-making contexts that are systematically sampled across a wide range. While the dominant approach to value aggregation up to now has been utilitarian, we find that participants strongly prefer the aggregations recommended by the contractualist algorithm, which also dominates the egalitarian approach across a range of parameterizations. Finally, we compare the judgments of large language models (LLMs) to those of our (human) participants, finding important misalignment between model and human preferences.