David Holtz, Iavor Bojinov, Ramesh Johari, Nathan Kallus, Apoorva Lal, Sathya Anand, Kyle Carlson, Brent Cohn, Tom Cunningham, Alex Deng, Maria Dimakopoulou, Amit Gandhi, Victor Kostyuk, Madhav Kumar, Shyue-Ming Loh, Widad Machmouchi, Jialiang Mao, James McQueen, John Meakin, Konrad Miziolek, Dominique Perrault-Joncas, Vladimir Petrovic, Sven Schmit, James Sorenson, Matthew Wardrop, Michael Zhao, Wenjing Zheng, Martin Tingley
Large-scale randomized experiments involving millions of observations have transformed social science across academic and applied domains. These methods are now widely used in fields ranging from economics and political science to public health, education and digital product development. Drawing on the experience of industry practitioners and academic collaborators, this Perspective outlines six areas where the expanding scale and scope of experimental studies have introduced new challenges. We describe open problems related to organizational incentives and experimental governance; privacy, fairness and ethics; long-term impact estimation; time-adaptive experimental studies; heterogeneous treatment effects; and generative artificial intelligence. For each area, we summarize recent methodological advances, highlight ongoing limitations and suggest directions for future work. Our goal is to surface practical challenges that merit greater attention from the research community and to encourage sustained collaboration between academics and practitioners in shaping the future of experimental studies.