Clintin P Davis-Stober, Konstantina Sokratous, Joachim Vandekerckhove
Here we highlight a different source of epistemic uncertainty or overconfidence; one that emerges even when statistical tools are used correctly, effect sizes are precisely estimated, findings replicate without controversy, and heterogeneity is explicitly acknowledged within hierarchical models.
Shiffrin et al. argue that scientific practice often produces illusions of understanding; situations in which familiar inferential tools generate misplaced confidence. We describe a related illusion that arises not from statistical misapplications, but from ordinary theory testing itself. Here we highlight a different source of epistemic uncertainty or overconfidence; one that emerges even when statistical tools are used correctly, effect sizes are precisely estimated, findings replicate without controversy, and heterogeneity is explicitly acknowledged within hierarchical models.