Gudrun Schappacher-Tilp, Markus Tilp
Adjustment for multiple statistical tests has become a default marker of rigor in clinical and health research. We argue that its routine application is often conceptually inappropriate and can obscure meaningful results. The need for multiplicity correction depends not simply on how many tests are performed, but on the inferential question being asked. When the aim is to determine whether any detectable effect exists, an omnibus test is the appropriate inferential tool, and subsequent post hoc comparisons serve a descriptive role. When a small number of directed hypotheses are specified a priori , routine blanket correction across all conceivable contrasts is not required, provided the hypothesis set is fixed independently of the data. By contrast, when the aim is to identify where an effect is visible across a broader set of candidate contrasts, the problem is one of multiple inference and calls for explicit error control, such as false discovery rate control or, where stricter control is required, familywise error control. A related but distinct issue arises when multiple variables reflect different aspects of the same underlying biological process. Here, simple count-based adjustment penalizes measurement richness without addressing a coherent inferential problem and should be replaced by consideration of the latent structure of the outcome space. We propose a context-dependent framework, operationalized through a small set of guiding questions, that links inferential aims to appropriate statistical procedures across three modes: global detection, directed hypothesis testing, and exploratory localization. This framework supports more transparent and defensible statistical practice.