Establishing empirically derived non-inferiority margins for large-scale trials: An umbrella review of meta-analytic comparisons of <i>bona fide</i> psychological therapies

Objective Setting non-inferiority margins has been problematic with decision making often appearing to be arbitrary. A pragmatic approach is to use the existing literature to provide empirical reference values for these margins. The current meta-analytic umbrella review estimates empirically derived non-inferiority margins for direct psychological therapy comparisons meeting bona fide conditions.Methods A systematic search identified meta-analytic estimates of bona fide comparisons for primary and secondary outcomes. We conducted multilevel meta-analyses of these estimates to obtain generalizable empirically derived point-estimate non-inferiority margins.Results Most patients included in the analyses represented populations commonly expected in outpatient psychological therapy settings. Overall, 46 meta-analytic standardized mean differences (SMD) nested in 19 meta-analyses (mean of 15.6 primary studies and 1302 participants in each meta-analysis) were identified to estimate generalizable non-inferiority margins. The estimated homogeneously distributed margins for the primary outcomes were SMD = −.17 (95% CI = −.21, −.13) and for secondary outcomes SMD = −.16 (95% CI = −.20, −.11), respectively.Discussion This study provides empirically derived estimates for non-inferiority margins that are generalizable to large-scale trials to differentiate between small but nuanced effects and negligible non-inferior effects. Research on bona fide psychological therapy that accepts working on small effects needs careful consideration of confounding effects.