Choice of estimands and estimators affected the interpretation of results for some outcomes in a cluster-randomised trial (RESTORE) due to informative cluster size.

OBJECTIVE: In cluster-randomised trials (CRTs), different estimands can be targeted, such as the individual- or cluster-average effect. These two estimands can differ in magnitude when outcomes or treatment effects vary with cluster size (termed informative cluster size). When informative cluster size is present, commonly used estimators for CRTs, such as mixed-effects models and generalised estimating equations with an exchangeable correlation structure (termed GEEs(exch)), can be biased for both these estimands. With little documented evaluation of informative cluster size, it is currently unknown how commonly it occurs in practice. The aim of this work was to explore whether informative cluster size is present in a published CRT and to investigate its impact on trial results. STUDY DESIGN AND SETTING: We re-analysed the RESTORE CRT, which compared protocolised sedation with usual care for critically ill children. For each outcome, we first modelled the association between cluster size and outcome/treatment effect; next, we assessed the impact of informative cluster size by comparing differences between (i) individual- vs. cluster-average estimates and (ii) estimates from mixed-effects models and GEEs(exch) (which can be affected by informative cluster size) to those from IEEs (which are robust to informative cluster size). RESULTS: We found evidence of an association between cluster size and either outcomes or treatment effects for 16/33 outcomes (48%). This led to statistically significant differences between the individual- and cluster-average treatment effects for 5 of 33 outcomes (15%). There were >10% differences between (i) individual- and cluster-average treatment effect estimates for 17 outcomes (52%) and (ii) estimates from mixed-effects models/GEEs(exch) and estimates from unweighted IEEs for 12 outcomes (36%). For some outcomes, differences in the choice of estimator or estimand led to differences in the interpretation of results. For example, for the outcome postextubation stridor, the individual-average estimate showed a significant harmful effect (OR=1.65, 95% CI 1.02 to 2.67), unlike the cluster-average (OR=1.38, 95% CI 0.87 to 2.19) or GEEs(exch) estimate (OR=1.57, 95% CI 0.98, 2.50). CONCLUSION: Informative cluster size can occur in CRTs, and the use of estimators that are not clearly aligned to the target estimand can affect the interpretation of some results.