Evaluating the impact of methodological trade-offs on meta-analyses and certainty of the evidence according to GRADE: Part 5 of a meta-research study of rapid reviews.

OBJECTIVE: To determine whether methodological trade-offs used in rapid reviews result in changes in treatment effects or GRADE certainty of the evidence (COE) compared to systematic reviews. DESIGN AND SETTING: We used three systematic reviews on acupuncture, education, and TENS for the management of chronic low back pain (LBP) conducted to inform the WHO guideline on non-surgical management of chronic primary LBP and simulated the conduct of three rapid reviews. We assessed the impact of three commonly used trade-offs: restricted database selection, single reviewer screening and single reviewer risk of bias (ROB) assessment. We meta-analysed studies included for each rapid review and compared them to the results of the WHO systematic reviews and identified when treatment effects would change recommendations. We used evidence profiles to update any changes to GRADE criteria and resulting COE. RESULTS: Compared to the WHO systematic reviews, our rapid reviews resulted in no changes to recommendations based on the meta-analyses in the acupuncture or TENS reviews and 15.4% in the education review. Further, between 2.9% to 51.6% of descriptive syntheses were not performed because the single study in the analysis were excluded from the rapid review, 25.8% of descriptive syntheses became meta-analyses because of false positives from single reviewer screening, and 6.45% to 13.33% of meta-analyses became descriptive syntheses because the review was reduced to a single study. The COE was affected in 13.3% in the TENS review and 51.6% in the education review due to descriptive syntheses that were not performed (exclusion of the single study in the analysis). In the acupuncture review, the COE changed in 23.5% of analyses. CONCLUSIONS: Rapid review methodological trade-offs led to few changes in treatment effects and COE. However, it is difficult to predict how they will impact quantitative and descriptive syntheses and COE because multiple factors are involved, including the breadth of the research question, volume and quality of studies, and review topic. When possible, review authors should consider few comparators and outcomes, well defined eligibility criteria and using fewer methodological trade-offs.