Publisher coverage bias in the Stanford/Ioannidis “top 2% of scientists” rankings

Abstract The “top 2% of scientists” rankings have grown in popularity as a means of research evaluation, with the potential to influence hiring, promotion, and funding decisions for over 230,000 researchers worldwide. We documented 123 universities across 32 countries and cited these rankings in official communications. The rankings rely exclusively on Scopus, a citation database owned by Elsevier, which also publishes a large portfolio of journals indexed in Scopus. We drew a stratified random sample of 600 researchers from the rankings (n=564 successfully matched) across book-heavy, mixed, and journal-heavy fields, and compared each researcher’s Scopus publication count to their OpenAlex record. Median Scopus coverage was 45.3% of OpenAlex publications. We documented systematic coverage disparities: researchers publishing in Elsevier journals had 30.6 percentage points higher coverage than those publishing elsewhere (20.8 pp under an any-Elsevier-vs-none contrast), even though our sample contained more than twice as many Oxford, Cambridge, Wiley, and Springer publications as Elsevier publications. The Elsevier coverage advantage persists even when comparing journals of similar citation impact. Book-focused scholars fare worse: humanities researchers show 43.6 percentage points lower coverage than STEM researchers, with Nobel and Pulitzer Prize winners among those affected. When we recalculated the rankings using the complete OpenAlex dataset, individual researchers shifted by a median of 142,276 positions, with the largest shift exceeding 1.2 million positions. These findings indicate that widely used scientist rankings may reflect both database coverage patterns and research impact, raising concerns about their use in career-determining decisions.