Bastien Le Guellec, Kokou Adambounou, Lisa C. Adams, Thibault Agripnidis, Sung Soo Ahn, Radhia Ait Chalal, Tugba Akinci D’Antonoli, P. Amouyel, Henrik Andersson, Raphaël Bentegeac, Claudio Benzoni, A Blandino, Felix Busch, Elif Can, Riccardo Cau, Armando Ugo Cavallo, C. Chavihot, Erwin Chiquete, Renato Cuocolo, Eugen Divjak, Barbara Dziadkowiec-Macek, Armel Elogne, Salvatore Claudio Fanni, Carlos Ferrarotti, Claudia Fossataro, Federica Fossataro, Katarzyna Fułek, Michał Fułek, Paweł Gać, Martyna Gachowska, Ignacio García-Juárez, Marco Gatti, Natalia Gorelik, Alexia Maria Goulianou, Aghiles Hamroun, Nicolas Fanantenana Herinirina, Quentin Holay, Gordana Ivanac, Felipe Kitamura, Michail E. Klontzas, Anna Kompanowska, Rafał Kompanowski, Krzysztof Kraik, Dominik Krupka, Alexandre Lefèvre, Tristan Lemke, Maximilian Lindholz, Piotr Macek, Marcus Makowski, Luigi Mannacio, Aymen Meddeb, L Müller, Antonio Natale, Beatrice Nguema Edzang, Adriana Ojeda, Yae Won Park, Federica Piccione, Andrea Ponsiglione, Małgorzata Poręba, Rafał Poręba, Philipp Prucker, J PRUVO, Rosa Alba Pugliesi, Feno Hasina Rabemanorintsoa, V. Rafailidis, Katarzyna Resler, Jan Rotkegel, L Saba, Ezann Siebert, Arnaldo Stanzione, Ali Fuat Tekin, Liz Toapanta‐Yanchapaxi, Matthaios Triantafyllou, Ekaterini Tsaoulia, Szymon Urban, Evangelia E Vassalou, Federica Vernuccio, Weilang Wang, Johan Wassélius, Adrian Włodarczak, Szymon Włodarczak, Andrzej Wysocki, Lina Xu, Tomasz Zatoński, S. L. Zhang, Sebastian Ziegelmayer, Grégory Kuchcinski, Keno K. Bressem
Aims To develop and validate PARROT (Polyglottal Annotated Radiology Reports for Open Testing), a multicentric, open-access dataset of fictional radiology reports spanning multiple languages for testing natural language processing applications in radiology. Methods From May to September 2024, radiologists were invited to contribute fictional radiology reports following their standard reporting practices. Contributors provided at least 20 reports with associated metadata including anatomical region, imaging modality, clinical context, and for non-English reports, English translations. All reports were assigned ICD-10 codes. A human vs. AI report differentiation study was conducted with 154 participants (radiologists, healthcare professionals, and non-healthcare professionals) assessing whether reports were human-authored or AI-generated. Results The dataset comprises 2,658 radiology reports from 76 authors across 21 countries and 13 languages. Reports cover multiple imaging modalities (CT: 36.1%, MRI: 22.8%, radiography: 19.0%, ultrasound: 16.8%) and anatomical regions, with chest (19.9%), abdomen (18.6%), head (17.3%), and pelvis (14.1%) being most prevalent. In the differentiation study, participants achieved 53.9% accuracy (95% CI: 50.7%-57.1%) in distinguishing between human and AI-generated reports, with radiologists performing significantly better (56.9%, 95% CI: 53.3%-60.6%, p<0.05) than other groups. Conclusion PARROT represents the largest open multilingual radiology report dataset, enabling testing and validation of natural language processing applications across linguistic, geographic, and clinical boundaries without privacy constraints.