Intra- and interobserver variability in response evaluation metrics used for localized soft tissue sarcoma

Background Reliable imaging-based response assessment in localized soft tissue sarcoma (STS) is essential for guiding treatment and interpreting clinical trial outcomes. We quantified intra- and interobserver variability and evaluated its impact on classification using Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 and magnetic resonance (MR)-adapted Choi criteria across different measurement tools. Materials and methods Nineteen pre- and postradiotherapy magnetic resonance imaging pairs were evaluated by 10 radiologists using visual estimation, digital rulers, and segmentation; signal intensity was normalized using circular or freehand muscle regions of interest. Intraobserver error was assessed through mirrored lesions. Interobserver error analysis was split into systematic and random components. Clinical impact of measurement variability was evaluated by analyzing response category threshold crossings and agreement in RECIST 1.1 and MR-adapted Choi classifications under realistic clinical scenarios, including when baseline and follow-up scans were assessed by the same or different readers. Results Segmentation-based diameters showed the lowest variability and highest agreement. RECIST 1.1 classifications were consistent across readers when using segmentation (Fleiss' k = 0.92-1.00), but ruler-based measurements showed lower agreement (Fleiss' k = 0.40-0.46). MR-adapted Choi criteria were more variable, particularly for signal intensity measurements, which led to false progression/response calls in up to 16% of identical scan comparisons. Agreement when comparing real-world baseline follow-up scenarios was also lower (Fleiss' k = 0.31-0.53). Conclusions Substantial intra- and interobserver variability exists in STS response assessment, especially with intensity-based criteria. Segmentation-derived diameters improve reproducibility and agreement. Refinement and automation of intensity-based methods are needed to ensure reliable response evaluation in clinical trials.