Substantial interreader agreement for biopsy with reduction in biopsy rate: A multireader diagnostic performance study of ACR TI-RADS

Abstract

PurposeTo assess diagnostic performance of ACR TI-RADS in thyroid cancer detection and evaluate interobserver agreement among radiologists with lower interpreting experience. MethodsFour radiologists retrospectively assessed 295 biopsied thyroid nodules from ultrasound studies performed between 2009 and 2019, blinded to histopathology. Diagnostic performance for cancer detection was determined individually, and interobserver agreement among four readers was evaluated with Fleiss kappa coefficient (ⱪ). Results245 (83.1%) benign and 50 (16.9%) malignant nodules were evaluated. Diagnostic performance based on final TR level was consistent and without significant difference among four readers, with excellent sensitivity (≥98.0%) and negative predictive value (NPV) [≥94.4%] for TR levels 3 to 5. Diagnostic performance based on recommendation to biopsy has moderate sensitivity (≥62%) and high NPV (≥84.7%). Retrospective scoring with established ACR TI-RADS criteria would have substantially reduced the number of biopsies performed, with 63.2% of malignancy not biopsied meeting criteria for sonographic surveillance. Interobserver agreement on TI-RADS scoring for final TR level was fair (ⱪ = 0.39, 95% CI 0.32, 0.47), with substantial agreement for recommendation to biopsy (ⱪ = 0.64, 95% CI 0.58, 0.70). ConclusionsSubstantial reduction in biopsy rate (up to 48%) would have been achieved using the ACR TI-RADS criteria, with 63% of malignancy not biopsied meeting criteria for sonographic surveillance. Interobserver agreement was fair for TI-RADS level scoring and substantial for recommendation to biopsy.