Assessment of interobserver concordance in radiomic tools for lung nodule classification, with a focus on BRODERS and SILA

Abstract

While CT lung cancer screening reduces lung cancer-specific mortality, there are remaining challenges. Radiomic tools promiss to address these challenges, however, they are subject to interobserver variability if semi-automated segmentation techniques are used. Herein we report interobserver variability for two validated radiomic tools, BRODERS (Benign versus aggRessive nODule Evaluation using Radiomic Stratification) and CANARY (Computer-Aided Nodule Assessment and Risk Yield). We retrospectively analyzed the CT images of 95 malignant lung nodules of the adenocarcinoma spectrum using BRODERS and CANARY. Cases were identified at Mayo Clinic (n = 45) and Vanderbilt University Medical Center and Nashville/Veteran Administration Tennessee Valley Health Care System (n = 50). Three observers with different training levels (medical student, internal medicine resident and thoracic radiology fellow) each performed lung nodule segmentation. All methods were carried out in accordance with relevant guidelines and regulations. Interclass correlation coefficients (ICC) of 0.77, 0.98 and 0.97 for the average nodule volume, BRODERS cancer probability and Score Indicative of Lesion Aggression (SILA) which summarizes the distribution of the CANARY exemplars indicated good to excellent reliability, respectively. The dice similarity coefficient was 0.79 and 0.81 for the data sets from the two institutions. BRODERS and CANARY are robust radiomics tools with excellent interobserver variability. These tools are simple and reliable regardless the observer/operator’s level of training.