Multi-Specialty Physician Performance in Predicting Radiographic Extranodal Extension in Nodal Metastases of Oropharyngeal Squamous Carcinoma

Abstract

The presence of extranodal extension (ENE) in oropharyngeal (OPC) cancer patients is an important prognostic factor and can be used to determine the optimal course of treatment; however, currently, the gold standard method for ENE assessment is performed pathologically, which can only be done in a post-hoc fashion after surgical treatment has already been performed. Anatomic imaging features are being explored as a possible method for the pre-therapeutic determination of ENE, but there is currently no objective standard for the assessment of ENE from radiographic images. In this study we recruited expert clinicians, including surgeons, radiation oncologists, and radiologists, across multiple institutions to individually evaluate the presence of ENE from CT scans in order to assess the performance of radiographic ENE evaluation in human experts across different specialties. Pre-therapy contrast-enhanced CT scans were collected from 25 OPC patients with lymph node metastasis that were pathologically evaluated for ENE after surgical resection. 5 scans were randomly chosen to be duplicated and left/right inverted, resulting in a total of 30 scans of which 21 had pathologically-confirmed ENE. To hide the inversion, all images were cropped to only show the oropharynx region. 34 expert head and neck cancer physicians, comprised of 12 surgeons, 11 radiation oncologists, and 11 radiologists, then separately evaluated the 30 CT scans using 3D Slicer for ENE presence or absence with their prediction confidence. For each physician, discriminative performance metrics were measured by calculating the accuracy, sensitivity, specificity, area under the receiver-operating characteristic curve (AUC), and Brier score, a measure of the probabilistic prediction accuracy calculated from their confidence where a lower Brier score is better. Statistical tests were performed using the Mann Whitney U test. The median (interquartile) study results are shown in Table 1. There was no statistically significant difference among groups for accuracy or AUC, but significant differences among groups for Brier score, sensitivity, and specificity. In this study we provide evidence that expert physicians, regardless of specialty, show poor performance in assessing the presence of ENE from CT scans in OPC patients. These results agree with conclusions from previous literature, and suggest the need for further research in the automated analysis of radiographic ENE.