Clinical Acceptability of Automatically Generated Elective Lymph Node Volumes for Head and Neck Cancer Patients

Abstract

Manual contouring of head and neck lymph node levels is a time-intensive process prone to provider-specific variation. The purpose of this work is to generate a clinical segmentation tool while minimizing the amount of manual effort required by physicians to develop training datasets and review contours. Here we investigate an approach to curate, develop, and clinically validate an auto-contouring model for standard cervical lymph node volumes in the head and neck using a publicly available deep learning architecture. This model updates our previously validated tool to reflect modern practices in lymph node segmentation. With the assistance of a resident physician, five radiation oncologists manually contoured individual lymph node levels on CT scans for three separate patients treated definitively with radiation or chemoradiation for oropharynx cancer, resulting in 15 unique ground truth cases. These cases were then used to train an nnUnet deep-learning model to generate automated contours for 32 additional cases. These 32 cases were reviewed, manually edited, and used to create the final model. Finally, the model was used to generate contours on the original 15 CT scans (testing cohort), and providers compared these automated contours with the ground-truth (manual) contours. Two blinded studies were performed. In a double-blinded fashion, providers were first asked to select which set of contours they would prefer to use in clinical practice as a starting point for actual cases. Second, they scored each contour on a Likert scale (1-5) to indicate clinical acceptability, ranging from completely unusable to usable without modification. Across all lymph node levels (IA, IB, II, III, IV, V, RP), average Dice Similarity Coefficient ranged from 0.77 to 0.89 for AI vs manual contours in the testing cohort. These AI and manual lymph node contours were reviewed by 5 physicians each, resulting in 525 preference scores. Across all lymph nodes, the AI contour was superior to or equally preferred to the manual contours at rates ranging from 75% to 91% in the first blinded study. In the second blinded study, physician preference for the manual vs AI contour was statistically different for only the RP contours (p < 0.01). Thus, there was not a significant difference in clinical acceptability for nodal levels I-V for manual versus AI contours. Across all physician-generated contours, 82% were rated as usable with stylistic to no edits, and across all AI-generated contours, 92% were rated as usable with stylistic to no edits. An approach to generate clinically acceptable automated contours for cervical lymph node levels in the head and neck was demonstrated. Furthermore, for nodal levels I-V, there was no significant difference in clinical acceptability in manual vs AI contours. Because we were able to generate and validate a model for each lymph node level individually, the output is applicable to a complete range of disease in which cervical lymph nodes are treated.