Analysis of Resident/Faculty Contour Concordance: A Potential Tool for Quantitative Assessment of Residents’ Performance in Target Volume Delineation

Abstract

<h3>Purpose/Objective(s)</h3> Accurate target delineation/contouring is essential for radiation treatment planning and the clinical efficacy of radiation therapy. As a result, improving the quality of target delineation is an important goal in the education of radiation oncology residents. However, there is limited quantitative data on the quality of residents' contours. Therefore, it would be beneficial to track performance and improvement in resident target delineation during residency. The purpose of this study was to track the concordance of radiation oncology residents' contours with faculty physicians' contours over the course of one year to assess for patterns. <h3>Materials/Methods</h3> Residents were asked to contour target volumes (GTV, CTV, ITV, PTV, etc.); these contours were saved as separate structures and compared to the finalized, faculty physician-approved contours. Concordance between resident and faculty physician contours was determined by calculating the Jaccard Concordance Index (JCI). The JCI was calculated by dividing the intersection volume of the resident/faculty contours by the union volume of the resident/faculty contours. A score of 1 indicates complete agreement while a score of 0 indicates no agreement. Resident contours were stratified by PGY level (i.e., PGY-2, PGY-3, PGY-4). Data was collected from the 2020-2021 academic year. Two-tailed, unpaired t testing and one-way ANOVA testing was performed to compare mean values in JCI. A P-value of < 0.05 was considered statistically significant. <h3>Results</h3> In total, 904 structures were available for analysis. The mean JCI was 0.766, 0.749, and 0.618 for the PGY-4 level resident, PGY-3 level resident, and PGY-2 level resident, respectively. There was statistically significant difference in mean JCI between PGY-levels (P < 0.001). On subset analysis of disease type, there was statistically significant difference in mean JCI between PGY-levels for brain metastases, primary gastrointestinal cancers, primary genitourinary cancers, primary gynecologic cancers, primary head/neck cancers, primary thoracic cancers, and sarcomas (P = 0.020, P < 0.001, P = 0.012, P < 0.001, P = 0.018, P < 0.001, respectively). There was no significant difference in mean JCI between PGY-levels for primary brain tumors, primary breast cancers, and lymphomas; however, there were fewer than 30 structures per resident for these disease types available for analysis. <h3>Conclusion</h3> Tracking and comparing the concordance of resident contours and faculty physician contours is an intriguing method of assessing resident performance in target volume delineation and could potentially serve as a quantitative metric in radiation oncology resident evaluation, which is lacking currently. However, additional study is necessary before this technique can be incorporated into residency assessments.