Measuring resident/faculty contour concordance as a potential tool for quantitative assessment of residents' performance in target volume delineation: a feasibility study

Abstract

<h3>Background</h3> Accurate target delineation/contouring is essential for radiation treatment planning and the clinical efficacy of radiation therapy. Clinical trial data has demonstrated the importance of quality contouring and treatment planning for optimal survival outcomes. As a result, improving the quality of target delineation is an important goal in the education of radiation oncology residents. However, there is limited quantitative data on the quality of residents' contours. Therefore, it would be beneficial to track performance and improvement in resident target delineation during residency. The purpose of this study was to determine if it was feasible to track the concordance of radiation oncology residents' contours with faculty physicians' contours. <h3>Methods</h3> Residents were asked to contour target volumes (GTV, CTV, ITV, PTV, etc.) based on patient history, physical exam, clinical stage and fused diagnostic imaging and were allowed to use any available outside resources, including textbooks, review articles, consensus guidelines, and online atlases. Resident contours were saved as separate structures. Finalized, faculty physician-approved contours were also saved. Saved contour structures and data from October 2019 through June 2020 were reviewed for feasibility. <h3>Results</h3> In total, 209 structures had both resident and faculty versions saved within Eclipse and were available for analysis. The Boolean Operations Tool in Eclipse (Varian Medical Systems, Palo Alto, CA, USA) was used to create an intersection volume of the resident/faculty contours. Separately, the Boolean Operations Tool was used to create a union volume of the resident/faculty contours. Based on these two volumes, the Jaccard Concordance Index (JCI) was calculated by dividing the intersection volume (in cubic centimeters [cc]) by the union volume (cc). The JCI could be successfully calculated for 203 (97.1%) of the 209 structures. For 6 structures (2.9%), both the intersection and union volumes could not be calculated because the volumes were too small for Eclipse to determine. All 6 of these structures were small brain metastases. <h3>Discussion</h3> Tracking and comparing the concordance of resident contours and faculty physician contours is feasible using available tools in Eclipse. Additional data collection and assessment is necessary before this technique can be more widely utilized.