Using machine learning algorithms to determine the most impactful variables on frozen section accuracy in gynecological oncology

Abstract

Objectives: Incorporation of critical staging procedures in gynecologic oncology operations rely on accurate intraoperative consultations of frozen section (F/S) tissue to help determine a diagnosis of malignancy. Traditionally, these consultations are reliable, with a discrepant rate of 3 - 8% from the final pathology report. Our aim is to use data from the intraoperative consultation process on predictive machine learning algorithms to improve the quality and reliability of intraoperative F/S consultations. Methods: All gynecologic oncology pathology cases that required intra-operative F/S at an academic urban center from 2014 to 2018 were identified by the pathology department. Twenty-four variables including patient demographics, the pathologist's specialty, the time taken to review the samples, as well as factors directly involved in the operation and frozen section pathology process were abstracted from the electronic medical record (Figure 1). Variables were split and analyzed based on whether they were continuous or categorical and utilized t-tests or chi-square respectively. We used generalized linear model (GLM), Support Vector Machine (SVM), and random forest (Boruta package) machine learning algorithms to assess what variable impacted accuracy most. Results: We identified 1386 correct F/S diagnosis cases, a random sample of 52 random samples were used. There was a total of 18 (1.3%) discrepant diagnoses. We analyzed the random sample and all discrepant cases for a total of 70 cases. The median patient age was 52 (range: 27-83). Specimens sent for F/S included: adnexa, 29; endometrial curettings, 17; lymph nodes, 4; uterus, 3; other, 17. Tissue volume median was 7.71 cm3 (range: 0.02 cm3 - 6642.00 cm3) and the median time to review each sample intraoperatively was 67.5 minutes (range: 17 to 108 minutes). Twenty-six pathologists in 14 different specialties conducted the frozen section review, while 7 pathologists, all with expertise in gynecology, performed the final pathology reviews. A random forest model performed the best with an accuracy of 66.67% 95% CI [0.41 0.87]. However, the random forest model had a low sensitivity of 0.167 with a higher specificity of 0.92. Overall, an algorithm did not reliably predict F/S diagnosis. The Boruta package, based on a random forest model identified tissue volume, surgeon, robotic operation, tissue diagnosis, F/S pathologist specialty and patient age as the most important factors in feature selection (Figure 1). Download : Download high-res image (103KB) Download : Download full-size image Conclusions: In our review of cases we found a low rate of discrepant cases, however, the machine learning algorithm identified a few important factors that correlated with a more reliable F/S diagnosis: smaller volumes of tissue submitted, expertise from the surgeon, robotic operations, the patient's age, tumor type, and utilization of F/S pathologists familiar with gynecologic malignancies. The data is preliminary and needs validation with a larger cohort of F/S cases to implement operational changes in the future.