422 Utilizing Situational Judgment Test in Neurosurgery Residency Online Interviews

Abstract

INTRODUCTION: Neurosurgery residency selection is heavily reliant on pre-residency factors and unstructured interviews. These metrics, however, have not been shown to reliably predict residency performance, and have potential to introduce bias and discrimination. Outside of medicine, the science of selection is much better developed, and methods such as structured interviews, situational judgment tests (SJT), and personality assessments are routinely used with robust evidence for efficacy and reliability at predicting job performance. METHODS: We designed a set of 4 SJT questions mimicking common situations a neurosurgery resident would encounter during residency. Each question had individualized grading rubrics scored on a scale of 1 to 4. For the 2021 residency application cycle, our program incorporated these SJT questions as part of the online interview process. Four volunteer faculty interview pairs delivered one question each for every candidate, and recorded the scores. Other traditional evaluation metrics from the candidate’s application and interview were also collected post interview. RESULTS: All 44 interview candidates applying to our program during the 2021 cycle were assessed. Due to time constraints, the number of questions each candidate responded to varied from 2 to 3. The average score was 3.6 (SD 0.44, range 2-4). The SJT score was not associated with USMLE Step 1 score (p = 0.31), number of publications (p = 0.42), or letter of recommendation score (p = 0.21). The SJT scores had a moderate positive correlation with personality on interview (r = 0.42, p < 0.01), and overall interview score (r = 0.30, p = 0.04). CONCLUSIONS: It is feasible to incorporate SJT questions into the neurosurgery resident online interview process. They may provide objective metrics independent of academic and research abilities. Further investigations with long term follow up is needed to see if SJT can be validated and correlated with residency performance.