Objective Residency Applicant Assessment Using a Linear Rank Model

Abstract

Residency applicant assessment is imperfect, with little objectivity built into the process, which, unfortunately, impacts recruitment diversity. Linear rank modeling (LRM) is an algorithm that standardizes applicant assessment to model expert judgment. Over the last 5 years, we have used LRM to assist with screening and ranking integrated plastic surgery (PRS) residency applicants. This study's primary objective was to determine if LRM scores are predictive of match success and, secondarily, to compare LRM scores between gender and self-identified race categories. Data was collected on applicant demographics, traditional application metrics, global intuition rank, and match success. LRM scores were calculated for screened and interviewed applicants, and scores were compared by demographic groups. Univariate logistic regression was used to evaluate the association of LRM scores and traditional application metrics with match success. University of Wisconsin, Division of Plastic and Reconstructive Surgery. Academic institution. Six hundred seventeen candidates who applied to a single institution over 4 application cycles (2019-2022). Using area under the curve modeling, LRM score was the most predictive indicator for match success. With every one-point increase in LRM score, there was an 11% and 8.3% increase in the likelihood of screened and interviewed applicant match success (p < 0.001). An algorithm was developed to estimate the probability of match success based on LRM score. No significant differences in LRM scores were appreciated for interviewed applicant gender or self-identified race groups. LRM score is the most predictive indicator of match success for PRS applicants and can be used to estimate an applicant's probability of successfully matching into an integrated PRS residency. Furthermore, it provides a holistic evaluation of the applicant that can streamline the application process and improve recruitment diversity. In the future, this model could be applied to assist in the match process for other specialties.