Can knowledge tests and situational judgement tests predict selection centre performance?

Abstract

Written tests are an integral part of selection into general practice specialty training in the UK. Evidence supporting their validity and reliability as shortlisting tools has prompted their introduction into the selection processes of other medical specialties. This study explores whether candidate performance on two written tests predicts performance on subsequent workplace-based simulation exercises. A prospective analysis of candidate performance (n = 135) during the general practice selection process was undertaken. Candidates were shortlisted using their scores on two written tests, a clinical problem-solving test (CPST) and a situational judgement test (SJT). Successful candidates then undertook workplace-based simulation exercises at a selection centre (SC). Scores on the CPST and SJT were correlated with SC scores. Regression analysis was undertaken to explore the predictive validity of the CPST and SJT for SC performance. The data show that the CPST and SJT are predictive of performance in workplace-based simulations (r = 0.598 for the CPST, r = 0.717 for the SJT). The SJT is a better predictor of SC performance than the CPST (R(2) = 0.51 versus R(2) = 0.35). However, the two tests together provide the greatest degree of predictive ability, accounting for 57% of the variance seen in mean scores across SC exercises. The CPST and SJT play valuable roles in shortlisting and are predictive of performance in workplace-based SC exercises. This study provides evidence for their continued use in selection for general practice training and their expansion to other medical specialties.