Widening access in selection using situational judgement tests: evidence from the UKCAT.

Abstract

Widening access promotes student diversity and the appropriate representation of all demographic groups. This study aims to examine diversity-related benefits of the use of situational judgement tests (SJTs) in the UK Clinical Aptitude Test (UKCAT) in terms of three demographic variables: (i) socio-economic status (SES); (ii) ethnicity, and (iii) gender. Outcomes in medical and dental school applicant cohorts for the years 2012 (n=15581) and 2013 (n=15454) were studied. Applicants' scores on cognitive tests and an SJT were linked to SES (parents' occupational status), ethnicity (White versus Black and other minority ethnic candidates), and gender. Firstly, the effect size for SES was lower for the SJT (d=0.13-0.20 in favour of the higher SES group) than it was for the cognitive tests (d=0.38-0.35). Secondly, effect sizes for ethnicity of the SJT and cognitive tests were similar (d=~0.50 in favour of White candidates). Thirdly, males outperformed females on cognitive tests, whereas the reverse was true for SJTs. When equal weight was given to the SJT and the cognitive tests in the admission decision and when the selection ratio was stringent, simulated scenarios showed that using an SJT in addition to cognitive tests might enable admissions boards to select more students from lower SES backgrounds and more female students. The SJT has the potential to appropriately complement cognitive tests in the selection of doctors and dentists. It may also put candidates of lower SES backgrounds at less of a disadvantage and may potentially diversify the student intake. However, use of the SJT applied in this study did not diminish the role of ethnicity. Future research should examine these findings with other SJTs and other tests internationally and scrutinise the causes underlying the role of ethnicity.