Integrity situational judgement test for medical school selection: judging 'what to do' versus 'what not to do'.

Wendy E De Leng,Karen M Stegers-Jager,Marise Ph Born,Axel P N Themmen

doi:10.1111/medu.13498

Wendy E De Leng, Karen M Stegers-Jager + Show 2 more

Open Access

https://doi.org/10.1111/medu.13498

Copy DOI

Journal: Medical education	Publication Date: Jan 19, 2018
Citations: 13	License type: cc-by

Affiliation: Erasmus University Rotterdam

Abstract

ContextDespite their widespread use in medical school selection, there remains a lack of clarity on exactly what situational judgement tests (SJTs) measure.ObjectivesWe aimed to develop an SJT that measures integrity by combining critical incident interviews (inductive approach) with an innovative deductive approach. The deductive approach guided the development of the SJT according to two established theoretical models, of which one was positively related to integrity (honesty–humility [HH]) and one was negatively related to integrity (cognitive distortions [CD]). The Integrity SJT covered desirable (HH‐based) and undesirable (CD‐based) response options. We examined the convergent and discriminant validity of the Integrity SJT and compared the validity of the HH‐based and CD‐based subscores.MethodsThe Integrity SJT was administered to 402 prospective applicants at a Dutch medical school. The Integrity SJT consisted of 57 scenarios, each followed by four response options, of which two represented HH facets and two represented CD categories. Three SJT scores were computed, including a total, an HH‐based and a CD‐based score. The validity of these scores was examined according to their relationships with external integrity‐related measures (convergent validity) and self‐efficacy (discriminant validity).ResultsThe three SJT scores correlated significantly with all integrity‐related measures and not with self‐efficacy, indicating convergent and discriminant validity. In addition, the CD‐based SJT score correlated significantly more strongly than the HH‐based SJT score with two of the four integrity‐related measures.ConclusionsAn SJT that assesses the ability to correctly recognise CD‐based response options as inappropriate (i.e. what one should not do) seems to have stronger convergent validity than an SJT that assesses the ability to correctly recognise HH‐based response options as appropriate (i.e. what one should do). This finding might be explained by the larger consensus on what is considered inappropriate than on what is considered appropriate in a challenging situation. It may be promising to focus an SJT on the ability to recognise what one should not do.

Highlights

In addition to the cognitive instruments used in selection for medical school, there is an increasing need for tools that assess non-cognitive attributes
Medical Education published by Association for the Study of Medical Education and John Wiley & Sons Ltd; MEDICAL EDUCATION 2018 52: 427–437
The combination of a traditional inductive and an innovative deductive development approach resulted in an Integrity situational judgement tests (SJTs) which had convergent and discriminant validity

Summary

Introduction

In addition to the cognitive instruments used in selection for medical school, there is an increasing need for tools that assess non-cognitive attributes (e.g. integrity) This growing need has led to the introduction of new medical school selection instruments such as multiple mini-interviews, selection centres, personality and emotional intelligence assessments and situational judgement tests (SJTs).[1,2] The SJT presents applicants with challenging situations they may encounter during medical school. Previous studies on SJTs in medical school selection demonstrated predictive validity and incremental validity over cognitive ability tests.[4,5,6,7] SJTs result in less adverse impact than traditional cognitive tests with respect to applicants with backgrounds of low socio-economic status.[8]

Objectives

Methods

Results

Discussion

Conclusion