Evaluating the validity of an integrity-based situational judgement test for medical school admissions.

Adrian Husbands,Jon Dowell,Mark J Rodgerson,Fiona Patterson

doi:10.1186/s12909-015-0424-0

Adrian Husbands, Jon Dowell + Show 2 more

Open Access

https://doi.org/10.1186/s12909-015-0424-0

Copy DOI

Abstract

BackgroundWhile the construct of integrity has emerged as a front-runner amongst the desirable attributes to select for in medical school admissions, it is less clear how best to assess this characteristic. A potential solution lies in the use of Situational Judgement Tests (SJTs) which have gained popularity due to robust psychometric evidence and potential for large-scale administration. This study aims to explore the psychometric properties of an SJT designed to measure the construct of integrity.MethodsTen SJT scenarios, each with five response stems were developed from critical incident interviews with academic and clinical staff. 200 of 520 (38.5 %) Multiple Mini Interview candidates at Dundee Medical School participated in the study during the 2012–2013 admissions cycle. Participants were asked to rate the appropriateness of each SJT response on a 4-point likert scale as well as complete the HEXACO personality inventory and a face validity questionnaire. Pearson’s correlations and descriptive statistics were used to examine the associations between SJT score, HEXACO personality traits, pre-admissions measures namely academic and United Kingdom Clinical Aptitude Test (UKCAT) scores, as well as acceptability.ResultsCronbach’s alpha reliability for the SJT was .64. Statistically significant correlations ranging from .16 to .36 (.22 to .53 disattenuated) were observed between SJT score and the honesty-humility (integrity), conscientiousness, extraversion and agreeableness dimensions of the HEXACO inventory. A significant correlation of .32 (.47 disattenuated) was observed between SJT and MMI scores and no significant relationship with the UKCAT. Participant reactions to the SJTs were generally positive.ConclusionsInitial findings are encouraging regarding the psychometric robustness of an integrity-based SJT for medical student selection, with significant associations found between the SJTs, integrity, other desirable personality traits and the MMI. The SJTs showed little or no redundancy with cognitive ability. Results suggest that carefully-designed SJTs may augment more costly MMIs.

Highlights

While the construct of integrity has emerged as a front-runner amongst the desirable attributes to select for in medical school admissions, it is less clear how best to assess this characteristic
Scores from two participants who failed to complete a large proportion of Situational judgement test (SJT) and HEXACO items were discarded
Final analysis pertained to 198 participants, all of whom completed the Multiple Mini Interview (MMI), SJT, HEXACO inventory and acceptability questionnaire

Summary

Introduction

While the construct of integrity has emerged as a front-runner amongst the desirable attributes to select for in medical school admissions, it is less clear how best to assess this characteristic. Koenig et al [7] attempt to address this lack of consensus in a large-scale multicentre study which involved 98 North American medical schools to identify core personal characteristics and included literature reviews, role analyses and questionnaires. These authors identified ‘ethical responsibility to self and others’, followed by social skills as the most important among 9 personal competencies for successful medical school entrants. Another large-scale study by Husbands et al BMC Medical Education (2015) 15:144

Objectives

Methods

Results

Discussion

Conclusion