Investigation of answer changes on the USMLE\xae Step 2 Clinical Knowledge examination

Wenli Ouyang,Miguel A Paniagua,Polina Harik,Brian E Clauser

doi:10.1186/s12909-019-1816-3

Wenli Ouyang, Miguel A Paniagua + Show 2 more

Open Access

https://doi.org/10.1186/s12909-019-1816-3

Copy DOI

Journal: BMC medical education	Publication Date: Oct 23, 2019
Citations: 8	License type: open-access

Affiliation: National Board of Medical Examiners

Abstract

BackgroundExaminees often believe that changing answers will lower their scores; however, empirical studies suggest that allowing examinees to change responses may improve their performance in classroom assessments. To date, no studies have been able to examine answer changes during large scale professional credentialing or licensing examinations.MethodsIn this study, we expand the research on answer changes by analyzing responses from 27,830 examinees who completed the Step 2 Clinical Knowledge (CK) examination between August of 2015 and August of 2016.ResultsThe results showed that although 68% of examinees changed at least one item, the overall average number of changes was small. Among the examinees who changed answers, approximately 45% increased their scores and approximately 28% decreased their scores. On average, examinees spent shortest time on the item changes from wrong to right and they were more likely to change their scores from wrong to right than right to wrong.ConclusionsConsistent with previous studies, these findings support the beneficial effects of answer changes in high-stakes medical examinations and suggest that examinees who are overly cautious about changing answers may put themselves at a disadvantage.

Highlights

Examinees often believe that changing answers will lower their scores; empirical studies suggest that allowing examinees to change responses may improve their performance in classroom assessments
There are, no doubt, many areas where conventional wisdom stands in sharp contrast to a substantial body of empirical evidence
Materials seem to support the view that considerable caution should be exercised in changing answers, for example, How to Prepare for the GRE: Graduate Record Examination [5] and First Aid for the USMLE Step 1 [3] both include a recommendations of this sort

Summary

Introduction

Examinees often believe that changing answers will lower their scores; empirical studies suggest that allowing examinees to change responses may improve their performance in classroom assessments. Changes typically are from wrong to right answers more often than from right to wrong Examinees increased their scores 53% of the time by changing their answers according to one study [19] and 51% of the time according to Ouyang et al BMC Medical Education (2019) 19:389 another [18]. These results seem to be typical of the available literature on answer changes. There is only one published study which argues that answer changes could be detrimental [28] These authors have subsequently published a retraction of the initially reported results owing to an error with the data analysis. Bridgeman [4] subsequently reanalyzed the data and reported that 76% of examinees improved their scores by changing answers

Objectives

Methods

Results

Conclusion