Abstract
PurposeWe have observed that students' performance in our pre-clerkship curriculum does not align well with their United States Medical Licensing Examination (USMLE) STEP1 scores. Students at-risk of failing or underperforming on STEP1 have often excelled on our institutional assessments. We sought to test the validity and reliability of our course assessments in predicting STEP1 scores, and in the process, generate and validate a more accurate prediction model for STEP1 performance.MethodsStudent pre-matriculation and course assessment data of the Class of 2020 (n = 76) is used to generate a stepwise STEP1 prediction model, which is tested with the students of the Class of 2021 (n = 71). Predictions are developed at the time of matriculation and subsequently at the end of each course in the programing language R. For the Class of 2021, the predicted STEP1 score is correlated with their actual STEP1 scores, and data agreement is tested with means-difference plots. A similar model is generated and tested for the Class of 2022.ResultsSTEP1 predictions based on pre-matriculation data are unreliable and fail to identify at-risk students (R2 = 0.02). STEP1 predictions for most year one courses (anatomy, biochemistry, physiology) correlate poorly with students' actual STEP1 scores (R2 = 0.30). STEP1 predictions improve for year two courses (microbiology, pathology, and pharmacology). But integrated courses with customized NBMEs provide more reliable predictions (R2 = 0.66). Predictions based on these integrated courses are reproducible for the Class of 2022.ConclusionMCAT and undergraduate GPA are poor predictors of student's STEP1 scores. Partially integrated courses with biweekly assessments do not promote problem-solving skills and leave students' at-risk of failing STEP1. Only courses with integrated and comprehensive assessments are reliable indicators of students' STEP1 preparation.
Highlights
The United States Medical Licensing Examination (USMLE) STEP1 is the first of the four licensure examinations a medical graduate must pass to become a practicing allopathic medical doctor in the United States
The mean-difference plot highlights that MCAT/BCPM overpredict STEP1 scores for students scoring below the school mean [220] and underpredicts for students scoring above
Similar to MCAT/BCPM GPA, STEP1 predictions are generated at the end of each MS1 course
Summary
The United States Medical Licensing Examination (USMLE) STEP1 is the first of the four licensure examinations a medical graduate must pass to become a practicing allopathic medical doctor in the United States. This highly integrated exam assesses clinical problem-solving and the student’s ability to apply basic science concepts to principles and mechanisms associated with the organ systems’ structure, function, disease, and therapeutics. USMLE has recently changed the STEP1 scorereport format from a three-digit score to a pass/fail report, it remains a crucial milestone for the academic progression of a medical student. Considering the preceding, success on this standardized examination is of paramount importance to medical students and the medical institution, with significant individual and institutional effort spent on preparing for this examination
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
