NBME Subject Examination Research Articles

The Impact of Shortened Clinical Clerkships on Medical Student Performance and Clerkship Assessment.

IntroMedical schools sometimes need to adjust the length of third-year clinical clerkships. The literature surrounding the effects of shortened clerkships on student experience and performance is mixed.MethodsOur medical school shortened the third year by an average of 20% per clerkship to accommodate a curricular re-design in 2018-2019. We examined test scores and measures of clinical performance as well as student experience in order to understand the impact of this change.ResultsTwo hundred and eight students were included in the analysis, 104 in each cohort. No statistically significant differences were noted between cohorts on NBME subject examination results. There were no significant differences on Step 2 CK scores between the traditional curriculum cohort (M = 249.4, SD = 13.7) and shortened curriculum cohort (M = 248.7, SD = 15.8). Student performance on OSCE cases was similar. Similar percentages of students rated each clerkship either “good” or “excellent” in the traditional (77%) and shortened (78%) curriculum.ConclusionThere was no significant impact on student test scores after shortening the curriculum. Measures of student satisfaction and experience also remained stable, likely related to emphasis on retaining patient care experiences and streamlining of didactics. Curricular shortening during the third year of medical school was feasible and safe from the student perspective in our experience.

Impact of general surgery rotation exposure on surgery clerkship performance

BackgroundMedical students often have variable exposures to general surgery (GS) and subspecialty surgery (SS) during their surgical clerkship. We will evaluate the relationship between clinical exposure to GS and performance on the NBME Subject Examination in Surgery. MethodsStudent data was collected retrospectively from 2015 to 2018 at a single academic institution. Students were categorized based on their clinical clerkship exposure to GS. A linear model was used to estimate the mean difference in NBME performance between GS and strictly SS exposed students while controlling for prior standardized examination scores and completion of an internal medicine clerkship prior to surgery. Results365 (67%) of 547 students were exposed to a GS rotation prior to their NBME exam. Performance on the NBME exam was comparable between GS versus SS students (μdiff = 0.37, 95% CI: −0.73 to 1.48; p = .51). ConclusionsExposure to a GS rotation is not advantageous on the NBME surgery examination. Students who completed the medicine clerkship prior to surgery demonstrated superior performance on the NBME surgery examination.

Impact of a Pediatric Clerkship Academic Half Day on Third-Year Medical Students

This article was migrated. The article was not marked as recommended. The use of an academic half day (AHD) compared to a traditional noon conference (NC) has been studied in graduate medical education. Despite undergraduate medical education (UGME) use of AHDs there is no data to advocate for this teaching strategy. We explored the impact of an AHD on UGME during the pediatric clerkship rotation. During the 2017-18 academic year, 118 third-year medical students rotated on the pediatric clerkship rotation. Fifty-eight students were placed in the NC format and 60 in the AHD. NBME Subject Examination performance is similar for both groups (0.04 vs 0.23, F(1, 116)=1.31, p > 0.26). Attendance was significantly higher for AHD compared with NC (90.9% vs 73.3%, F(1, 114)=63.27, p<0.05). Additionally, the mean didactic rating is 3.81 for the AHD group, significantly higher than 3.60 for the NC group (F(1, 113)=6.56, p <0.05). The impact of an AHD in UGME appears promising and consistent with experiences of an AHD in graduate medical education on medical knowledge, attendance, and didactic satisfaction.

Impact of Review Sessions on NBME Subject Examination Performance (P1.062)

Impact of Review Sessions on NBME Subject Examination Performance (P1.062)

Does the NBME Surgery Shelf exam constitute a “double jeopardy” of USMLE Step 1 performance?

BackgroundScores from the NBME Subject Examination in Surgery (Surgery Shelf) positively correlate with United States Medical Licensing Examination Step 1 (Step 1). Based on this relationship, the authors evaluated the predictive value of Step 1 on the Surgery Shelf. MethodsSurgery Shelf standard scores were substituted for Step 1 standard scores for 395 students in 2012–2014 at one medical school. Linear regression was used to determine how well Step 1 scores predicted Surgery Shelf scores. Percent match between original (with Shelf) and modified (with Step 1) clerkship grades were computed. ResultsStep 1 scores significantly predicted Surgery Shelf scores, R2 = 0.42, P < 0.001. For every point increase in Step 1, a Surgery Shelf score increased by 0.30 points. Seventy-seven percent of original grades matched the modified grades. ConclusionReplacing Surgery Shelf scores with Step 1 scores did not have an effect on the majority of final clerkship grades. This observation raises concern over use of Surgery Shelf scores as a measure of knowledge obtained during the Surgery clerkship.

US Medical Student Performance on the NBME Subject Examination in Internal Medicine: Do Clerkship Sequence and Clerkship Length Matter?

Prior to graduation, US medical students are required to complete clinical clerkship rotations, most commonly in the specialty areas of family medicine, internal medicine, obstetrics and gynecology (ob/gyn), pediatrics, psychiatry, and surgery. Within a school, the sequence in which students complete these clerkships varies. In addition, the length of these rotations varies, both within a school for different clerkships and between schools for the same clerkship. The present study investigated the effects of clerkship sequence and length on performance on the National Board of Medical Examiner's subject examination in internal medicine. The study sample included 16,091 students from 67 US Liaison Committee on Medical Education (LCME)-accredited medical schools who graduated in 2012 or 2013. Student-level measures included first-attempt internal medicine subject examination scores, first-attempt USMLE Step 1 scores, and five dichotomous variables capturing whether or not students completed rotations in family medicine, ob/gyn, pediatrics, psychiatry, and surgery prior to taking the internal medicine rotation. School-level measures included clerkship length and average Step 1 score. Multilevel models with students nested in schools were estimated with internal medicine subject examination scores as the dependent measure. Step 1 scores and the five dichotomous variables were treated as student-level predictors. Internal medicine clerkship length and average Step 1 score were used to predict school-to-school variation in average internal medicine subject examination scores. Completion of rotations in surgery, pediatrics and family medicine prior to taking the internal medicine examination significantly improved scores, with the largest benefit observed for surgery (coefficient = 1.58 points; p value < 0.01); completion of rotations in ob/gyn and psychiatry were unrelated to internal medicine subject examination performance. At the school level, longer internal medicine clerkships were associated with higher scores on the internal medicine examination (coefficient = 0.23 points/week; p value < 0.01). The order in which students complete clinical clerkships and the length of the internal medicine clerkship are associated with their internal medicine subject examination scores. Findings may have implications for curriculum re-design.

Toward a better understanding of gender-based performance in the obstetrics and gynecology clerkship: women outscore men on the NBME subject examination at one medical school.

To better understand why women outperform men in the obstetrics and gynecology (Ob/Gyn) clerkship. The authors conducted a retrospective cohort study of students who rotated in the Ob/Gyn clerkship from 2008 to 2011 and graduated by May 2012 from the Icahn School of Medicine at Mount Sinai in New York City. They compared female and male students' performance on preclerkship standardized tests (the Medical College Admission Test [MCAT] and United States Medical Licensing Examination [USMLE] Step 1) and on Ob/Gyn clerkship components (clinical skills achievement, oral and written case presentations, clinical evaluations, and National Board of Medical Examiners [NBME] subject examination). The study included 163 (53.4%) women and 142 (46.6%) men. Among students who took the MCAT, women scored lower than men with a mean (standard deviation) of 33.2 (3.2) versus 34.6 (3.3) (P=.001). Similarly, among all students, women scored lower than men on the USMLE Step 1: 227 (19.1) versus 232.5 (18.8) (P=.012). There were no significant gender differences on most clerkship components. However, women scored higher than men on the Ob/Gyn NBME subject examination: 78.0 (7.5) versus 74.8 (8.4) (P=.001). Consequently, female students achieved higher overall clerkship scores than men: 88.5 (5.6) versus 87.1 (5.1) (P=.024). Analysis of covariance confirmed that gender is associated with NBME subject examination score (P<.001). Women outscored men on the NBME subject examination in Ob/Gyn and thereby outperformed men in the Ob/Gyn clerkship.

Team-based learning on a third-year pediatric clerkship improves NBME subject exam blood disorder scores

PurposeAt our institution, speculation amongst medical students and faculty exists as to whether team-based learning (TBL) can improve scores on high-stakes examinations over traditional didactic lectures. Faculty with experience using TBL developed and piloted a required TBL blood disorders (BD) module for third-year medical students on their pediatric clerkship. The purpose of this study is to analyze the BD scores from the NBME subject exams before and after the introduction of the module.MethodsWe analyzed institutional and national item difficulties for BD items from the NBME pediatrics content area item analysis reports from 2011 to 2014 before (pre) and after (post) the pilot (October 2012). Total scores of 590 NBME subject examination students from examinee performance profiles were analyzed pre/post. t-Tests and Cohen's d effect sizes were used to analyze item difficulties for institutional versus national scores and pre/post comparisons of item difficulties and total scores.ResultsBD scores for our institution were 0.65 (±0.19) compared to 0.62 (±0.15) nationally (P=0.346; Cohen's d=0.15). The average of post-consecutive BD scores for our students was 0.70(±0.21) compared to examinees nationally [0.64 (±0.15)] with a significant mean difference (P=0.031; Cohen's d=0.43). The difference in our institutions pre [0.65 (±0.19)] and post [0.70 (±0.21)] BD scores trended higher (P=0.391; Cohen's d=0.27). Institutional BD scores were higher than national BD scores for both pre and post, with an effect size that tripled from pre to post scores. Institutional BD scores increased after the use of the TBL module, while overall exam scores remained steadily above national norms.ConclusionsInstitutional BD scores were higher than national BD scores for both pre and post, with an effect size that tripled from pre to post scores. Institutional BD scores increased after the use of the TBL module, while overall exam scores remained steadily above national norms.

NBME Subject Examination in Surgery Scores Correlate with Surgery Clerkship Clinical Experience

Most medical schools in the United States use the National Board of Medical Examiners Subject Examinations as a method of at least partial assessment of student performance, yet there is still uncertainty of how well these examination scores correlate with clinical proficiency. Thus, we investigated which factors in a surgery clerkship curriculum have a positive effect on academic achievement on the National Board of Medical Examiners Subject Examination in Surgery. A retrospective analysis of 83 third-year medical students at our institution with 4 unique clinical experiences on the general surgery clerkship for the 2007-2008 academic year was conducted. Records of the United States Medical Licensing Examination Step 1 scores, National Board of Medical Examiners Subject Examination in Surgery scores, and essay examination scores for the groups were compared using 1-way analysis of variance testing. Rush University Medical Center, Chicago IL, an academic institution and tertiary care center. Our data demonstrated National Board of Medical Examiners Subject Examination in Surgery scores from the group with the heavier clinical loads and least time for self-study were statistically higher than the group with lighter clinical services and higher rated self-study time (p = 0.036). However, there was no statistical difference of National Board of Medical Examiners Subject Examination in Surgery scores between the groups with equal clinical loads (p = 0.751). Students experiencing higher clinical volumes on surgical services, but less self-study time demonstrated statistically higher academic performance on objective evaluation, suggesting clinical experience may be of higher value than self-study and reading.

Subspecialty Exposure in a Psychiatry Clerkship Does Not Improve Student Performance in the Subject Examination

The authors compared the NBME subject examination scores and subspecialty profiles of 3rd-year medical students who were assigned to psychiatry subspecialties during their clerkship with those who were not. The authors collated and analyzed the shelf examination scores, the clinical grades, and the child psychiatry and emergency psychiatry shelf profiles of 361 junior medical students in two medical schools during their psychiatry clerkship. There were no significant differences on these parameters between the students assigned to subspecialty sites and those who were not. This study, like some similar previous studies in other clerkships, calls attention to the lack of measurable academic benefit of fragmenting and compartmentalizing the psychiatry clerkship experience for 3rd-year medical students.

Obstetrics and gynecology clerkship for males and females: similar curriculum, different outcomes?

ObjectiveTo determine if performance differences exist between male and female students on a 6-week obstetrics and gynecology (Ob/Gyn) clerkship and to evaluate potential variables that might underlie any observed variations.Study DesignFinal clerkship grades and component scores (clinical evaluations, objective structured clinical examination [OSCE], oral examination, and National Board of Medical Examiners [NBME] subject examination) from July 2007 to June 2010 were matched by student and analyzed by gender. Basic science grade point average (GPA) and initial United States Medical Licensing Exam (USMLE) Step 1 scores were used to establish students’ baseline medical knowledge. On a post-clerkship questionnaire, a subset of students reported the numbers of procedures they performed during the clerkship; students also completed online pre- and post-clerkship questionnaires reflecting their self-assessed confidence in women's health clinical skills.ResultsScores were analyzed for 136 women and 220 men. Final clerkship grades were significantly higher for females than for males (89.05 vs. 87.34, p=0.0004, η2=0.08). Specifically, females outscored males on the OSCE, oral, and NBME subject examination portions of the clerkship but not clinical evaluations. Males reported completing fewer breast examinations (p=0.001, η2=0.14). Pre-clerkship, males were significantly less confident than females in women's health clinical skills (p<0.01) but reached similar levels upon completion of the clerkship. No gender differences were detected for basic science GPA and USMLE Step 1 scores.ConclusionStudent gender is associated with final grades on an Ob/Gyn clerkship. Further research regarding these differences should be explored.

University of Alabama School of Medicine

University of Alabama School of Medicine

WHOSE RIME ASSESSMENT PREDICTS STUDENT EXAMINATION PERFORMANCE: ATTENDINGS, RESIDENTS, OR INTERNS?

Purpose The most well-studied feedback system used in internal medicine clerkships is the RIME system. RIME is a descriptive framework in which students and their teachers can gauge progress throughout a clerkship from R (reporter) to I (interpreter) to M (manager) to E (educator). The purpose of this project was (1) to see if a student9s RIME rating is associated with end-of-clerkship examination performance and (2) to determine whose independent RIME rating is most predictive of a student9s examination performance: attendings, residents, or interns. Methods Students have two 4-week rotations on our general medicine services, working with two to four attendings (usually hospitalists), two residents, and four interns. For academic year 2004-05 and the first third of 2005-06, each attending, resident, and intern independently evaluated the student on a written form at the end of the 4-week block. On the form, each evaluator was asked to circle the RIME stage and its accompanying descriptor that most matched their assessment of the student. For the purpose of analysis, R stage = 1, I = 2, M = 3, and E = 4. The attending, resident, and intern RIME score was the mean score of all of their attendings, residents, and interns, respectively. End-of-clerkship examinations included the NBME Medicine Subject examination and a Clinical Performance Examination (CPE), a 12-station examination using standardized patients. Pearson9s correlations assessed the bivariate relationships between the various measures. Regression analyses were performed with the examination scores as dependent variables, with independent variables of mean attending RIME score, mean resident score, and mean intern score. Results Data were received regarding 122 students. Mean attending RIME rating was 2.86 (SD 0.8), resident rating was 2.87 (0.9), and intern rating was 3.11 (0.8). In regression analyses, significant predictors of NBME subject examination score were resident RIME rating (p = .008) and intern RIME rating (p = .02). Significant predictor of CPE performance was resident RIME rating (F = 6.50, p = .01); intern and attending ratings approached statistical significance (p = .09 and .07, respectively). Conclusion Housestaff ratings of students using the RIME taxonomy are associated with better student examination performance. These findings suggest that housestaff (including interns) are accurate in their evaluation of student ability, perhaps better than attendings.

Is there a relationship between attending physicians' and residents' teaching skills and students' examination scores?

Faculty development programs and faculty incentive systems have heightened the need to validate a connection between the quality of teaching and students' learning. This study was designed to determine the association between attending physicians' and residents' teacher ratings and their students' examination scores. From a database of 362 students, 138 faculty, and 107 residents in internal medicine, student-faculty (n = 476) and student-resident (n = 474) pairs were identified. All students were in their third year, rotating on inpatient general medicine and cardiology services, July 1994 through June 1996, at a single institution. The outcome measure for students' knowledge was the NBME Subject Examination in internal medicine. To control for students' baseline knowledge, the predictors were scores on the USMLE Step 1 and a sequential examination (a clinically-based pre- and post-clerkship examination). Teaching abilities of faculty and residents were rated by a global item on the post-clerkship evaluation. Faculty's ratings used only scores from prior to the study period; residents' ratings included those scores students gave during the study period. Multivariate analyses showed faculty's teaching ratings were a small but significant predictor of the increase in students' knowledge. Residents' teaching ratings did not predict an increase in students' knowledge. Attending faculty's clinical teaching ability has a positive and significant effect on medical students' learning.

Six-year Documentation of the Association between Excellent Clinical Teaching and Improved Studentsʼ Examination Performances

Six-year Documentation of the Association between Excellent Clinical Teaching and Improved Studentsʼ Examination Performances

Using in-clerkship tests to identify students with insufficient knowledge and assessing the effect of counseling on final examination performance.

To determine whether in-clerkship tests identify students with insufficient knowledge and whether counseling affects final examination pass rates. The authors reviewed students' mean scores from two internal medicine clerkship tests at the Wright State University School of Medicine from February 1993 to July 1996. To determine the sensitivity and specificity of the tests for identifying students with insufficient knowledge, they compared students in the lowest quartile of clerkship test results with those who scored 290 or less on the end-of-clerkship National Board of Medical Examiners' (NBME) subject examination in medicine. The authors also compared the final examination pass rates of counseled and non-counseled students. Twenty-five students scored 290 or less on the NBME subject examination. Of those, 17 had low mean clerkship test scores (sensitivity of 68%). The specificity of a low mean clerkship test score was 81%. Counseling did not improve final examination pass rates. In-clerkship tests can identify students who are at risk of failing an end-of-clerkship examination. Because counseling may be insufficient to raise final examination pass rates, further study into the appropriate clerkship intervention for low-achievement students is needed.

Evaluation of students' learning in an interdisciplinary medicine--surgery clerkship.

To evaluate the impact of an interdisciplinary medicine-surgery clerkship (created to foster generalist education) on students' performances on National Board of Medical Examiners' (NBME) subject examinations. Test data for the 226 students who participated in the 16-week combined clerkship and for the 265 students who had completed the traditional clerkships (12 weeks of medicine, 12 weeks of surgery) were compiled and analyzed using t-tests for independent samples. Mean scores on the NBME subject examination in medicine increased significantly after the combined medicine-surgery clerkship (from 433 to 455, p < or = 0.5). Mean scores on the NBME subject examination in surgery were similar to those achieved in the traditional clerkship years. Since the medicine and surgery clerkships were combined into a single, interdisciplinary clerkship, students' scores have increased on the medicine NBME subject examination and have remained relatively unchanged on the surgery NBME subject examination, despite a substantial reduction in students' clinical experience in the combined clerkship from the traditional clerkships (16 vs 24 weeks).

Using NBME Subject Examinations as a Pretest to Identify Medical Students “At Risk” in a Behavioral Sciences Course

Pretesting students has been considered a sound educational practice but the reports of its utility in medical education are sparse. The purpose of the study was to demonstrate the usefulness of the NBME Subject Examination as a pretest to identify students “at risk” in a medical school behavioral sciences course. The results indicated that the pretest was helpful in documenting course effectiveness and that it and other measures could be used to provide an early prediction of students’ behavioral sciences course grades.