National Board Of Medical Examiners Examination Research Articles

Effectiveness of Near-Peer-Taught Case Reviews on Students' Confidence in National Board of Medical Examiners (NBME) Exams.

Introduction A key element to a first-year medical student's (MS1) education is guidance and practice in applying anatomy, pathophysiology, diagnosis, and treatment concepts to clinical vignettes. One potential solution to providing effective clinical reasoning training is the involvement of second-year medical students (MS2s) in small group sessions as teachers to provide more personalized instruction via case-based learning sessions. Near-peer teaching has been shown to benefit students' confidence in learning and improve test scores. Similarly, case-based learning is heavily associated with an improved understanding of complex topics. As such, this study assessed the efficacy of near-peer teaching with concomitant case-based style presentations on improving the comfort of MS1s with their understanding of the curriculum content and their comfort with applying their knowledge to clinical scenarios. Methods This randomized controlled crossover trial included several small-group study sessions, each consisting of five MS1s and led by an MS2 who reviewed selected clinical cases in a standardized slide decks. The control arm was provided the same slide decks but did not participate in the MS2-led sessions. During the first course, students were assigned to either the control or intervention group and then crossed over to the opposite group (control to intervention and vice versa) in every subsequent course. Comfort with the curriculum material was then assessed through pre and post surveys, with the post surveys administered after the MS1s took their final NBME examination for that course. Results The study was cut short due to COVID-19 precautions limiting in-person sessions. Nevertheless, the post survey demonstrated an increased understanding of pathological concepts for the intervention group compared to the control group. Conclusions Future work on near-peer group study sessions should enroll a larger sample size with measures to improve the response rate to better test whether near-peer-led case reviews had a significant effect on students' understanding of anatomical concepts and confidence during NBME examinations.

Medical Student Education During COVID-19: Electronic Education Does Not Decrease Examination Scores.

During the COVID-19 emergency, medical students were mandated to remain home, creating challenges to providing education remotely for third-year clinical rotations. This study aims to assess student reception and investigate objective outcomes to determine if online learning is a suitable alternative. Medical students enrolled in the third-year surgical clerkship during COVID-19 were asked to participate in a survey. 19 of 27 (70%) students participated. Content, faculty-led lectures, and resident-led problem-based learning (PBL) sessions were assessed using a ten-point Likert scale. National Board of Medical Examiners (NBME) examination, weekly quiz, and oral examination scores were compared to previous years. Student t-tests compared the groups. The median age was 25years. Comparing in-person to electronic sessions, there was no difference in effectiveness of faculty sessions preparing students for NBME (6.2 vs. 6.7, P = .46) or oral examinations (6.4 vs. 6.8, P = .58); there was also no difference in resident-led PBL sessions preparing students for NBME (7.2 vs. 7.2, P = .92) or oral examinations (7.4 vs. 7.6, P = .74). Comparing this group to students from the previous academic year, there was no difference in weekly quiz (85.3 vs. 87.8, P = .13), oral examination (89.8 vs. 93.9, P = .07), or NBME examination (75.3 vs. 77.4, P = .33) scores. Surgical medical didactic education can effectively be conducted remotely through faculty-led lectures and resident-led PBL sessions. Students did not have a preference between in-person and electronic content in preparation for examinations. As scores did not change, electronic education may be adequate for preparing students for examinations in times of crisis such as COVID-19.

Differences in medical student performance on examinations: exploring score variance between Kolb's Learning Style Inventory classifications

BackgroundKolb’s Cycle of Learning Theory acts as a foundational framework for the evolution of knowledge gained by learners throughout their education. Through Kolb’s cycle of experiential learning, one’s preferred way of learning could impact academic achievement in the pre-clinical years of medical education.MethodsThe medical student classes of 2020 and 2021 at a public university in the southeastern U.S. were invited to complete Kolb’s Learning Style Inventory (LSI). For those participants completing the LSI, examination results for their pre-clinical blocks were obtained and matched to the LSI results. Examination scores (locally-developed examinations and customized National Board of Medical Examiners (NBME) final examinations) were compared by LSI classification for each examination using Kruskal-Wallis Test.ResultsOut of 360 possible participants, 314 (87.2%) completed the Learning Style Inventory. Convergers and Assimilators made up 84.1% [Convergers (n = 177, 56.4%), Assimilators (n = 87, 27.7%)]. Accommodators (n = 25, 7.9%) and Divergers (n = 25, 7.9%) made up the remaining sample. Accomodators’ scores were significantly lower on locally-developed examinations in Principles of Medicine, Hematology, and Gastrointestinal System. The only NBME examination that demonstrated a significant difference across learning styles was from the Cardiovascular block.ConclusionsUpon reviewing Kolb’s LSI, our study indicated that performance on the customized NBME examinations minimized the variance in performance compared to locally-developed examinations. The lack of variance across learning styles for all but one NBME final examination appears to provide a more equitable assessment strategy.

Correlation of Stroop Effect in Medical Students to Their Performance in Pharmacology National Board of Medical Examiners (NBME) Examination

ObjectiveThe Stroop test is used to assess selective attention in cognitive psychology. Interference between word meaning and ink color in Incongruent Stroop test prolongs reaction time, which involves the brain searching for some collection of the stored information for prompt use. This is called the Stroop effect. Previous studies have reported the strong correlations between the Stroop effect and academic performance. Therefore, we aimed to study the correlation of the Stroop effect in medical students to their performance in pharmacology NBME examination.MethodsFor this study, 40 medical students who completed their pharmacology NBME examination were selected. The Stroop, congruent and incongruent, tests were performed using the EncephalApp Stroop Test (Android/iOS). In Congruent test (Stroop Off), the subjects were asked to select ink color of the hashtag (#) symbols printed in different ink colors (red, blue, and green). In Incongruent test (Stroop On), the subjects were asked to select the ink color of the words RED, BLUE, and GREEN printed in different ink colors, e.g., the word “RED” printed in blue ink had to be selected as blue. Each subject practiced each test for two attempts and 5 attempts of each test were then recorded.The EncephalApp Stroop Test automatically recorded the results of the tests as duration in seconds, which include Congruent Stroop (OffTime duration), Incongruent Stroop (OnTime duration), OffTime plus OnTime duration, OnTime minus OffTime duration. The results of OnTime minus OffTime duration of the Stroop test and pharmacology NBME examination scores were tabulated in Microsoft Excel and descriptive analysis was done. The correlation of the “OnTime minus OffTime duration” to pharmacology NBME examination scores was determined by a Pearson correlation coefficient and regression analysis.ResultsThe correlation coefficient (adjusted r2) of our study was −0.0255 and significance F was 0.862. The “OnTime minus OffTime duration” in medical students was not correlated with the pharmacology NBME examination score (r (38) = 0.0283, p = 0.862). The regression model predicted only 0.08% (R2 = 0.0008) of the pharmacology NBME examination score of medical students.ConclusionThe Stroop effect in medical students is insignificantly correlated with their performance in pharmacology NBME examination. However, correlations of the Stroop effect in medical students with their performance in other subjects and examinations need to be assessed in further studies.Support or Funding InformationThe study was supported by the Trinity Medical Sciences University School of Medicine, St. Vincent and the Grenadines.

Impact of the Introduction of the Clinical Science Mastery Series on Family Medicine NBME Subject Examination Performance.

Many medical schools assess student clinical knowledge using the National Board of Medical Examiners (NBME) subject examinations. The Family Medicine Clinical Science Mastery Series (CSMS) self-assessments, which are made up of former Family Medicine NBME examination questions, became available in September 2017. They provide students with realistic practice questions and immediate performance feedback. To further assess the utility of various study tools available to our students, this study investigated the impact of the CSMS self-assessments for family medicine on the NBME family medicine subject examination performance. Data analysis was conducted to compare student performance on the end-of-rotation NBME Family Medicine Clinical Subject Examination before and after the introduction of the CSMS family medicine self-assessments. The effect size was measured using a Cohen d analysis. We conducted an independent t-test analysis to determine the effect the NBME Family Medicine CSMS self-assessments had on end-of-rotation clinical subject examination scores. The analysis revealed statistically significant improvement in students' clinical subject examination scores after the release of the CSMS in September 2017 (n=90) compared to the students' scores prior to the availability of the CSMS (n=95). Student scores improved with the introduction of the NBME CSMS family medicine self-assessment. These results support recommending student use of the CSMS as a study tool for their end-of-clerkship NBME subject examinations.

Effect of Repeat Examination on Family Medicine NBME Subject Scores.

National Board of Medical Examiners (NBME) subject examinations are used by many schools to assess student clinical knowledge. Studies indicate that mean scores on NBME examinations improve as the clinical year progresses. Literature review revealed no studies investigating changes in individual student scores when end-of-block examinations were repeated at the end of the clinical year. This study investigated NBME family medicine subject examination score changes for students who opted to repeat the examination at the end of the academic year. In 2014, students on a 4-week family medicine block clerkship took the NBME subject examination at the end of their clerkship block and were offered the opportunity to repeat this examination at the end of that clinical year; 25 of 80 students voluntarily repeated the examination. Paired t-tests were used to compare performance outcomes between the exam means at the end of the clerkship blocks to the means on the exam administration at the end of the academic year. Results showed a statistically significant improvement in scores between the first and second examination administration. Examinations given immediately after the students' clinical experience yielded scaled scores ranging from 60 to 80 compared to the national mean of 71.9. Examinations given at the end of the clinical year yielded scaled scores ranging from 57 to 90 (t[24]=-2.66, P=0.0006). Repeating the NBME subject examination at the end of the year led to slightly increased scores, suggesting that time spent during clerkships influences examination performance.

Text4peds: a randomised text-messaging trial.

The use of cell phone text messaging in the medical field is of growing interest, but there are few data examining its value in medical education. The text4peds educational text-messaging program was created for third-year medical students preparing for the National Board of Medical Examiners (NBME) paediatric subject examination. A randomised, controlled trial was conducted with third-year medical students on their paediatric clerkship. Students in the intervention group received daily messages consisting of multiple-choice questions with links to online material. The control group received no text messages. The impact of the intervention was assessed by examining the participants' NBME examination scores and by participant surveys. A total of 162 students participated in the study. There was no statistical difference between the two groups, with the texting group having a mean score of 77.7 and with the non-texting group having a mean score of 77.8 on the NBME. Subgroup analysis examining the effect of anticipated specialty and online material interaction also failed to find any statistically significant difference. Students reported satisfaction with the program, with 84% rating the text messages as helpful. The students rated their participation as high, with 92% saying that they looked at the messages more than 60% of the time. Among those who received text messages, 90% would recommend the program to others. There was no statistical difference between the two group CONCLUSION: A text-messaging-based educational tool had a perceived positive educational value for medical students. This study failed to show any statistically significant impact on NBME examination scores from the text-messaging intervention.

FmCASES National Examination as a Pretest in a Family Medicine Clerkship.

Pretests have been shown to contribute to improved performance on standardized tests by serving to facilitate development of individualized study plans. fmCASES is an existing validated examination used widely in family medicine clerkships throughout the country. Our study aimed to determine if implementation of the fmCASES National Examination as a pretest decreased overall failure rates on the end-of-clerkship National Board of Medical Examiners (NBME) subject examination, and to assess if fmCASES pretest scores correlate with student NBME scores. One hundred seventy-one and 160 clerkship medical students in different class years at a single institution served as the control and intervention groups, respectively. The intervention group took the fmCASES National Examination as a pretest at the beginning of the clerkship and received educational prescriptions based on the results. Chi-square analysis, Pearson correlation, and receiver operating curve analysis were used to evaluate the effectiveness and correlations for the intervention. Students completing an fmCASES National Examination as a pretest failed the end-of-clerkship NBME exam at significantly lower rates than those students not taking the pretest. The overall failure rate for the intervention group was 8.1% compared to 17.5% for the control group (P=0.01). Higher pretest scores correlated with higher NBME examination scores (r=0.55, P<0.001). fmCASES National Examination is helpful as a formative assessment tool for students beginning their family medicine clerkship. This tool introduces students to course learning objectives, assists them in identifying content areas most in need of study, and can be used to help students design individualized study plans.

Effectiveness of the Surgery Core Clerkship Flipped Classroom: a prospective cohort trial

BackgroundThe flipped classroom has been proposed as an alternative curricular approach to traditional didactic lectures but has not been previously applied to a surgery clerkship. MethodsA 1-year prospective cohort of students (n = 89) enrolled in the surgery clerkship was taught using a flipped classroom approach. A historical cohort of students (n = 92) taught with a traditional lecture curriculum was used for comparison. Pretest and post-test performance, end-of-clerkship surveys, and National Board of Medical Examiners (NBME) scores were analyzed to assess effectiveness. ResultsMean pretest and post-test scores increased across all modules (P < .001). There was no difference between mean NBME examination score in the prospective and historical cohorts (74.75 vs 75.74, P = .28). Mean ratings of career interest in surgery increased after curriculum completion (4.75 to 6.50, P < .001), with 90% reporting that the flipped classroom contributed to this increase. ConclusionsImplementation of a flipped classroom in the surgery clerkship is feasible and results in high learner satisfaction, effective knowledge acquisition, and increased career interest in surgery with noninferior NBME performance.

Mobile spaced education for surgery rotation improves National Board of Medical Examiners scores

BackgroundUse of mobile devices for medical education is increasing. The aim of this study was to describe the implementation of a spaced education–based app study program in a third year medical school surgery rotation and examine its effects on National Board of Medical Examiners (NBME) examination performance. Materials and methodsCase-based questions were created and distributed to students on a voluntary basis via a mobile app. Questions were repeated in a spaced based manner until mastered. Students completed surveys regarding study habits and app use. NBME examination scores and prior academic measures were obtained. Users of the app were compared to non-users of the app and the previous years' class. ResultsOne hundred fifty-two students were offered the app; 121 (80%) used it. App users had a final NBME score of 77.5% compared with 68.8% (P < 0.01) in non-users, although their prior academic achievement was better. Categorizing them by their academic achievement and intensity of app use demonstrated significantly higher scores in regular users of the app as compared with casual and non-users in mid (78 versus 75 versus 71.2%, P < 0.01) and low-achieving students (71.4 versus 70 versus 64.6, P < 0.01), but not high-achieving students (85.3 versus 81.1 versus 79.7, P = 0.09). Stepwise linear regression modeling revealed intensity of app use accounted for 6% of the variance in NBME scores. ConclusionsUse of the app-based program was an effective tool associated with higher scores on standardized tests in lower level achieving students.

Education research: case logs in the assessment of medical students in the neurology outpatient clinic.

Using outpatient neurology clinic case logs completed by medical students on neurology clerkships, we examined the impact of outpatient clinical encounter volume per student on outcomes of knowledge assessed by the National Board of Medical Examiners (NBME) Clinical Neurology Subject Examination and clinical skills assessed by the Objective Structured Clinical Examination (OSCE). Data from 394 medical students from July 2008 to June 2012, representing 9,791 patient encounters, were analyzed retrospectively. Pearson correlations were calculated examining the relationship between numbers of cases logged per student and performance on the NBME examination. Similarly, correlations between cases logged and performance on the OSCE, as well as on components of the OSCE (history, physical examination, clinical formulation), were evaluated. There was a correlation between the total number of cases logged per student and NBME examination scores (r = 0.142; p = 0.005) and OSCE scores (r = 0.136; p = 0.007). Total number of cases correlated with the clinical formulation component of the OSCE (r = 0.172; p = 0.001) but not the performance on history or physical examination components. The volume of cases logged by individual students in the outpatient clinic correlates with performance on measures of knowledge and clinical skill. In measurement of clinical skill, seeing a greater volume of patients in the outpatient clinic is related to improved clinical formulation on the OSCE. These findings may affect methods employed in assessment of medical students, residents, and fellows.

Correlating surgical clerkship evaluations with performance on the National Board of Medical Examiners examination

BackgroundEvaluation of medical students during the surgical clerkship is controversial. Performance is often based on subjective scoring, whereas objective knowledge is based on written examinations. Whether these measures correspond or are relevant to assess student performance is unknown. We hypothesized that student evaluations correlate with performance on the National Board Of Medical Examiners (NBME) examination. MethodsData were collected from the 2011–2012 academic year. Medical students underwent a ward evaluation using a seven-point Likert scale assessing six educational competencies. Students also undertook the NBME examination, where performance was recorded as a percentile score adjusted to national standards. ResultsA total of 129 medical students were studied. Scores on the NBME ranged from the 52nd to the 96th percentile with an average in the 75th percentile (±9). Clerkship scores ranged from 3.2–7.0 with a mean of 5.7 (±0.8). There was a strong positive association between higher NBME scores and higher clerkship evaluations shown by a Pearson correlation coefficient of 0.47 (P < 0.001). Students clustered with below average ward evaluations (3.0–4.0) were in the 69.5th percentile of NBME scores, whereas students clustered with above average ward evaluations (6.0–7.0) were in the 79.2th percentile (P < 0.001). ConclusionsA strong positive relationship exists between subjective ward evaluations and NBME performance. These data may afford some confidence to surgical faculty and surgical resident ability to accurately evaluate medical students during clinical clerkships. Understanding factors in student performance may help in improving the surgical clerkship experience.

Case Based Review Questions, Review Sessions, and Call Schedule Type Enhance Knowledge Gains in a Surgical Clerkship

Medical students in surgical clerkships must gain surgical knowledge while doing clinical rotations. We developed a self-learning program, which is case based, the Case Review Question (CRQ) system. Our hypothesis was that students who used CRQs would score higher on the summative test, Surgery NBME (National Board of Medical Examiners). The setting is a surgical clerkship in a Liaison Council for Medical Education (LCME) approved Medical School, with summative examination using the NBME shelf examination in Surgery. Each CRQ document is a series of 20 to 25 questions based on cases. The cases are a paragraph, with pertinent medical facts and extra facts as distracters. The students are encouraged to use these questions to guide study. Students must come to a review session to hear the answers and a discussion. We review the NBME Surgery shelf examination scores taken before and after this program was initiated, along with changes in rotation group size, call schedule type, and other changes. CRQs, review sessions, and call schedule changes improved scores. NBME examination average rose from 77.12 to 82.01 (p = 0.004) after the CRQ program was initiated. Call schedule revision improved scores: intermittent call NBME score was 80.98, whereas night float schedule mean NBME was 84.66 (p < 0.001). During night float call, the CRQ program was already in effect, so that there is no non-CRQ program comparison group. Students scored higher in the second semester, throughout the study. First semester students scored a mean of 79.11, whereas second semester students scored 84.195 (p < 0.001). By the end of the study, there were no failures on the NBME examination, even in the first semester. The factors of presence of the CRQ program, review sessions, call schedule type, and which semester were all significant in a correlation matrix against NBME scores and in a regression analysis (p < 0.001). The CRQ program of case-based self-study was associated with higher scores on the NBME shelf examination, as a summative test of medical knowledge in surgery.

Comparison of Student Performance on Internally Prepared Clerkship Examinations and NBME Subject Examinations

Background: This pilot study compared performance of University of Calgary students on internal clerkship examinations with corresponding National Board of Medical Examiners (NBME) subject examinations.Methods: Between April and October 2007, students completed internal and NBME subject examinations following six mandatory rotations. Local faculty within each discipline set the minimum performance level (MPL) for internal examinations. Two methods of standard setting were considered for NBME exams and a sensitivity analysis was performed. Corresponding internal and NBME examination scores were compared using McNemar’s discordant pair analysis.Results: A significant and unexpected difference in failure rate between internal and external examinations was found in all clerkships. 1.4% of students were below the MPL for internal examinations and 27.3% (modified Angoff) or 25.9% (mean Hofstee compromise) (p&lt;0.0001 for both) for the NBME. The proportion of students below MPL for internal examinations was also below the lower limit of the Hofstee compromise (14.4%).Conclusion: Possible explanations include leniency bias in internal standard setting, discrepant content validity between local curriculum and NBME examinations, difference in student perception of examinations, and performance bias due to unfamiliar units.

Changing the Student Clerkship from Traditional Lectures to Small Group Case-Based Sessions Benefits the Student and The Faculty

Attempts to improve the third year surgery clerkship led to the implementation of faculty-led small group teaching instead of large group lecture-style dissemination of the general surgical curriculum. The intent was to facilitate better faculty-to-student relationships, provide more favorable balance between classroom and clinical surgery, and enhance overall surgical education. Didactic student sessions were reduced from 33 lectures to 8 small group sessions and surgical specialty lectures. A case-based surgical curriculum was utilized and students were organized into small groups led by assigned faculty members. A uniform schedule of topics was prearranged to ensure continuity and avoid duplication of material. The National Board of Medical Examiners (NBME) surgery subject examination raw score and percentile rank assignments were analyzed for 1 medical graduating class taught using the traditional method and compared with the subsequent class taught in small groups. A survey was administered to assess student and faculty regarding the new format. Average NBME percentile rank score for students educated in small groups versus lecture-only groups improved significantly (61.2 vs 55.9, p = 0.04, Student t test). The students reported increased time spent preparing for small group over lecture and more satisfaction with the small group teaching environment. Faculty members reported an increase in time needed to deliver the session but otherwise gave strong positive feedback. Concerns that student performance on standardized testing would suffer from the proposed change were not substantiated as performance on NBME subject examinations actually improved. Additional preparation time, method preference, favorable balance of classroom to clinical exposure, and direct interactions with faculty may be responsible for the observed increase in NBME examination percentile scores. Faculty members were overwhelmingly in favor of the new model and the additional direct contact with students may prove beneficial in junior faculty promotion and career development.

Comparison of the Effectiveness of Two Lecture Methods in an OB/GYN Clerkship

DOI: http://dx.doi.org/10.5915/39-4-8525 Background: A structured, consistent, didactic lecture system is extremely important in a third-year medical clerkship. Yet, its development for the obstetrics/ gynecology (OB/GYN) program is challenging for many reasons. Development of a system that is less dependent on a given lecturer's availability yet still provides complete coverage of the Association of Professors of Gynecology and Obstetrics (APGO) topics for third-year students is desirable. Objective: The present study was designed to compare the effectiveness on third-year OB/GYN medical students of live lectures versus lectures previously recorded to a compact disc. Outcome measures were a midterm internal examination and the National Board of Medical Examiners (NBME) examination scores at the end of the clerkship. Experimental Design: Third-year medical students were assigned by lottery either to the Medical College of Georgia (MCG) campus in Augusta, Georgia, or one its several community-based sites (CBSs) throughout the state of Georgia. All third-year clerkship lectures given by different instructors were videotaped and saved to a compact disc to be played at all CBSs. The students at CBSs were required to observe the lectures in the same order as the live lectures were delivered to students at the MCG campus in Augusta. Results: Student performance on the midterm internal exam and on the NBME examination was compared between the MCG campus and CBSs. Results are shown as raw score mean (SD). Midterm results showed: MCG was 65.6 (8.7) and CBS was 70.4 (17.0). NBME results showed: MCG was 81.1 (6.3) and CBS was 82.2 (7.6). Conclusion: Using examination scores as the outcome, our preliminary data suggest there may be no significant difference in performance on the shelf exam between students receiving a live lecture versus students observing the same lecture electronically.

Use of a personal digital assistant to monitor surgery student work and sleep hours

Background We developed a personal digital assistant (PDA)-based program to assess compliance with our medical student work hours policy, and to correlate work hours with sleep and performance. Methods Medical students on surgery clerkship logged real-time work and sleep hours for 1 week. Estimated work hours, clinical evaluations, and score on the National Board of Medical Examiners (NBME) surgery examination were recorded. Results Thirty-seven students logged work hours, which correlated poorly with estimated work hours and sleep hours. The majority of students overestimated work hours by a mean of 19.5 hours. Twenty-four students transgressed written policy. Increased in-hospital study hours correlated with improved clinical ratings but poorer NBME examination scores. Increased operating room hours correlated with higher NBME examination scores. Conclusions Medical students inaccurately estimate work hours; a PDA-based log facilitates hours monitoring. Unenforced work hour policies are frequently transgressed. Work activity patterns, but not total work hours, correlated with outcomes on standardized written tests and clinical ratings.

The essential elements of undergraduate medical education in obstetrics and gynecology: A comparison of the Association of Professors of Gynecology and Obstetrics Medical Student Educational Objectives and the National Board of Medical Examiners Subject Examination

The objective of this study was to investigate whether the essential elements of the Association of Professors of Gynecology and Obstetrics (APGO) Medical Student Educational Objectives were adequately represented on the National Board of Medical Examiners (NBME) obstetrics and gynecology subject examination, and that the topics questioned on that examination were covered by the APGO objectives. The Undergraduate Medical Education Committee of APGO and the NBME staff separately reviewed the same 2 NBME obstetrics and gynecology subject examinations. The questions were mapped to the 15 essential elements of the APGO educational objectives and comparisons were made to check how well they matched. All the essential elements of the educational objectives were covered by the NBME subject examination. Of the questions on the examination, 99% were deemed appropriate for medical students with 70% of the questions mapping to "Priority 1" objectives. The NBME examination provides an appropriate assessment of mastery of what a medical student should learn, as represented by the APGO Medical Student Educational Objectives.

Why should the first be last? “seasonal” variations in the National Board of Medical Examiners (NBME) Subject Examination Program for medical students in surgery

We recently noticed that many of the students whom past experience had proved good performers did not do as well on the early (July) NBME Surgery examination as expected. Moreover, students with weaker records did better than expected in the later (April) tests. Because the raw scores of the NBME examinations are important for students’ course grades and residency placement, we decided to quantitatively analyze the observed trend to determine if some correction of “seasonal” scores might be necessary. If the raw surgical subject scores vary seasonally, the ability to accurately compare students taking the examination at different rotations is undercut. Medical students are educated and evaluated in a wide variety of styles and settings. Recognizing this, medical schools have increasingly relied on more objective, standardized criteria to aid in the grading and ranking of medical students upon the completion of surgical clerkships. To this end, the NBME has prepared and offers a subject examination (sometimes called a “shelf” examination) for use by medical schools to aid in the evaluation of medical students. Some schools require threshold scores on the NBME subject examination in order for students to pass their given clerkships. Subject examination scores are variably used as a major component in the determination of final student grade in the surgery clerkship. In many schools, the raw scores on the NBME subject examination serve as a prerequisite for achieving clerkship honors. Clerkship honors, in turn, are required by many of the “better” residency programs for consideration for an interview and possible appointment. Thus, the NBME subject examination can play a substantial role in evaluation of surgical clinical clerks and potential residents. Although no comprehensive study has been done on this topic, Ripkey et al have mentioned that medical students sitting for subject examinations in Medicine and Surgery later in the examination season tended to have higher scores relative to their peers who were tested earlier in the examination season. The NBME already acknowledges “seasonal” variations for the subject examination in Medicine. The NBME states in its Spring 2001 Subject Examination Newsletter that, “it is common knowledge that scores in certain clinical exams are progressively higher for students of equivalent ability who take the relevant rotation later in the academic year. This is particularly true for Medicine, where the nature of the subject enables substantive contributions to the score from knowledge accumulated over a variety of rotations. Beginning July 2001, national norms will be provided each quarter as well as the entire year.” The NBME, however, does not account for “seasonal” variations in any of its other subject examinations, such as Surgery, where we thought the greatest degree of “seasonal” variation existed. Our study considers the need for a reexamination of the method by which the raw scores of the Surgery subject examination are interpreted by the NBME and program directors.

Assessing the effectiveness of combining evaluation methods for the early identification of students with inadequate knowledge during a clerkship

This study examined the predictive validity of in-clerkship evaluation methods to identify medical students who have insufficient knowledge. Study subjects were 124 third-year medical students at the Uniformed Services University. Insufficient knowledge was defined by: (1) a clerkship 'pre-test' score one standard deviation below the mean or lower; or (2) any teacher verbally rating a student's general knowledge as 'marginal' or less; or (3) a student did not pass Step One of the United States Medical Licensing Examination (USMLE). We determined sensitivity and specificity using a standard score of h 300 on the end of clerkship National Board of Medical Examiners (NBME) subject examination in medicine as the outcome variable. Sixteen students scored h 300 on the NBME examination. The sensitivity of the 'pre-test' or verbal comments alone was 44% (seven of 16 students). By combining methods, 11 students were identified, for a sensitivity of 69%. The specificity of all methods was > 90%. Using USMLE Step One pass-fail performance did not improve sensitivity. Combining a 'pre-test' and instructors' formal evaluation session comments improves the early identification of students with insufficient knowledge, allowing for formative feedback and remediation during the clerkship.