National Board Of Medical Examiners Examination Scores Research Articles

Medical Student Education During COVID-19: Electronic Education Does Not Decrease Examination Scores.

During the COVID-19 emergency, medical students were mandated to remain home, creating challenges to providing education remotely for third-year clinical rotations. This study aims to assess student reception and investigate objective outcomes to determine if online learning is a suitable alternative. Medical students enrolled in the third-year surgical clerkship during COVID-19 were asked to participate in a survey. 19 of 27 (70%) students participated. Content, faculty-led lectures, and resident-led problem-based learning (PBL) sessions were assessed using a ten-point Likert scale. National Board of Medical Examiners (NBME) examination, weekly quiz, and oral examination scores were compared to previous years. Student t-tests compared the groups. The median age was 25years. Comparing in-person to electronic sessions, there was no difference in effectiveness of faculty sessions preparing students for NBME (6.2 vs. 6.7, P = .46) or oral examinations (6.4 vs. 6.8, P = .58); there was also no difference in resident-led PBL sessions preparing students for NBME (7.2 vs. 7.2, P = .92) or oral examinations (7.4 vs. 7.6, P = .74). Comparing this group to students from the previous academic year, there was no difference in weekly quiz (85.3 vs. 87.8, P = .13), oral examination (89.8 vs. 93.9, P = .07), or NBME examination (75.3 vs. 77.4, P = .33) scores. Surgical medical didactic education can effectively be conducted remotely through faculty-led lectures and resident-led PBL sessions. Students did not have a preference between in-person and electronic content in preparation for examinations. As scores did not change, electronic education may be adequate for preparing students for examinations in times of crisis such as COVID-19.

Correlation of Stroop Effect in Medical Students to Their Performance in Pharmacology National Board of Medical Examiners (NBME) Examination

ObjectiveThe Stroop test is used to assess selective attention in cognitive psychology. Interference between word meaning and ink color in Incongruent Stroop test prolongs reaction time, which involves the brain searching for some collection of the stored information for prompt use. This is called the Stroop effect. Previous studies have reported the strong correlations between the Stroop effect and academic performance. Therefore, we aimed to study the correlation of the Stroop effect in medical students to their performance in pharmacology NBME examination.MethodsFor this study, 40 medical students who completed their pharmacology NBME examination were selected. The Stroop, congruent and incongruent, tests were performed using the EncephalApp Stroop Test (Android/iOS). In Congruent test (Stroop Off), the subjects were asked to select ink color of the hashtag (#) symbols printed in different ink colors (red, blue, and green). In Incongruent test (Stroop On), the subjects were asked to select the ink color of the words RED, BLUE, and GREEN printed in different ink colors, e.g., the word “RED” printed in blue ink had to be selected as blue. Each subject practiced each test for two attempts and 5 attempts of each test were then recorded.The EncephalApp Stroop Test automatically recorded the results of the tests as duration in seconds, which include Congruent Stroop (OffTime duration), Incongruent Stroop (OnTime duration), OffTime plus OnTime duration, OnTime minus OffTime duration. The results of OnTime minus OffTime duration of the Stroop test and pharmacology NBME examination scores were tabulated in Microsoft Excel and descriptive analysis was done. The correlation of the “OnTime minus OffTime duration” to pharmacology NBME examination scores was determined by a Pearson correlation coefficient and regression analysis.ResultsThe correlation coefficient (adjusted r2) of our study was −0.0255 and significance F was 0.862. The “OnTime minus OffTime duration” in medical students was not correlated with the pharmacology NBME examination score (r (38) = 0.0283, p = 0.862). The regression model predicted only 0.08% (R2 = 0.0008) of the pharmacology NBME examination score of medical students.ConclusionThe Stroop effect in medical students is insignificantly correlated with their performance in pharmacology NBME examination. However, correlations of the Stroop effect in medical students with their performance in other subjects and examinations need to be assessed in further studies.Support or Funding InformationThe study was supported by the Trinity Medical Sciences University School of Medicine, St. Vincent and the Grenadines.

Text4peds: a randomised text-messaging trial.

The use of cell phone text messaging in the medical field is of growing interest, but there are few data examining its value in medical education. The text4peds educational text-messaging program was created for third-year medical students preparing for the National Board of Medical Examiners (NBME) paediatric subject examination. A randomised, controlled trial was conducted with third-year medical students on their paediatric clerkship. Students in the intervention group received daily messages consisting of multiple-choice questions with links to online material. The control group received no text messages. The impact of the intervention was assessed by examining the participants' NBME examination scores and by participant surveys. A total of 162 students participated in the study. There was no statistical difference between the two groups, with the texting group having a mean score of 77.7 and with the non-texting group having a mean score of 77.8 on the NBME. Subgroup analysis examining the effect of anticipated specialty and online material interaction also failed to find any statistically significant difference. Students reported satisfaction with the program, with 84% rating the text messages as helpful. The students rated their participation as high, with 92% saying that they looked at the messages more than 60% of the time. Among those who received text messages, 90% would recommend the program to others. There was no statistical difference between the two group CONCLUSION: A text-messaging-based educational tool had a perceived positive educational value for medical students. This study failed to show any statistically significant impact on NBME examination scores from the text-messaging intervention.

FmCASES National Examination as a Pretest in a Family Medicine Clerkship.

Pretests have been shown to contribute to improved performance on standardized tests by serving to facilitate development of individualized study plans. fmCASES is an existing validated examination used widely in family medicine clerkships throughout the country. Our study aimed to determine if implementation of the fmCASES National Examination as a pretest decreased overall failure rates on the end-of-clerkship National Board of Medical Examiners (NBME) subject examination, and to assess if fmCASES pretest scores correlate with student NBME scores. One hundred seventy-one and 160 clerkship medical students in different class years at a single institution served as the control and intervention groups, respectively. The intervention group took the fmCASES National Examination as a pretest at the beginning of the clerkship and received educational prescriptions based on the results. Chi-square analysis, Pearson correlation, and receiver operating curve analysis were used to evaluate the effectiveness and correlations for the intervention. Students completing an fmCASES National Examination as a pretest failed the end-of-clerkship NBME exam at significantly lower rates than those students not taking the pretest. The overall failure rate for the intervention group was 8.1% compared to 17.5% for the control group (P=0.01). Higher pretest scores correlated with higher NBME examination scores (r=0.55, P<0.001). fmCASES National Examination is helpful as a formative assessment tool for students beginning their family medicine clerkship. This tool introduces students to course learning objectives, assists them in identifying content areas most in need of study, and can be used to help students design individualized study plans.

Mobile spaced education for surgery rotation improves National Board of Medical Examiners scores

BackgroundUse of mobile devices for medical education is increasing. The aim of this study was to describe the implementation of a spaced education–based app study program in a third year medical school surgery rotation and examine its effects on National Board of Medical Examiners (NBME) examination performance. Materials and methodsCase-based questions were created and distributed to students on a voluntary basis via a mobile app. Questions were repeated in a spaced based manner until mastered. Students completed surveys regarding study habits and app use. NBME examination scores and prior academic measures were obtained. Users of the app were compared to non-users of the app and the previous years' class. ResultsOne hundred fifty-two students were offered the app; 121 (80%) used it. App users had a final NBME score of 77.5% compared with 68.8% (P < 0.01) in non-users, although their prior academic achievement was better. Categorizing them by their academic achievement and intensity of app use demonstrated significantly higher scores in regular users of the app as compared with casual and non-users in mid (78 versus 75 versus 71.2%, P < 0.01) and low-achieving students (71.4 versus 70 versus 64.6, P < 0.01), but not high-achieving students (85.3 versus 81.1 versus 79.7, P = 0.09). Stepwise linear regression modeling revealed intensity of app use accounted for 6% of the variance in NBME scores. ConclusionsUse of the app-based program was an effective tool associated with higher scores on standardized tests in lower level achieving students.

Education research: case logs in the assessment of medical students in the neurology outpatient clinic.

Using outpatient neurology clinic case logs completed by medical students on neurology clerkships, we examined the impact of outpatient clinical encounter volume per student on outcomes of knowledge assessed by the National Board of Medical Examiners (NBME) Clinical Neurology Subject Examination and clinical skills assessed by the Objective Structured Clinical Examination (OSCE). Data from 394 medical students from July 2008 to June 2012, representing 9,791 patient encounters, were analyzed retrospectively. Pearson correlations were calculated examining the relationship between numbers of cases logged per student and performance on the NBME examination. Similarly, correlations between cases logged and performance on the OSCE, as well as on components of the OSCE (history, physical examination, clinical formulation), were evaluated. There was a correlation between the total number of cases logged per student and NBME examination scores (r = 0.142; p = 0.005) and OSCE scores (r = 0.136; p = 0.007). Total number of cases correlated with the clinical formulation component of the OSCE (r = 0.172; p = 0.001) but not the performance on history or physical examination components. The volume of cases logged by individual students in the outpatient clinic correlates with performance on measures of knowledge and clinical skill. In measurement of clinical skill, seeing a greater volume of patients in the outpatient clinic is related to improved clinical formulation on the OSCE. These findings may affect methods employed in assessment of medical students, residents, and fellows.

Comparison of Student Performance on Internally Prepared Clerkship Examinations and NBME Subject Examinations

Background: This pilot study compared performance of University of Calgary students on internal clerkship examinations with corresponding National Board of Medical Examiners (NBME) subject examinations.Methods: Between April and October 2007, students completed internal and NBME subject examinations following six mandatory rotations. Local faculty within each discipline set the minimum performance level (MPL) for internal examinations. Two methods of standard setting were considered for NBME exams and a sensitivity analysis was performed. Corresponding internal and NBME examination scores were compared using McNemar’s discordant pair analysis.Results: A significant and unexpected difference in failure rate between internal and external examinations was found in all clerkships. 1.4% of students were below the MPL for internal examinations and 27.3% (modified Angoff) or 25.9% (mean Hofstee compromise) (p&lt;0.0001 for both) for the NBME. The proportion of students below MPL for internal examinations was also below the lower limit of the Hofstee compromise (14.4%).Conclusion: Possible explanations include leniency bias in internal standard setting, discrepant content validity between local curriculum and NBME examinations, difference in student perception of examinations, and performance bias due to unfamiliar units.

Use of a personal digital assistant to monitor surgery student work and sleep hours

Background We developed a personal digital assistant (PDA)-based program to assess compliance with our medical student work hours policy, and to correlate work hours with sleep and performance. Methods Medical students on surgery clerkship logged real-time work and sleep hours for 1 week. Estimated work hours, clinical evaluations, and score on the National Board of Medical Examiners (NBME) surgery examination were recorded. Results Thirty-seven students logged work hours, which correlated poorly with estimated work hours and sleep hours. The majority of students overestimated work hours by a mean of 19.5 hours. Twenty-four students transgressed written policy. Increased in-hospital study hours correlated with improved clinical ratings but poorer NBME examination scores. Increased operating room hours correlated with higher NBME examination scores. Conclusions Medical students inaccurately estimate work hours; a PDA-based log facilitates hours monitoring. Unenforced work hour policies are frequently transgressed. Work activity patterns, but not total work hours, correlated with outcomes on standardized written tests and clinical ratings.

Are NBME examination scores useful in selecting radiology residency candidates?

The quality of the graduates of radiology residency programs is powerfully affected by the quality of the candidates admitted to them, making the evaluation and ranking of residency applicants one of the most important tasks of radiology education. In the applicant evaluation process, evaluators frequently take into account applicants' scores on the National Board of Medical Examiners (NBME) examination, operating on the implicit assumption that these scores help predict future performance as residents. The purpose of this study was to test that assumption. Using a data set consisting of information about 99 residents admitted over a period of 10 residency class years, the authors compared scores for parts I and II of the NBME examination with later results on the American Board of Radiology (ABR) written and oral examinations. Logistic regression was used to model differences in resident performance on the NBME and ABR examinations and to determine whether variance in ABR results could be predicted by variance in NBME examination scores. The authors were unable to demonstrate any statistically significant value of NBME examination scores in predicting eventual performance on the ABR written and oral examinations. The authors' attempt to validate the widespread practice of using NBME examination scores to evaluate and rank applicants to radiology residency programs demonstrated no discernible value for these scores in predicting the success of residents on the ABR examinations.

Use of the USMLE to select residents.

Many studies have examined the relationships between students' performances on the National Board of Medical Examiners (NBME) Part I and Part II examinations and their postgraduate clinical performances. Most studies have found a positive relationship between students' NBME examination scores and the ratings of residents' clinical performances and/or scores on specialty board certification examinations. Surveys of residency directors have also shown that NBME scores are used as part of the process for selecting residents, although other data and other selection criteria are considered more useful than the NBME scores. One area of continued concern is that a large body of data on the United States Medical Licensing Examination (USMLE) is not yet available. However, the predictive validity of the NBME scores supports the use of USMLE scores as part of the review process to screen potential residents. Residency directors should continue to use a variety of criteria in their final selection decisions and are encouraged to use their own program-specific data to establish and monitor particular cutoff scores for screening applicants.

Score reporting on NBME examinations

Because the National Board of Medical Examiners (NBME) will introduce new comprehensive Part I and Part II examinations in 1991, a review has been made of score reporting methods to be used in the new examinations. The review was conducted also because of concern expressed by some that NBME examination scores are misused in medical schools and in resident selection. In this paper, selected aspects of score reporting are defined, the uses of score reports outlined, and the potential for misuse described. It should be noted that the NBME is obliged to make available numerical scores to state medical boards and to examinees. Individual scores are reported to others only with the permission of the examinee. The results of an opinion poll conducted by the NBME of medical educators and medical students are presented. The range of opinion is broad but favors numerical score reporting and a designated pass/fail score.

Using ratings of resident competence to evaluate NBME examination passing standards.

The passing standards of the National Board of Medical Examiners (NBME) examinations were empirically evaluated by analyzing the distributions of examination scores received by 1,994 graduates of one medical school in relation to the clinical competence ratings given to the graduates by their first-year residency directors. A significant association was found between the NBME examination scores and ratings in the cognitive areas of clinical competence. The graduates who scored 420 or less on the NBME Part I or the Part II examination received significantly lower medical knowledge ratings than did the total group of graduates. A similar analysis of NBME Part III examination scores was less clear-cut but also suggested that a score of 420 or less could identify those graduates at significant risk of receiving low knowledge ratings. When low ratings were used as an outcome measure, analysis showed that the NBME Part II examination was not sensitive in detecting such graduates. Based on these data, the authors do not propose changes in the passing standards for the NBME examinations but recommend that these standards continue to be reassessed and further measures be taken to strengthen the internal evaluation methods in medical schools.

Predicting academic performance at a predominantly black medical school.

A study was conducted by the authors that examined the validity of the Medical College Admission Test (MCAT), undergraduate grade-point average (GPA), and "competitiveness" of undergraduate college in predicting the performance of students at a predominantly black college of medicine. The performance measures used in the analysis consisted of course grades in all four years of medical school and scores on both Part I and Part II of the National Board of Medical Examiners (NBME) examinations. The predictive validities of the MCAT scores and undergraduate GPAs were found to be similar to those revealed in earlier studies conducted at predominantly white schools. Two exceptions to these similarities were found. First, the MCAT scores at the black school had a somewhat lower validity in predicting NBME examination scores than was the case at the other schools. Second, of the six MCAT subtest scores, Skills Analysis: Reading had the highest correlation with first-year grades, in contrast to results at the other schools. No differences between men and women were found in the validity of MCAT scores and the GPA. The competitiveness of the undergraduate college attended was found to contribute significantly to the prediction of all measures of medical school performance.