States Medical Licensing Examination Scores Research Articles

Are Away Rotations Critical for a Successful Match in Orthopaedic Surgery?

Surveys have suggested one of the most important determinants of orthopaedic resident selection is completion of an orthopaedic clerkship at the program director's institution. The purpose of this study was to further elucidate the significance of visiting externships on the resident selection process. We retrospectively reviewed data for all medical students applying for orthopaedic surgery residency from six medical schools between 2006 and 2008, for a total of 143 applicants. Univariate and multivariate regression analyses were used to compare students who matched successfully versus those who did not in terms of number of away rotations, United States Medical Licensing Examination scores, class rank, and other objective factors. Of the 143 medical students, 19 did not match in orthopaedics (13.3%), whereas the remaining 124 matched. On multiple logistic regression analysis, whether a student did more than one home rotation, how many away rotations a student performed, and United States Medical Licensing Examination Step 1 score were factors in the odds of match success. Orthopaedic surgery is one of the most competitive specialties in medicine; the away rotation remains an important factor in match success.

Do United States Medical Licensing Examination (USMLE) Scores Predict In-Training Test Performance for Emergency Medicine Residents?

Background: Residency selection committees commonly utilize USMLE scores as criteria to screen residency applicants. Objectives: The purpose of this study is to evaluate the relationship between United States Medical Licensing Examination (USMLE) and American Board of Emergency Medicine (ABEM) in-training examination scores (ITEs). Methods: In an Accreditation Council for Graduate Medical Education-accredited emergency medicine residency program, data were collected for this retrospective cohort study for the classes of 2002–2006. USMLE Step 1 and 2 scores and the ABEM ITEs were recorded for each post-graduate year (PGY) within the aforementioned time frame. Step 1 and 2 scores were compared to consecutive PGY ABEM ITEs to evaluate for an association. Results: There were 51 USMLE Step 1 and 39 Step 2 scores available for comparison with 153 ABEM ITEs. The mean USMLE Step 1 and Step 2 scores were 228.9 (range 197–252) and 228.4 (range 168–259), respectively. The mean in-training percentiles for the PGY 1, 2, and 3 years were 40.4, 68.3, and 81.7, respectively. The R-squared values for the Step 1 scores compared to the PGY 1, 2, and 3 years' ITEs were 0.25, 0.18, and 0.16, respectively. The R-squared values for Step 2 scores as compared to the ABEM ITEs for the PGY 1, 2, and 3 years were 0.43, 0.44, and 0.38, respectively. Residents who scored below 200 on either USMLE Step 1 or Step 2 had significantly lower mean ABEM ITEs than residents who scored above 200 ( p < 0.05) and were 10-fold more likely than residents who scored above 220 to score below the 70 th percentile in their PGY3 ABEM ITE. Conclusions: USMLE Step 1 scores are mildly correlated and Step 2 scores are moderately correlated with ABEM ITEs. Scoring below 200 on either test is associated with significantly lower ABEM ITEs.

Associations Between United States Medical Licensing Examination (USMLE) and Internal Medicine In-Training Examination (IM-ITE) Scores

Little is known about the associations of previous standardized examination scores with scores on subsequent standardized examinations used to assess medical knowledge in internal medicine residencies. To examine associations of previous standardized test scores on subsequent standardized test scores. Retrospective cohort study. One hundred ninety-five internal medicine residents. Bivariate associations of United States Medical Licensing Examination (USMLE) Steps and Internal Medicine In-Training Examination (IM-ITE) scores were determined. Random effects analysis adjusting for repeated administrations of the IM-ITE and other variables known or hypothesized to affect IM-ITE score allowed for discrimination of associations of individual USMLE Step scores on IM-ITE scores. In bivariate associations, USMLE scores explained 17% to 27% of the variance in IME-ITE scores, and previous IM-ITE scores explained 66% of the variance in subsequent IM-ITE scores. Regression coefficients (95% CI) for adjusted associations of each USMLE Step with IM-ITE scores were USMLE-1 0.19 (0.12, 0.27), USMLE-2 0.23 (0.17, 0.30), and USMLE-3 0.19 (0.09, 0.29). No single USMLE Step is more strongly associated with IM-ITE scores than the others. Because previous IM-ITE scores are strongly associated with subsequent IM-ITE scores, appropriate modeling, such as random effects methods, should be used to account for previous IM-ITE administrations in studies for which IM-ITE score is an outcome.

Selection Criteria for the Integrated Model of Plastic Surgery Residency

The purpose of this study was to identify those qualities and characteristics of fourth-year medical students applying for the Integrated Model of Plastic Surgery residency training that will make a successful plastic surgery resident. A three-part questionnaire was distributed to the training program directors of the 20 Integrated Model of Plastic Surgery programs accredited by the Residency Review Committee for Plastic Surgery by the Accreditation Council on Graduate Medical Education. The first section focused on 19 objective characteristics that directors use to evaluate applicants (e.g., Alpha Omega Alpha Honor Society membership, United States Medical Licensing Examination scores). The second section consisted of 20 subjective characteristics commonly used to evaluate applicants during the interview process. The third section consisted of reasons why, if any, residents failed to successfully complete the training program. Fifteen of the 20 program directors responded to the questionnaire. The results showed that they considered membership in the Alpha Omega Alpha Honor Society to be the most important objective criterion, followed by publications in peer-reviewed journals and letters of recommendation from plastic surgeons known to the director. Leadership capabilities were considered the most important subjective criterion, followed by maturity and interest in academics. Reasons residents failed to complete the training program included illness or death, academic inadequacies, and family demands. The authors conclude that applicants who have achieved high academic honors and demonstrate leadership ability with interest in academics were viewed most likely to succeed as plastic surgery residents by program directors of Integrated Model of Plastic Surgery residencies.

Impact of student ethnicity and primary childhood language on communication skill assessment in a clinical performance examination.

Clinical performance examinations (CPX) with standardized patients (SPs) have become a preferred method to assess communication skills in US medical schools. Little is known about how trainees' backgrounds impact CPX performance. The objective of this paper is to examine the impact of student ethnicity, primary childhood language, and experience of diversity on the communication scores of a high-stakes CPX using SPs. This research was designed as an observational study. The participants of this study were third-year medical students at one US medical school. The measurements used in this study were CPX scores from mandatory exam, student demographics and experience with diversity measured by self-report on a survey, and Medical College Admission Test (MCAT) and United States Medical Licensing Examination (USMLE) scores. A total of 135 students participated. Asian and black students scored lower than white students on the communication portion of the CPX by approximately half a standard deviation (Asian, 67.4%; black, 64.4%; white, 69.4%, p < .05). There were no differences by ethnicity on history/physical exam scores. Multivariate analysis controlling for MCAT verbal scores reduced ethnic differences in communication scores (Asian-white mean differences = 1.95, p = 0.02), but Asian-white differences were eliminated only after sequential models included primary childhood language (difference = 0.57, p = 0.6). Even after controlling for English language knowledge as measured in MCAT verbal scores, speaking a primary childhood language other than English is associated with lower CPX communication scores for Asian students. While poorer communication skills cannot be ruled out, SP exams may contain measurement bias associated with differences in childhood language or culture. Caution is indicated when interpreting CPX communication scores among diverse examinees.

Misrepresentation of Publications Among Radiation Oncology Residency Applicants

Authorship misrepresentations have been described for residency and fellowship applications for various medical specialties. This study assessed the prevalence of misrepresented publications in radiation oncology residency applications. The authors reviewed 117 applications to their residency program for a single 2004 position offered through the National Resident Matching Program. Publications listed on the applications were verified for accuracy, with the results and applicants' demographic information recorded. A total of 49 applicants (42%) claimed authorship of published research citations. The number of published citations averaged 3.6 per applicant (range, 1-23). Of the applicants reporting citations, 22% (11 of 49) listed inaccurate citation information. Overall, 9% of the citations (15 of 174) were considered misrepresentations, with 9% of the total number of applicants (11 of 117) responsible for inaccurate bibliographies. There was a significant relationship of United States Medical Licensing Examination score with publication misrepresentation, in which those with scores of 235 or greater who listed publications were more than 7 times more likely to have inaccurately listed citations (odds ratio, 7.67; 95% confidence interval, 1.12-52.31; P = .04). The misrepresentation of bibliographic citations does exist among radiation oncology residency applicants. Using a comprehensive search, the authors found that 22% of those who had listed at least 1 article had misrepresented publications on their applications.

Orthopaedic In-Training Examination Scores: A Correlation with USMLE Results

Both the United States Medical Licensing Examination and the Orthopaedic In-Training Examination measure factual recall as well as interpretative and problem-solving skills. The former examination is used to a variable degree by postgraduate programs in resident selection. Orthopaedic In-Training Examination scores are one measure of the medical knowledge of residents and are used by all American orthopaedic residency programs on a yearly basis. This investigation was performed to retrospectively review Orthopaedic In-Training Examination scores of orthopaedic residents who took the examination in our program. In addition, we sought to determine whether a relationship existed between performance on the Orthopaedic In-Training Examination and the United States Medical Licensing Examinations taken while in medical school. The records of each orthopaedic resident who took the examination from November 1993 through November 2000 were reviewed. Correlation coefficients and 95% confidence intervals were calculated to assess the relationship, if any, between the Orthopaedic In-Training Examination percentiles and the three-digit scores on the Step-1 and Step-2 United States Medical Licensing Examination. In addition, examination scores were evaluated longitudinally from year-in-training 1 through 4. A significant moderate-sized correlation was found between United States Medical Licensing Examination Step-2 scores and Orthopaedic In-Training Examination score percentiles (p < 0.05); however, with the numbers available, no correlation was seen between United States Medical Licensing Examination Step-1 scores and Orthopaedic In-Training Examination scores. The mean Orthopaedic In-Training Examination scores were in the 66th percentile for year-in-training 1, the 53rd percentile for year 2, the 57th percentile for year 3, and the 50th percentile for year 4. Residents in the laboratory for one year scored in the 88th percentile while in the laboratory (year 0), in the 86th percentile in year 1, and in the 48th percentile in year 4. Although Step-1 United States Medical Licensing Examination scores have been used by our department as a major factor in resident selection historically, our data failed to reveal a significant correlation with performance on the Orthopaedic In-Training Examination. The decrease in Orthopaedic In-Training Examination scores over time for our residents who worked in the laboratory is most likely attributable to multiple factors, including clinical workload hours.

ORTHOPAEDIC IN-TRAINING EXAMINATION SCORES

Background: Both the United States Medical Licensing Examination and the Orthopaedic In-Training Examination measure factual recall as well as interpretative and problem-solving skills. The former examination is used to a variable degree by postgraduate programs in resident selection. Orthopaedic In-Training Examination scores are one measure of the medical knowledge of residents and are used by all American orthopaedic residency programs on a yearly basis. This investigation was performed to retrospectively review Orthopaedic In-Training Examination scores of orthopaedic residents who took the examination in our program. In addition, we sought to determine whether a relationship existed between performance on the Orthopaedic In-Training Examination and the United States Medical Licensing Examinations taken while in medical school. Methods: The records of each orthopaedic resident who took the examination from November 1993 through November 2000 were reviewed. Correlation coefficients and 95% confidence intervals were calculated to assess the relationship, if any, between the Orthopaedic In-Training Examination percentiles and the three-digit scores on the Step-1 and Step-2 United States Medical Licensing Examination. In addition, examination scores were evaluated longitudinally from year-in-training 1 through 4. Results: A significant moderate-sized correlation was found between United States Medical Licensing Examination Step-2 scores and Orthopaedic In-Training Examination score percentiles (p < 0.05); however, with the numbers available, no correlation was seen between United States Medical Licensing Examination Step-1 scores and Orthopaedic In-Training Examination scores. The mean Orthopaedic In-Training Examination scores were in the 66th percentile for year-in-training 1, the 53rd percentile for year 2, the 57th percentile for year 3, and the 50th percentile for year 4. Residents in the laboratory for one year scored in the 88th percentile while in the laboratory (year 0), in the 86th percentile in year 1, and in the 48th percentile in year 4. Conclusion: Although Step-1 United States Medical Licensing Examination scores have been used by our department as a major factor in resident selection historically, our data failed to reveal a significant correlation with performance on the Orthopaedic In-Training Examination. The decrease in Orthopaedic In-Training Examination scores over time for our residents who worked in the laboratory is most likely attributable to multiple factors, including clinical workload hours.

Significantly Improved American Board of Surgery In-Training Examination Scores Associated With Weekly Assigned Reading and Preparatory Examinations

Weekly reading assignments combined with weekly examinations can significantly improve American Board of Surgery In-Training Examination (ABSITE) scores among a group of residents already scoring above the national average. Prospective educational study of surgical residents. Beginning in July 2001, residents were given weekly reading assignments, followed by a multiple-choice examination, prepared and scored by the program director. Mean change in ABSITE scores from 2001 to 2002. The mean total test ABSITE scores significantly improved from 58.7% in 2001 to 75.2% in 2002 (P =.008). The improvement was significantly greater in the basic science portion of the ABSITE (57.6% in 2001 vs 72.5% in 2002, P =.04) than in the clinical management section (57.6% in 2001 vs 68.9% in 2002, P =.11). There were no differences in mean ABSITE scores in other years (1999 to 2000 or 2000 to 2001). There was no correlation between the change in ABSITE scores from 2001 to 2002, level of residency training, United States Medical Licensing Examination scores, or performance on weekly preparatory examinations. Weekly reading assignments combined with weekly preparatory examinations significantly improved mean overall ABSITE scores among a group of residents who were already scoring above the national average.

Selection of obstetrics and gynecology residents on the basis of medical school performance

Objective: The purpose of this study was to determine whether United States Medical Licensing Examination scores during medical school predict resident-in-training examination scores and whether other criteria of medical student performance correlate with the faculty's subjective evaluation of resident performance. Study Design: United States Medical Licensing Examination step I and II scores for 24 residents were compared to their scores on in-training examinations. Faculty evaluated 20 graduated residents by rating both their cognitive and noncognitive clinical performance. Scores from these evaluations were compared with several criteria of their medical school performance. Statistical analysis for all comparisons was linear regression. Results: United States Medical Licensing Examination scores positively correlated with in-training examination scores. United States Medical Licensing Examination scores, honor grades in student clinical rotations, and student interview scores did not correlate with the faculty evaluation of resident performance. Conclusion: Standardized tests of medical student cognitive function predict the resident's performance on standardized tests. Selection criteria that are based on other medical school achievements do not necessarily correlate with overall performance as residents in obstetrics and gynecology. (Am J Obstet Gynecol 2002;186:1091-4.)

Residency selection: Should interviewers be given applicants’ board scores?

Objective: The aim of this study was to determine the influence of advance knowledge of board scores on interviewers’ assessments of residency applicants. Study Design: During a 2-year period we prospectively evaluated our residency selection process. In year 1 interviewers were provided with each candidate’s entire application, whereas in year 2 the United States Medical Licensing Examination scores were not included. In each year interviewers were asked to provide numerical assessments of the applicants solely on the basis of their own impressions of the interviews. Analysis was performed only for evaluations by interviewers who participated during both study periods under review. Interview scores were compared with United States Medical Licensing Examination part I scores within each year by means of a scatter plot and correlation coefficients. Results: Applicant demographic characteristics were similar during years 1 and 2. Interview scores did not differ between year 1 (4.2 ± 0.1) and year 2 (4.3 ± 0.1; P >.05). During year 1 interview and board scores were significantly correlated (correlation coefficient, 0.64; slope of best-fit line, 13.9), whereas there was a negative correlation in year 2 (correlation coefficient, –0.06; slope, –1.3). Conclusion: When they are available to interviewers, markers of academic achievement such as United States Medical Licensing Examination scores may bias the interview evaluation. The interview process when conducted in this manner may simply be a validation process for candidates already judged on the basis of the application alone. Knowledge of United States Medical Licensing Examination scores by the interviewers may therefore negate the interview as an independent means of evaluating candidates. (Am J Obstet Gynecol 2001;184:508-13.)

Is the quality of surgical residency applicants deteriorating?

Background: Among directors of general surgery residencies, there is a concern that the quality of medical students applying to surgical residencies is declining. Methods: Quality of surgical applicants was assessed by several methods including subjective opinions determined by survey and by objective data including student United States Medical Licensing Examination (USMLE) scores of matched candidates. The number of applicants interviewed, total interviews granted, proportion of Alpha Omega Alpha (AOA) students, and the rank order of the candidates matched was obtained by survey. The survey included data on postgraduate year 1 (PGY-1) residents from July 1996 to July 1999. Three mailings were made to 226 US surgical residency programs. Results: Data were obtained from 90 programs. Surgery program directors disagreed with a survey statement that overall quality of applicants had declined ( P <0.01), but agreed with a statement that activities of medical schools to enroll graduating students into primary care had hurt recruitment ( P <0.001). Objective data revealed no change in mean USMLE part I scores of PGY-1 residents over the 4 years ( P = 0.265, power = 0.81). There was no change in proportion of matched residents who were AOA over time. The mean score of all new PGY-1 residents, the rank of the first matched resident, the rank of the last ranked resident, and proportion of AOA students was higher in programs with five or more categorical spots when compared with programs of at most four ( P <0.001). Across all programs, there was a trend to go lower on the rank list to fill categorical positions over time ( P <0.001). Conclusions: There is a perception that medical school policies act to discourage recruitment of quality medical students into general surgery programs, and surgery programs are going deeper into their rank lists to fill categorical positions. However, the average USMLE part I score of applicants to surgical residencies and proportion of AOA applicants has not decreased.

Minorities and the Orthopaedic Profession

Orthopaedic surgery represents a specialty in which applicants are abundant, residency slots are limited, and all candidates presumably are qualified. Orthopaedic residency programs have members of the resident selection committee review medical school performance measures for the purpose of screening candidates to interview and, subsequently, include on their rank order lists for the residency match process. The performance measures universally reviewed include the United States Medical Licensing Examination scores, Alpha Omega Alpha distinction, grade point averages, class rank, personal statements and letters of recommendation. A significant percentage of the residency positions available for orthopaedic training continue to be reserved for candidates who participate in the National Residency Matching Program. Although the selection process is open to any medical student or medical school graduate, the process fails to make any substantive strides in addressing the diminutive representation of certain ethnic and racial minorities within the profession. The intent of this manuscript is to expatiate on the effect various dynamics have on the orthopaedic profession, relative to underrepresented minorities in the profession. Those dynamics comprise (1) the size of the applicant pool; (2) the applicant screening criteria; (3) the underrepresented minority specialty preference; and (4) the perception underrepresented minorities have of orthopaedic surgeons.

The process of resident selection: A view from the residency director's desk

In 1992, the Executive Council of the Association of American Medical Colleges recommended that a computerized application system be investigated. Under this system, the application package would include: 1) a standardized application form, 2) a personal statement, 3) a dean's letter, 4) transcripts, 5) a chairman's letter, and 6) United States Medical Licensing Examination (USMLE) scores. To investigate the value residency directors placed on these items, all directors from Accreditation Council for Graduate Medical Education--approved residency programs in obstetrics and gynecology and family practice residency programs were surveyed. The directors were asked to prioritize the usefulness of the proposed components of the Electronic Residency Application Service application package. Transcripts, the dean's letter, and USMLE scores ranked highest (in descending order) for the obstetrics and gynecology residency directors for making decisions about "inviting a student for an interview," whereas the dean's letter, personal statement, and transcripts ranked highest for the family practice residency directors. Both ranked the interview as most valuable in ranking a student in the residency match. Although these two disciplines do not represent all fields, our study indicates that residency programs are not homogeneous in their value systems and that certain characteristics may predict those values.