Orthopaedic In-Training Examination Research Articles

Analysis of the Pediatric Orthopedic Surgery Questions on the Orthopaedic In-Training Examination, 2002 through 2006

Pediatric orthopedics has been a frequently tested topic on the Orthopaedic In-Training Examination (OITE). Our goal was to provide direction for resident education efforts by: (1) analyzing the exam's number, topics, and types of pediatric orthopedic surgery questions; (2) examining references cited in the postexam answer packet supplied by the American Academy of Orthopaedic Surgeons; and (3) examining the efficacy of the Orthopaedic Knowledge Update (OKU): Pediatrics 3 book as a source for answers to the pediatric orthopedic questions. We reviewed 5 years (2002 through 2006) of OITEs and the associated American Academy of Orthopaedic Surgeons' answer packets and assessed the OKU: Pediatrics 3 book for topic relativity. Each question was classified into 1 of 6 categories and labeled with a cognitive taxonomy level: 1 (simple recall), 2 (interpretation of data), or 3 (advanced problem-solving). The 6 categories included: (1) pediatric orthopedic knowledge; (2) knowledge of treatment modalities; (3) diagnosis; (4) diagnosis with recognition of associated conditions; (5) diagnosis with further studies; and (6) diagnosis with treatment. The overall percentage of pediatric questions was 14.1%. The most commonly addressed were pediatric elbow fractures, osteomyelitis, and scoliosis. The most common question types were categories 1 (pediatric orthopedic knowledge) and 6 (diagnosis with treatment). The most frequently referenced textbooks were Lovell and Winter's Pediatric Orthopaedics (31%) and Tachdjian's Pediatric Orthopaedics (16%). The most frequently referenced journals were the Journal of Pediatric Orthopaedics (American) (29%) and the Journal of Bone and Joint Surgery (American) (19%). Using only the OKU: Pediatrics 3 review textbook, 65% of the questions could be answered. Knowledge of the topics more likely to be tested may help the orthopedic educator direct a didactic curriculum geared toward the OITE and American Board of Surgery examinations. Although the OKU: Pediatrics 3 book seems to be a good, concise resource for studying for the board examination and OITE, residents should be encouraged to supplement their studying with primary sources. Not applicable.

Educational Resources for the Orthopedic In-Training Examination

Various educational tools are available for the Orthopedic In-Training Examination (OITE). However, many residents are unsure of which resources are the most useful. This study assessed what educational resources residents consider to be most useful and how the top performers prepare for the examination. A web-based survey was distributed to orthopedic residency programs. Residents indicated their program type, year in residency, and most recent OITE score. They also rated (0-4 points) the value of various educational resources. An chi(2) analysis identified resources that were used more frequently by residents who scored in the top 20%. We compared the useful educational resources identified by the entire group, as well as by the top 20% of residents, with what has been the most commonly cited recommended references for the OITE during a 5-year period. Three hundred and sixty residents completed the OITE questionnaire. The journal that was most commonly used by orthopedic residents to prepare for the OITE was the Journal of the American Academy of Orthopedic Surgeons (n = 257, 71%). However, this journal was the third most frequently cited reference for the OITE. Past OITE questions were the highest scored resource with 215 (60%) residents rating them as a 4 ("very useful"). The residents who scored in the top 20% participated in more surgical cases than the other residents (53% vs 35% with 350 or more cases in each group, respectively). The results of this study suggest that many residents, especially those who do well on the OITE, are using the American Academy of Orthopedic Surgeons self-assessment examinations. The results also support the educational importance of exposure to surgical cases in addition to reading and reviewing journals and/or textbooks.

Evaluation of the Orthopaedic In-Training Examination: Spine Questions

The annual Orthopaedic In-Training Examination (OITE) is an objective evaluation administered annually to all residents by the American Academy of Orthopaedic Surgeons. To our knowledge, there are no guidelines for the type of material included on the examination; therefore, it is difficult for many academic centers to develop education programs directed toward improving resident performance on the OITE. Our goals were to determine the most commonly tested subjects in the spine portion of the OITE and to help direct development of an associated teaching program. We analyzed the number, type, anatomic focus, subject matter, and visual diagnostic modalities of spine questions on the OITEs from 2002 through 2007 and identified the most commonly referenced journals. The average annual number of spine questions was 23.1 (8.4% of the examination). The most common types of spine questions related to knowledge (44.5%), evaluation and decision making (29.1%), and diagnosis (26.3%); the most common subject matters were trauma (15.1%) and anatomy (13.7%). The most frequently examined anatomic locations were the cervical (30.9% of questions) and lumbar (17.4%) spines. General spine information (no anatomic focus) accounted for 31.6% of questions. The most commonly referenced journals were Spine and The Journal of Bone and Joint Surgery, American Volume. Developing a study plan focusing on these journals and the most commonly tested topics and question types will better prepare orthopedic residents for the spine questions on the OITE.

Do Scores of the USMLE Step 1 and OITE Correlate with the ABOS Part I Certifying Examination?: A Multicenter Study

The US Medical Licensing Examination (USMLE) and Orthopaedic In-Training Examination (OITE) are commonly used to select medical students or residents, respectively. Knowing how well these examinations predict performance on the American Board of Orthopaedic Surgery (ABOS) Part I certifying examination is important to provide evaluations for medical students and residents. Previous studies comparing the OITE scores with the ABOS Part 1 scores have been limited to one program. Therefore, we compared the scores on the USMLE Step 1 and OITE examinations with those on the ABOS Part I certifying examination using data from four ACGME approved residency programs. We reviewed 202 resident files from 1996 to 2008 from four programs in the same geographic region. Of those, 181 (90%) had complete records. De-identified data were used to compare USMLE Step 1 scores, OITE percentile rank scores, and ABOS Part I percentile rank scores. Pearson coefficients and receiver operator curves were calculated to assess the relationships between tests. We found a correlation of 0.53 between the USMLE Step 1 and ABOS Part I, and an average correlation of 0.50 for postgraduate years (PGY) 2 through 5 OITE scores and ABOS Part I. There was a stepwise increase in correlation from PGY 2 through PGY 5 between the OITE scores and ABOS scores. Those who averaged in the 27th percentile or lower on the OITE had a 57% chance of failing the ABOS Part I examination. USMLE Step 1 scores correlated with ABOS Part I certifying examination scores, and we therefore believe it may be used as one factor in resident selection. Use of the OITE scores in guiding education and feedback appears to be justified.

An Analysis of the Orthopaedic In-Training Examination Sports Section: The Importance of Reviewing the Current Orthopaedic Subspecialty Literature

The Orthopaedic In-Training Examination (OITE), produced by the American Academy of Orthopaedic Surgeons, was first administered in 1963. It was the first surgical subspecialty examination of its kind to be administered to resident trainees. The inaugural OITE consisted of 150 questions derived from the American Board of Orthopaedic Surgery (ABOS) examination1. The number of questions on the OITE and the number of examinees taking the test have increased substantially during subsequent test administrations. The most recent examination (2008) consisted of 275 questions that were administered to 4137 examinees. The current test is divided into twelve domains: sports medicine, foot and ankle, hand, hip and reconstruction, medically related issues, orthopaedic diseases, basic science and tumors, pediatric orthopaedics, rehabilitation, shoulder and elbow, spine, and musculoskeletal trauma. After the OITE examination, the score reports are returned to individual examinees, along with a list of preferred responses for each question and associated literature or textbook references. As a result, the OITE serves as an important educational tool identifying areas of weakness and future study for individual examinees. Recently, Frassica et al.2 and Marker et al.3 published analyses of the OITE pathology and hand sections. The information derived from these studies enables trainees to study more comprehensively, facilitating the development of a core orthopaedic knowledge base. Additionally, faculty may use this information to direct journal club topics and didactic lectures and to improve their educational curriculum. To our knowledge, no analysis of the sports medicine questions on the OITE has been published. The purpose of this study was to systematically examine the OITE sports medicine questions, along with the associated answers and recommended reading lists, during a five-year period. This analysis produced a list of commonly tested topics and provides an educational resource that residents and attending surgeons may use to …

Relationship Among United States Medical Licensing Step I, Orthopedic In-Training, Subjective Clinical Performance Evaluations, and American Board of Orthopedic Surgery Examination Scores: A 12-Year Review of an Orthopedic Surgery Residency Program

To improve the understanding of relationships among United States Medical Licensing Examination (USMLE Step I), Orthopedic In-Training Examination (OITE), Subjective Clinical Performance Evaluations, and American Board of Orthopedic Surgery Examination Part I (Abos-I) and Part II (Abos-II), which would help residency programs better achieve their educational mission. A 12-year descriptive study of retrospectively collected data. One residency program with 47 resident participants. Residents that failed Abos-I and Abos-II had lower program mean OITE year-in-training (YIT) percentile rank scores. The program mean OITE YIT percentile rank score had a moderate relationship with Abos-I (% correct) score (r = 0.68, p < 0.0001) and an insignificant relationship with USMLE Step I (3-digit) score (r = 0.22, p = 0.13). Residents with upper quartile (>or=220) USMLE Step I (3-digit) scores for our program had higher program mean OITE YIT percentile rank scores and Abos-I (% correct) scores than residents with lower quartile scores (<or=202). Residents who scored in the upper quartile (>or=55) for the program mean OITE YIT percentile rank score had higher Abos-I (% correct) scores than residents who did not. Residents who scored in the lower quartile for the third postgraduate year (PGY-3) program OITE YIT percentile rank score or for the program mean OITE YIT percentile rank score had a 5.2 and 5.8 time greater Abos-I failure risk, respectively. The program PGY-3 OITE YIT percentile rank score was the strongest Abos-I (% correct) score discriminator. Resident Abos-I (% correct), program mean OITE YIT, and program PGY-3 OITE YIT percentile rank scores were the strongest discriminators for Abos-II passage. Residents with a program mean OITE YIT percentile rank score >or=28, program PGY-3 OITE YIT percentile rank score >or=39, and USMLE Step I (3-digit) score >or=207 were more likely to pass Abos-I and II. Residents that had lower quartile USMLE Step I (3-digit) scores for our program had a 2.3 time greater Abos-I failure risk. Program residents with >or=2 below-average subjective clinical performance evaluations had lower Abos-I (% correct) scores but had similar Abos-I and II pass rates. Our program uses the USMLE Step I (3-digit) score as a preacceptance estimate of likely supplemental guided mentoring needs. Program mean OITE YIT percentile rank and PGY-3 OITE YIT percentile rank scores help identify educational deficiencies and predict eventual Abos-I and II passage. Subjective clinical performance evaluations provide important supplemental information regarding professionalism, communication, and patient care skills.

Levels of Evidence Are Low for Clinical Management Questions on the Orthopaedic In-Training Examination

The Orthopaedic In-Training Examination (OITE) has been administered to orthopaedic surgery residents throughout the United States since 1963 and, over the past forty-five years, has become “an integral part of orthopaedic education.”1 The test was designed as an educational tool for residents, but it also is used to assess the performance of residents relative to that of others. Additionally, an implicit function of the test is to help to define the corpus of core knowledge for orthopaedic surgery residency. As noted by the participants of the Academic Orthopaedic Society symposium on musculoskeletal education2, “Testing organizations are influential in shaping the curriculum. Faculty are correct to teach what students need to know to pass their [tests].” The Evaluation Committee of the American Academy of Orthopaedic Surgeons (AAOS) writes the questions, amalgamates the examination, and scores the OITE. As such, test takers can reasonably infer that the material tested on the OITE reflects the Academy’s vision of the orthopaedic surgery core curriculum for residents in training, and, owing to the central position of the AAOS, the OITE is an important influence on the orthopaedic surgery curriculum for residents in the United States. For this reason, among others, it is worthwhile to examine the OITE for the purposes of studying what exactly residents are expected to master. One aspect of that analysis centers on the levels of evidence on which the OITE questions are based. Although the principles of evidence-based medicine3 do not limit the practitioner to the randomized trial (or any particular study type), there has been particular recent emphasis on the level of evidence associated with individual studies. In their editorial introducing the policy of explicitly listing this level, for example, the editors of this journal stated: “Orthopaedic surgeons have always based their clinical care on evidence… …

Analysis of the Foot and Ankle Section of the Orthopaedic In-Training Examination (OITE)

The Orthopaedic In-Training Examination (OITE) was first administered in 1963 to orthopaedic surgical residents. In 2007, an analysis of the OITE tumor-related questions was published. To our knowledge, no analysis of the foot and ankle-related questions is available in the literature. We analyzed the foot and ankle questions on the OITE over a 5-year period (2004 to 2008). The average number of foot and ankle questions, topics, specific imaging modalities, treatment options, and taxonomic classification were quantified and described. Each year, approximately 38 of the 275 (14.2%) examination questions required knowledge of foot and ankle topics. Commonly tested topics include basic foot and ankle anatomy, the diabetic foot, ankle fractures, and calcaneus fractures. Approximately seventeen questions per year test simple recall of specific facts, while approximately sixteen questions test the examinee's ability to establish a diagnosis and subsequent treatment plan from clinical history and/or radiographs. Analysis of the OITE foot and ankle domain question content and type may aid test-writers when they write future OITE questions. This information can be used to ensure adequate coverage of the breadth of foot and ankle surgery on future OITE examinations.

Orthopaedic Journal Publications and Their Role in the Preparation for the Orthopaedic In-Training Examination

In the last decade, there has been an effort to refocus on the efficacy of resident education practices. Much of this effort was summarized by a statement released in 1999 by the Accreditation Council for Graduate Medical Education, which defined medical knowledge as one of six clinical care domains in which residents must receive instruction and show competency. Since then, there have been a number of subsequent reformations to residency training, such as the limitation of resident work hours to no more than eighty hours per week. Within the orthopaedic community, these changes have resulted in a tremendous interest in optimizing resident education. This interest is reflected in the literature by a near doubling in the number of studies concerning orthopaedic resident education and/or training in the past five years from 2004 to 2008 (n = 75) as compared with the preceding five years from 1999 to 2003 (n = 46) (Fig. 1). This interest served as one of the justifications for the present study as we attempted to understand the role of using the recent medical literature for the education and training of orthopaedic residents. Fig. 1 Line graph depicting the number of studies listed on PubMed related to orthopaedic resident education or training, stratified by year of publication. One of the tools that the orthopaedic community has established to evaluate resident training is the Orthopaedic In-Training Examination (OITE)1. The examination focuses on material supported by current scientific literature. The purpose of the present study was to evaluate what literature resources might be most beneficial for orthopaedic residents who are preparing for the OITE. The primary questions for the present study were: (1) What are the recommended references for the questions for each of the subject areas tested on the examination? (2) Do the subject areas tested correlate with …

Predictors of success on the American Board of Orthopaedic Surgery examination.

Predictors of success of orthopaedic residents on the American Board of Orthopaedic Surgery (ABOS) examination are controversial. We therefore evaluated numerous variables that may suggest or predict candidate performance on the ABOS examination. We reviewed files of 161 residents (all graduates) from one residency program distributed into two study groups based on whether they passed or failed their first attempt on the ABOS Part I or Part II examination from 1991 through 2005. Predictors of success/failure on the ABOS I included the mean percentile score on the Orthopaedic In-Training Examination (OITE) (Years 2 through 4), the percentile OITE score in the last year of training, US Medical Licensing Examination (USMLE) score, Dean's letter, election to Alpha Omega Alpha (AOA), and number of honors in selected third-year clerkships. All but the USMLE score predicted passing the ABOS Part II examination. These data suggest there are objective predictors of residents' performance on the ABOS Part I and Part II examinations.

Impact of the 80-hour Workweek on Surgical Exposure and National In-Training Examination Scores in an Orthopedic Residency Program

This study examines the impact of the 80-hour workweek on the number of surgical cases performed by PGY-2 through PGY-5 orthopedic residents. We also evaluated orthopedic in-training examination (OITE) scores during the same time period. Data were collected from the Accreditation Council for Graduate Medical Education (ACGME) national database for 3 academic years before and 5 years after July 1, 2003. CPT surgical procedure codes logged by all residents 3 years before and 5 years after implementation of the 80-hour workweek were compared. The average raw OITE scores for each class obtained during the same time period were also evaluated. Data were reported as the mean +/- standard deviation (SD), and group means were compared using independent t-tests. No statistical difference was noted in the number of surgical procedure codes logged before or after the institution of the 80-hour week during any single year of training. However, an increase in the number of CPT codes logged in the PGY-3 years after 2003 did approach significance (457.7 vs 551.9, p = 0.057). Overall, the average number of cases performed per resident increased each year after implementation of the work-hour restriction (464.4 vs 515.5 cases). No statistically significant difference was noted in the raw OITE scores before or after work-hour restrictions for our residents or nationally. We found no statistical difference for each residency class in the average number of cases performed or OITE scores, although the total number of cases performed has increased after implementation of the work-hour restrictions. We also found no statistical difference in the national OITE scores. Our data suggest that the impact of the 80-hour workweek has not had a detrimental effect on these 2 resident training measurements.

Current Hand Surgery Literature as an Educational Tool for the Orthopaedic In-Training Examination

The origins of hand surgery as a distinct surgical field in the United States are often credited to the pioneering efforts of Asa Sterling Bunnell around the time of the Second World War1. It has since evolved into an increasingly complex specialty involving hand transplantation, joint arthroplasty, and microsurgery2-6. Recognizing the need for oversight to ensure proper training and education, the orthopaedic, plastic, and surgery primary boards jointly approved a Certificate of Added Qualification in Hand Surgery in 1985, and the first examination for certification in hand surgery was administered in 19897. Because of the continually emerging complexities in this field, it has become increasingly important to evaluate the training of orthopaedic residents in order to ensure that their education and understanding of hand surgery is adequate prior to graduation and possible application for fellowship. The Orthopaedic In-Training Examination (OITE) was established as a measure for orthopaedic resident education and knowledge in the 1960s8. The OITE provides a standardized format to evaluate the hand knowledge of orthopaedic residents. This examination is designed to focus on evidence-based knowledge and skills that a surgeon should know in order to be able to practice effectively. Recent orthopaedic publications are often recommended as one of the main sources of information for surgeons in training, and it has recently been reported that there is a positive correlation between performance on the OITE and frequent review of current orthopaedic journals9. However, we know of no published study that has assessed the correlation between the literature and the hand surgery content tested in the OITE. The primary questions in the present study were: (1) What literature should be recommended to orthopaedic residents who are receiving training and instruction regarding hand surgery and are preparing for …

Orthopaedic In-Training Examination Performance: A Nine-Year Review of a Residency Program Database

The Orthopaedic In-Training Examination (OITE) provides an objective way for both the residency program and the resident to monitor progress. This longitudinal descriptive study tracks the OITE performance of 16 residents from one orthopaedic surgery residency program over nine years (1997-2005). Domain comparisons are made to national averages (N > 3000) by mean difference and split middle celeration line assessment trend analysis to evaluate program strengths and weaknesses. Further evaluation by postgraduate year (PGY) is made of domains that were deemed to be in need of moderate attention. Resident performance for the medically related issues, rehabilitation, and sports medicine domains were deemed acceptable. Resident performance on the orthopaedic science, orthopaedic diseases, hip and knee reconstruction, spine, and shoulder and elbow domains were deemed to be in need of minor attention. Musculoskeletal trauma, pediatric orthopaedics, foot and ankle, and hand domains were deemed to be in need of moderate attention. Further analysis of these domains by PGY revealed less positive slopes for the pediatric orthopaedics and musculoskeletal trauma domains, and a negative slope for the hand domain between PGY-3 and PGY-4, each indicative of lower domain-specific OITE scores than the national mean. A similar, less positive slope was identified between PGY-2 and PGY-3 for the foot and ankle domain. A longitudinal descriptive review of orthopaedic surgery resident OITE performance enables the departmental education committee to identify curricular content areas that require attention, select the most efficient modes of educational information delivery, facilitate learning through the application of innovative educational technologies, establish measurable educational program goals, and more prescriptively allocate personnel and equipment resources.

Resident Selection

Selection of the best applicants for orthopaedic residency programs remains a difficult problem. Most quantifiable factors for residency selection evaluate test-taking ability and grades rather than other aspects, such as patient care, professionalism, moral reasoning, and integrity. Four current department members on our resident selection committee ranked four consecutive classes of orthopaedic residents interviewed for residency. We ranked incoming residents in order of best to least qualified and compared those rankings with rank lists by the same faculty on completion of residency. Rankings also were compared with the residents' United States Medical Licensing Examination (USMLE) Part I scores, American Board of Orthopaedic Surgery (ABOS) Part I scores, and fourth-year Orthopaedic-in-Training Examination (OITE) scores. We found fair or poor correlations between the residents' initial rankings, rankings on graduation, and their USMLE, ABOS, and OITE scores. The only relatively strong correlation found was between the OITE and ABOS scores. Despite the faculty's consensus regarding selection criteria, interviewers did not agree in their rankings of residents on graduation. Additional work is necessary to refine the inexact yet important science of selecting residency applicants.

A Quantitative Composite Scoring Tool for Orthopaedic Residency Screening and Selection

The ability to accurately screen and select orthopaedic resident applicants with eventual successful outcomes has been historically difficult. Many preresidency selection variables are subjective in nature and a more standardized objective scoring method seems desirable. A quantitative composite scoring tool (QCST) to be used in a standardized manner to help predict orthopaedic residency performance from application materials was developed. In 64 orthopaedic residents, four predictors (United States Medical Licensing Examination [USMLE] Part I scores, Alpha Omega Alpha status, junior year clinical clerkship honors grades, and the QCST score) were analyzed with respect to four residency outcomes assessments. The outcomes included three standardized assessments, the orthopaedic in-training examination scores (OITE), the American Board of Orthopaedic Surgery (ABOS) written and oral examinations, and an internal outcomes assessment, attainment of satisfactory chief resident associate (CRA) status. Collectively, the QCST score had the strongest association as a predictor for all three standardized outcomes assessments (p < 0.001). Honors grades during junior years clinical clerkships was most strongly associated with satisfactory CRA status (p < 0.001). A composite scoring tool that is an effective predictor of orthopaedic resident outcomes can be developed. Additional work is still required to refine this scoring tool for orthopaedic residency screening and selection.

Orthopaedic In-Training Examination Scores: A Correlation with USMLE Results

Both the United States Medical Licensing Examination and the Orthopaedic In-Training Examination measure factual recall as well as interpretative and problem-solving skills. The former examination is used to a variable degree by postgraduate programs in resident selection. Orthopaedic In-Training Examination scores are one measure of the medical knowledge of residents and are used by all American orthopaedic residency programs on a yearly basis. This investigation was performed to retrospectively review Orthopaedic In-Training Examination scores of orthopaedic residents who took the examination in our program. In addition, we sought to determine whether a relationship existed between performance on the Orthopaedic In-Training Examination and the United States Medical Licensing Examinations taken while in medical school. The records of each orthopaedic resident who took the examination from November 1993 through November 2000 were reviewed. Correlation coefficients and 95% confidence intervals were calculated to assess the relationship, if any, between the Orthopaedic In-Training Examination percentiles and the three-digit scores on the Step-1 and Step-2 United States Medical Licensing Examination. In addition, examination scores were evaluated longitudinally from year-in-training 1 through 4. A significant moderate-sized correlation was found between United States Medical Licensing Examination Step-2 scores and Orthopaedic In-Training Examination score percentiles (p < 0.05); however, with the numbers available, no correlation was seen between United States Medical Licensing Examination Step-1 scores and Orthopaedic In-Training Examination scores. The mean Orthopaedic In-Training Examination scores were in the 66th percentile for year-in-training 1, the 53rd percentile for year 2, the 57th percentile for year 3, and the 50th percentile for year 4. Residents in the laboratory for one year scored in the 88th percentile while in the laboratory (year 0), in the 86th percentile in year 1, and in the 48th percentile in year 4. Although Step-1 United States Medical Licensing Examination scores have been used by our department as a major factor in resident selection historically, our data failed to reveal a significant correlation with performance on the Orthopaedic In-Training Examination. The decrease in Orthopaedic In-Training Examination scores over time for our residents who worked in the laboratory is most likely attributable to multiple factors, including clinical workload hours.

ORTHOPAEDIC IN-TRAINING EXAMINATION SCORES

Background: Both the United States Medical Licensing Examination and the Orthopaedic In-Training Examination measure factual recall as well as interpretative and problem-solving skills. The former examination is used to a variable degree by postgraduate programs in resident selection. Orthopaedic In-Training Examination scores are one measure of the medical knowledge of residents and are used by all American orthopaedic residency programs on a yearly basis. This investigation was performed to retrospectively review Orthopaedic In-Training Examination scores of orthopaedic residents who took the examination in our program. In addition, we sought to determine whether a relationship existed between performance on the Orthopaedic In-Training Examination and the United States Medical Licensing Examinations taken while in medical school. Methods: The records of each orthopaedic resident who took the examination from November 1993 through November 2000 were reviewed. Correlation coefficients and 95% confidence intervals were calculated to assess the relationship, if any, between the Orthopaedic In-Training Examination percentiles and the three-digit scores on the Step-1 and Step-2 United States Medical Licensing Examination. In addition, examination scores were evaluated longitudinally from year-in-training 1 through 4. Results: A significant moderate-sized correlation was found between United States Medical Licensing Examination Step-2 scores and Orthopaedic In-Training Examination score percentiles (p < 0.05); however, with the numbers available, no correlation was seen between United States Medical Licensing Examination Step-1 scores and Orthopaedic In-Training Examination scores. The mean Orthopaedic In-Training Examination scores were in the 66th percentile for year-in-training 1, the 53rd percentile for year 2, the 57th percentile for year 3, and the 50th percentile for year 4. Residents in the laboratory for one year scored in the 88th percentile while in the laboratory (year 0), in the 86th percentile in year 1, and in the 48th percentile in year 4. Conclusion: Although Step-1 United States Medical Licensing Examination scores have been used by our department as a major factor in resident selection historically, our data failed to reveal a significant correlation with performance on the Orthopaedic In-Training Examination. The decrease in Orthopaedic In-Training Examination scores over time for our residents who worked in the laboratory is most likely attributable to multiple factors, including clinical workload hours.

Relation of Residency Selection Factors to Subsequent Orthopaedic In-training Examination Performance

Orthopaedic surgery remains one of the most competitive specialties, with more than a 99% match fill rate in the past several years. An oversupply of qualified applicants leads to intense competition for these residency spots, allowing program directors to be more selective in choosing their future residents. Although many previous studies have documented factors important to program directors in the admission process, less is known about how preselection factors correlate with subsequent performance in a residency program. The relation of both demographic and academic factors with subsequent performance on the Orthopaedic In-Training Examination (OITE) were studied. These factors include United States Medical Licensing Examination (USMLE) step I scores, Alpha Omega Alpha (AOA) status, research publications, age entering residency, marital status, and medical school affiliation. In this study, the only statistically significant correlations to OITE scores were USMLE step I performance and marital status. Those residents who had previously scored above 220 on the USMLE step I had higher average OITE scores than those scoring below 220. Residents who were married also had higher average OITE scores. A trend with regard to AOA status also was found, with residents scoring slightly higher on the OITE if they were members of AOA. Few preresidency variables correlate to success during an orthopaedic residency.

Measuring medical education and curriculum during orthopedic surgical residency 1

Evaluating residency programs requires objective assessment tools, but few are readily available. The purpose of this study was to measure education by correlating resident test scores with several measurements of educator performance. The study group included residents and educators from a single residency program. We performed a retrospective analysis of scores from the Orthopaedic In-Training Examination collected during a 6-year period. Resident examination scores were indexed by dividing program averages by national averages to determine yearly score trends and then were correlated with educator attendance and teaching hours. Subspecialty scores were ranked to gauge residency strengths and weaknesses. Teaching hours devoted to subspecialties were compared with test scores to measure curricular emphases and to appraise teaching efficiency. Yearly average examination scores were proportional to national averages (P < 0.001). However, of 3436 possible educator-score associations, only 15 scores correlated highly (r > 0.9) with educators, and only 26 were significant (P < 0.05). Trend analysis put subspecialty scores in yearly perspective. Ranking was inaccurate until scores were indexed to the national average. In 2002, the distribution of 238 teaching hours ranged from 4 to 48 h for subspecialties, and 9 of 12 subspecialties were emphasized disproportionately to the examination. Teaching efficiency varied more than 10-fold by subspecialty. The creation of a score index helped to identify and address imbalances between teaching hours devoted to subspecialties and resident needs as evidenced by low In-Training examination scores. The present study improved educator accountability by correlating measurements of teaching and learning.

Passing the Boards

Klein, Gregg R. MD1; Austin, Matthew S. MD1; Randolph, Susan1; Sharkey, Peter F. MD1; Hilibrand, Alan S. MD1 Author Information