Orthopaedic In-Training Examination Research Articles

Analysis of Hip Preservation Questions on the Orthopaedic In-Training Examination Over the Past 20 Years.

The orthopaedic in-training examination (OITE) is a 275-question test for orthopaedic residents administered annually. As the field of orthopaedics changes, the OITE evolves its content. The incidence of hip preservation-related procedures has increased substantially over the past decade; nonetheless, an analysis of the trends of hip preservation questions on the OITE has not yet been performed. The purpose of the study was to evaluate the number and type of questions on the OITE related to hip preservation to determine whether trends over time paralleled the increases in hip-related care in clinical practice. It was hypothesized that the frequency of hip preservation questions on the OITE would increase with time. Cross-sectional study. Each OITE between 2002 and 2021 was reviewed for questions related to hip preservation. The types of questions included under "hip preservation" were those related to femoroacetabular impingement (FAI), athletic soft tissue injuries of the hip, acetabular labral tears, hip arthroscopy, and surgical management of adult hip dysplasia-excluding arthroplasty. Questions were quantified and categorized by topic, taxonomy level, associated imaging, and cited sources. There were 30 hip preservation-related questions between 2002 and 2021. Of these, 77% occurred within the past 10 years. Also, 14 questions (47%) had associated images in the question stem-the most common being radiographs (n = 8 questions). The most commonly tested subcategories were FAI (n = 11 questions [37%]), athletic injuries (n = 7 questions [23%]), and anatomy (n = 7 questions [23%]). Over the last 10 years, 97.9% of citations were from journal articles-the most common being the Journal of the American Academy of Orthopaedic Surgeons, Clinical Orthopedics and Related Research, and the American Journal of Sports Medicine. The frequency of hip preservation-related questions on the OITE has increased with time, reflecting trends in clinical practice.

Performance of Two Artificial Intelligence Generative Language Models on the Orthopaedic In-Training Examination.

Artificial intelligence (AI) generative large language models are powerful and increasingly accessible tools with potential applications in health care education and training. The annual Orthopaedic In-Training Examination (OITE) is widely used to assess resident academic progress and preparation for the American Board of Orthopaedic Surgery Part 1 Examination. Open AI's ChatGPT and Google's Bard generative language models were administered the 2022 OITE. Question stems that contained images were input without and then with a text-based description of the imaging findings. ChatGPT answered 69.1% of questions correctly. When provided with text describing accompanying media, this increased to 77.8% correct. In contrast, Bard answered 49.8% of questions correctly. This increased to 58% correct when text describing imaging in question stems was provided (P<.0001). ChatGPT was most accurate in questions within the shoulder category, with 90.9% correct. Bard performed best in the sports category, with 65.4% correct. ChatGPT performed above the published mean of Accreditation Council for Graduate Medical Education orthopedic resident test-takers (66%). There is significant variability in the accuracy of publicly available AI models on the OITE. AI generative language software may play numerous potential roles in the future in orthopedic education, including simulating patient presentations and clinical scenarios, customizing individual learning plans, and driving evidence-based case discussion. Further research and collaboration within the orthopedic community is required to safely adopt these tools and minimize risks associated with their use. [Orthopedics. 2024;47(3):e146-e150.].

Evaluating ChatGPT's Capabilities on Orthopedic Training Examinations: An Analysis of New Image Processing Features.

Introduction The efficacy of integrating artificial intelligence (AI) models like ChatGPT into the medical field, specifically orthopedic surgery, has yet to be fully determined. The most recent adaptation of ChatGPT that has yet to be explored is its image analysis capabilities. This study assesses ChatGPT's performance in answering Orthopedic In-Training Examination (OITE) questions, including those that require image analysis. Methods Questions from the 2014, 2015, 2021, and 2022 AAOS OITE were screened for inclusion. All questions without images were entered into ChatGPT 3.5 and 4.0 twice. Questions that necessitated the use of images were only entered into ChatGPT 4.0 twice, as this is the only version of the system that can analyze images. The responses were recorded and compared to AAOS's correct answers, evaluating the AI's accuracy and precision. Results A total of 940 questions were included in the final analysis (457 questions with images and 483 questions without images). ChatGPT 4.0 performed significantly better on questions that did not require image analysis (67.81% vs 47.59%, p<0.001). Discussion While the use of AI in orthopedics is an intriguing possibility, this evaluation demonstrates how, even with the addition of image processing capabilities, ChatGPT still falls short in terms of its accuracy. As AI technology evolves, ongoing research is vital to harness AI's potential effectively, ensuring it complements rather than attempts to replace the nuanced skills of orthopedic surgeons.

Generative Artificial Intelligence Performs at a Second-Year Orthopedic Resident Level.

Introduction Artificial intelligence (AI) models using large language models (LLMs) and non-specific domains have gained attention for their innovative information processing. As AI advances, it's essential to regularly evaluate these tools' competency to maintain high standards, prevent errors or biases, and avoid flawed reasoning or misinformation that could harm patients or spread inaccuracies.Our study aimed to determine the performance of Chat Generative Pre-trained Transformer (ChatGPT) by OpenAIand Google BARD (BARD) in orthopedic surgery, assess performance based on question types, contrast performance between different AIs and compare AI performance to orthopedic residents. Methods We administered ChatGPT and BARD 757 Orthopedic In-Training Examination (OITE) questions. After excluding image-related questions, the AIs answered 390 multiple choice questions, all categorized within 10 sub-specialties (basic science, trauma, sports medicine, spine, hip and knee, pediatrics, oncology, shoulder and elbow, hand, and food and ankle) and three taxonomy classes (recall, interpretation, and application of knowledge).Statistical analysis was performed to analyze the number of questions answered correctly by each AI model, the performance returned by each AI model within the categorized question sub-specialty designation, and the performance of each AI model in comparison to the results returned by orthopedic residents classified by their respective post-graduate year (PGY) level. Results BARD answered more overall questions correctly (58% vs 54%, p<0.001). ChatGPT performed better in sports medicine and basic science and worse in hand surgery, while BARD performed better in basic science (p<0.05). The AIs performed better in recall questions compared to the application of knowledge (p<0.05). Based on previous data, it ranked in the 42nd-96th percentile for post-graduate year ones (PGY1s), 27th-58thfor PGY2s, 3rd-29th for PGY3s, 1st-21st for PGY4s, and 1st-17th for PGY5s. Discussion ChatGPT excelled in sports medicine but fell short in hand surgery, while both AIs performed well in the basic science sub-specialty but performed poorly in the application of knowledge-based taxonomy questions. BARD performed better than ChatGPT overall.Although the AI reached the second-year PGY orthopedic resident level, it fell short of passing the American Board of Orthopedic Surgery (ABOS). Its strengths in recall-based inquiries highlight its potential as an orthopedic learning and educational tool.

Comparison of Artificial Intelligence to Resident Performance on Upper-Extremity Orthopaedic In-Training Examination Questions

Comparison of Artificial Intelligence to Resident Performance on Upper-Extremity Orthopaedic In-Training Examination Questions

Can generative artificial intelligence pass the orthopaedic board examination?

BackgroundResident training programs in the US use the Orthopaedic In-Training Examination (OITE) developed by the American Academy of Orthopaedic Surgeons (AAOS) to assess the current knowledge of their residents and to identify the residents at risk of failing the Amerian Board of Orthopaedic Surgery (ABOS) examination. Optimal strategies for OITE preparation are constantly being explored. There may be a role for Large Language Models (LLMs) in orthopaedic resident education. ChatGPT, an LLM launched in late 2022 has demonstrated the ability to produce accurate, detailed answers, potentially enabling it to aid in medical education and clinical decision-making. The purpose of this study is to evaluate the performance of ChatGPT on Orthopaedic In-Training Examinations using Self-Assessment Exams from the AAOS database and approved literature as a proxy for the Orthopaedic Board Examination. Methods301 SAE questions from the AAOS database and associated AAOS literature were input into ChatGPT's interface in a question and multiple-choice format and the answers were then analyzed to determine which answer choice was selected. A new chat was used for every question. All answers were recorded, categorized, and compared to the answer given by the OITE and SAE exams, noting whether the answer was right or wrong. ResultsOf the 301 questions asked, ChatGPT was able to correctly answer 183 (60.8%) of them. The subjects with the highest percentage of correct questions were basic science (81%), oncology (72.7%, shoulder and elbow (71.9%), and sports (71.4%). The questions were further subdivided into 3 groups: those about management, diagnosis, or knowledge recall. There were 86 management questions and 47 were correct (54.7%), 45 diagnosis questions with 32 correct (71.7%), and 168 knowledge recall questions with 102 correct (60.7%). ConclusionsChatGPT has the potential to provide orthopedic educators and trainees with accurate clinical conclusions for the majority of board-style questions, although its reasoning should be carefully analyzed for accuracy and clinical validity. As such, its usefulness in a clinical educational context is currently limited but rapidly evolving. Clinical relevanceChatGPT can access a multitude of medical data and may help provide accurate answers to clinical questions.

The Rapid Development of Artificial Intelligence: GPT-4's Performance on Orthopedic Surgery Board Questions.

Advances in artificial intelligence and machine learning models, like Chat Generative Pre-trained Transformer (ChatGPT), have occurred at a remarkably fast rate. OpenAI released its newest model of ChatGPT, GPT-4, in March 2023. It offers a wide range of medical applications. The model has demonstrated notable proficiency on many medical board examinations. This study sought to assess GPT-4's performance on the Orthopaedic In-Training Examination (OITE) used to prepare residents for the American Board of Orthopaedic Surgery (ABOS) Part I Examination. The data gathered from GPT-4's performance were additionally compared with the data of the previous iteration of ChatGPT, GPT-3.5, which was released 4 months before GPT-4. GPT-4 correctly answered 251 of the 396 attempted questions (63.4%), whereas GPT-3.5 correctly answered 46.3% of 410 attempted questions. GPT-4 was significantly more accurate than GPT-3.5 on orthopedic board-style questions (P<.00001). GPT-4's performance is most comparable to that of an average third-year orthopedic surgery resident, while GPT-3.5 performed below an average orthopedic intern. GPT-4's overall accuracy was just below the approximate threshold that indicates a likely pass on the ABOS Part I Examination. Our results demonstrate significant improvements in OpenAI's newest model, GPT-4. Future studies should assess potential clinical applications as AI models continue to be trained on larger data sets and offer more capabilities. [Orthopedics. 2024;47(2):e85-e89.].

Effect of a surgical skills month on intern performance in the orthopaedic in-training examination

Background: The orthopaedic in-training examination (OITE) is an annual national assessment of knowledge in orthopaedic surgery residents. Our primary aim was to determine if intern performance in the OITE (measured by correct answers) differed before and after the adoption of a formal surgical skills month (SSM) in the residency curriculum. A secondary aim was to evaluate the relationship between intern OITE and post-graduate year two (PGY-2) scores. Methods: A single institution retrospective cohort study was performed, comparing orthopaedic surgery interns who took the OITE before (15 residents) and after (16 residents) the year 2013 (when an SSM was introduced). De-identified OITE raw and percentage correct scores were obtained from all resident records. Statistical testing included independent sample t test and linear regression, with a significance level of 0.05. Results: Comparison of OITE percentage of correct answers overall, between interns before and after 2013, showed a significant increase after initiating the SSM (44.8%±4.13% vs. 49.9%±8.44%, P=0.0414). In addition, regression analysis demonstrated a positive linear relationship between intern and PGY-2 OITE scores after the SSM was implemented (R 2=0.380, P=0.011, β=0.424, CI: 0.1135341 –1.190913). Conclusion: Implementation of an SSM led to increased intern OITE scores. Moreover, intern OITE scores were more predictive of PGY-2 scores after SSM implementation. Earlier education (such as a surgical skills month) for orthopaedic surgery interns can aid knowledge acquisition and career development at the beginning of their training. Level of Evidence: Level III.

Can Artificial Intelligence Pass the American Board of Orthopaedic Surgery Examination? Orthopaedic Residents Versus ChatGPT.

Advances in neural networks, deep learning, and artificial intelligence (AI) have progressed recently. Previous deep learning AI has been structured around domain-specific areas that are trained on dataset-specific areas of interest that yield high accuracy and precision. A new AI model using large language models (LLM) and nonspecific domain areas, ChatGPT (OpenAI), has gained attention. Although AI has demonstrated proficiency in managing vast amounts of data, implementation of that knowledge remains a challenge. (1) What percentage of Orthopaedic In-Training Examination questions can a generative, pretrained transformer chatbot (ChatGPT) answer correctly? (2) How does that percentage compare with results achieved by orthopaedic residents of different levels, and if scoring lower than the 10th percentile relative to 5th-year residents is likely to correspond to a failing American Board of Orthopaedic Surgery score, is this LLM likely to pass the orthopaedic surgery written boards? (3) Does increasing question taxonomy affect the LLM's ability to select the correct answer choices? This study randomly selected 400 of 3840 publicly available questions based on the Orthopaedic In-Training Examination and compared the mean score with that of residents who took the test over a 5-year period. Questions with figures, diagrams, or charts were excluded, including five questions the LLM could not provide an answer for, resulting in 207 questions administered with raw score recorded. The LLM's answer results were compared with the Orthopaedic In-Training Examination ranking of orthopaedic surgery residents. Based on the findings of an earlier study, a pass-fail cutoff was set at the 10th percentile. Questions answered were then categorized based on the Buckwalter taxonomy of recall, which deals with increasingly complex levels of interpretation and application of knowledge; comparison was made of the LLM's performance across taxonomic levels and was analyzed using a chi-square test. ChatGPT selected the correct answer 47% (97 of 207) of the time, and 53% (110 of 207) of the time it answered incorrectly. Based on prior Orthopaedic In-Training Examination testing, the LLM scored in the 40th percentile for postgraduate year (PGY) 1s, the eighth percentile for PGY2s, and the first percentile for PGY3s, PGY4s, and PGY5s; based on the latter finding (and using a predefined cutoff of the 10th percentile of PGY5s as the threshold for a passing score), it seems unlikely that the LLM would pass the written board examination. The LLM's performance decreased as question taxonomy level increased (it answered 54% [54 of 101] of Tax 1 questions correctly, 51% [18 of 35] of Tax 2 questions correctly, and 34% [24 of 71] of Tax 3 questions correctly; p = 0.034). Although this general-domain LLM has a low likelihood of passing the orthopaedic surgery board examination, testing performance and knowledge are comparable to that of a first-year orthopaedic surgery resident. The LLM's ability to provide accurate answers declines with increasing question taxonomy and complexity, indicating a deficiency in implementing knowledge. Current AI appears to perform better at knowledge and interpretation-based inquires, and based on this study and other areas of opportunity, it may become an additional tool for orthopaedic learning and education.

Update on Educational Resources and Evaluation Tools for Orthopaedic Surgery Residents.

Innovations in orthopaedic resident educational resources and evaluation tools are essential to ensuring appropriate training and ultimately the graduation of competent orthopaedic surgeons. In recent years, there have been several advancements in comprehensive educational platforms within orthopaedic surgery. Orthobullets PASS, Journal of Bone and Joint Surgery Clinical Classroom, and American Academy of Orthopaedic Surgery Resident Orthopaedic Core Knowledge each have their own unique advantages in preparation for the Orthopaedic In-Training Examination and American Board of Orthopaedic Surgery board certification examinations. In addition, the Accreditation Council for Graduate Medical Education Milestones 2.0 and the American Board of Orthopaedic Surgery Knowledge Skills Behavior program each provide objective assessment of resident core competencies. Understanding and using these new platforms will help orthopaedic residents, faculty, residency programs, and program leadership to best train and evaluate their residents.

The Impact of Post-graduate Year and Program Accreditation Status on In-Training Examination Performance in Orthopaedic Surgery.

Introduction The progression of medical knowledge competency during surgical residency training is poorly understood. This study measures the acquisition of medical knowledge as orthopedic surgery residents advance during training and the impact of accreditation status on orthopedic in-training examination (OITE) performance. Methods Orthopedic surgery residents taking the OITE during 2020 and 2021 were included. Residents were grouped into cohorts by post-graduate year (PGY) and Accreditation Council for Graduate Medical Education (ACGME) accreditation status. Comparisons were made with parametric tests. Results Eight thousand eight hundred and seventy-one ACGME-accredited residents (89%) and 1,057 non-ACGME-accredited residents (11%) were evenly distributed by the PGY level (range, 19-21%). Residents in both ACGME- and non-ACGME-accredited residency programs had a significant increase in OITE performance at each PGY level (P<0.001). At ACGME-accredited programs, OITE performance increased from PGY1 (51%), PGY2 (59%), PGY3 (65%), PGY4 (68%), and PGY5 (70%) (P<0.001). There were progressively smaller percentage increases in OITE performance during accredited residency training (range, 2-8%), but this increase was linear in non-accredited residency training (range, 4%). At each PGY level, residents at accredited programs outperformed their counterparts at non-accredited programs (P<0.001). Conclusion OITE performance increases during residency training. Among ACGME-accredited residents, performance on the OITE progresses rapidly during junior years and plateaus during senior years. Residents in ACGME-accredited residency programs outperform their counterparts in non-accredited residency programs. More research is needed to understand optimal training environments that promote medical knowledge acquisition during orthopedic surgery residency.

Acquisition of Medical Knowledge Varies During Allopathic and Osteopathic Orthopedic Surgery Residency Training.

Currently, little is known about the differences in medical knowledge acquisition between osteopathic and allopathic orthopedic surgery residents. The purpose of this study was to determine the relationship between Orthopaedic In-Training Examination (OITE) performance and training pathway in orthopedic surgery. This was a retrospective cohort study of all orthopedic surgery residents taking the OITE during the 2019-2020 academic year. Comparisons in OITE performance were made with parametric tests. A total of 4407 orthopedic surgery residents were in allopathic (86%) and osteopathic (14%) training programs. There was significant improvement in OITE performance between subsequent postgraduate year (PGY) levels among allopathic residents (P<.001). Among osteopathic residents, OITE performance increased between PGY1 and PGY4 (P<.001) but plateaued between PGY4 and PGY5 (P>.05). At the PGY1 level, osteopathic residents had higher OITE performance than allopathic residents (P<.001), but scores were equivalent at the PGY2 to PGY4 levels (P>.05). At the PGY5 level, allopathic residents had higher OITE performance than osteopathic residents (P<.001). Allopathic medical students scored higher on the Step 1 (248±19 vs 242±17, P<.001) and Step 2 (255±16 vs 250±15, P<.001) board examinations. Medical knowledge increases during orthopedic surgery residency. Disparities exist by training pathway, with osteopathic residents outperforming allopathic residents at the PGY1 level but then underperforming at the PGY5 level. Ultimately, this study provides insights into how resident promotion and training pathway impacts the acquisition of medical knowledge during orthopedic surgery residency. [Orthopedics. 2023;46(6):379-383.].

Fifty Most-Cited Research Articles in Elbow Surgery: A Modern Reading List

Bibliometric analysis is a common method for evaluating current trends within a scientific field. The primary aim of this study was to define and analyze the 50 most frequently cited articles in the field of elbow surgery, both of all time and those published during the 21st century. We searched the Journal Citation Report to identify articles related to elbow surgery within academic journals. Articles were sorted by total citations. The overall top 50 articles and those published since 2000 were identified, and data were collected, including title, journal of publication, publication year, country of publication, citation density, level of evidence, article type, institution, and sex of the lead and senior authors, and inclusion on the reference list for the Orthopaedic In-Training Examination within the last 5 years. Descriptive statistics were reported, and correlation analysis was performed using Spearman test. For the most-cited elbow surgery articles, "fracture" was overall the most reported topic, whereas "lateral epicondylosis" and "fracture" were equal for those published since 2000. The United States was the most represented overall and for articles published since 2000. Women comprised 1/50 (2%) of lead authors overall, increasing to 8/50 (16%) for articles published during the 21st century. Most articles in during both periods contained level IV evidence, with level I evidence appearing infrequently (4%). Six percent of the most-cited articles of all time had appeared on the reference list of the Orthopaedic In-Training Examination within the past 5 years. The top 50 most-cited elbow surgery articles often assess fracture and lateral epicondylosis, most commonly originating from the United States. Level IV retrospective series comprises over half of the articles on this list. Women remain underrepresented as authors. This study provides a modern reading list for upper-extremity surgeons about impactful elbow surgery articles.

The Evolution of the Musculoskeletal Trauma Section of the Orthopaedic In-Training Examination

In 1963, the American Academy of Orthopaedic Surgeons administered the Orthopaedic In-Training Examination (OITE), the first and longest running yearly medical specialty examination. There have been no recent studies to evaluate the content of the musculoskeletal trauma section of the OITE. We analyzed all questions that were classified by the American Academy of Orthopaedic Surgeons as musculoskeletal trauma from 2012 to 2019. We recorded the number of musculoskeletal trauma questions in each examination, the topics and imaging modalities tested, the references cited, and the taxonomy classification of each question. We extrapolated from a similar musculoskeletal trauma study published in 2011 to create the previous examination cohort for comparison. For the current cohort, the average number of musculoskeletal trauma questions was 43.5 questions per examination (18.4%). The most frequently tested topics were proximal tibia fractures, pediatric trauma, hip fractures, and diaphyseal femur fractures, respectively. In previous examinations, questions from T1 and 2 were tested significantly more frequently compared with the current examinations (P < 0.001 and P = 0.02, respectively). In the current cohort, T3 questions were tested significantly more frequently than previous examinations (P = 0.001). Previous examinations had significantly more questions without an image (36 questions per year versus 25 questions per year, P < 0.001). In current versions of the examination, radiographs are tested significantly more frequently than other imaging modalities (P < 0.001). The musculoskeletal trauma section of the OITE has evolved. To improve and focus study efforts, residents may use this study as a guide when preparing for the examination.

Analysis of Orthopaedic In-Training Examination Trauma Questions: 2017 to 2021

The Orthopaedic In-Training Examination (OITE) is a multiple-choice examination developed by the American Academy of Orthopaedic Surgeons annually since 1963 to assess orthopaedic residents' knowledge. This study's purpose is to analyze the 2017 to 2021 OITE trauma questions to aid orthopaedic residents preparing for the examination. The 2017 to 2021 OITEs on American Academy of Orthopaedic Surgeons' ResStudy were retrospectively reviewed to identify trauma questions. Question topic, references, and images were analyzed. Two independent reviewers classified each question by taxonomy. Trauma represented 16.6% (204/1,229) of OITE questions. Forty-nine percent of trauma questions included images (100/204), 87.0% (87/100) of which contained radiographs. Each question averaged 2.4 references, of which 94.9% were peer-reviewed articles and 46.8% were published within 5 years of the respective OITE. The most common taxonomic classification was T1 (46.1%), followed by T3 (37.7%) and T2 (16.2%). Trauma represents a notable portion of the OITE. Prior OITE trauma analyses were published greater than 10 years ago. Since then, there has been an increase in questions with images and requiring higher cognitive processing. The Journal of Orthopaedic Trauma (24.7%), Journal of the American Academy of Orthopaedic Surgeons (10.1%), and Journal of Bone and Joint Surgery, American Volume (9.3%) remain the most cited sources.

Comparison of Sports Medicine Questions on the Orthopaedic In-Training Examination Between 2009 and 2012 and 2017 and 2020 Reveals an Increasing Number of References

To provide an updated analysis of the sports medicine section of the Orthopedic In-Training Examination (OITE). A cross-sectional review of OITE sports medicine questions from 2009 to 2012 and 2017-2020 was performed. Subtopics, taxonomy, references, and use of imaging modalities were recorded and changes between the time periods were analyzed. The most tested sports medicine subtopics included ACL (12.6%), rotator cuff (10.5%), and throwing injuries to the shoulder (7.4%) in the early subset, while ACL (10%), rotator cuff (6.25%), shoulder instability (6.25%), and throwing injuries to the elbow (6.25%) were the most common in the later subset. The American Journal of Sports Medicine (28.3%) was the most cited journal referenced from 2009 to 2012, while The Journal of the American Academy of Orthopaedic Surgeons (17.5%) was most referenced in questions from 2017 to 2020. The number of references per question increased from the early to the late subset (P < .001). There was a trend toward an increased taxonomy type one questions (P= .114), while type 2 questions had a decreased trend (P= .263) when comparing the new subset to the early group. When comparing sports medicine OITE questions from 2009 to 2012 and 2017 to 2020, there was an increase in the number of references per question. Subtopics, taxonomy, lag time, and use of imaging modalities did not show statistically significant changes. This study provides a detailed analysis of the sports medicine section of the OITE, which can be used by residents and program directors to direct their preparation for the annual examination. The results of this study may help examining boards align their examinations and provide a benchmark for future studies.

Allopathic and Osteopathic Residents Perform Similarly on the Orthopedic In-Training Examination (OITE)

There is a bias in the medical community that allopathic training is superior to osteopathic training, despite the lack of substantiation. The orthopedic in-training examination (OITE) is a yearly exam evaluating educational advancement and orthopedic surgery resident's scope of knowledge. The purpose of this study was to compare OITE scores between doctor of osteopathic medicine (DO) and medical doctor (MD) orthopedic surgery residents to determine whether any appreciable differences exist in the achievement levels between the 2 groups. The American Academy of Orthopedic Surgeons 2019 OITE technical report, which reports the scores from the 2019 OITE for MDs and DOs, was evaluated to determine OITE scores for MD and DO residents. The progression of scores obtained during various postgraduate years (PGY) for both groups was also analyzed. MD and DO scores throughout PGY 1-5 were compared with independent t-tests. PGY-1 DO residents outperformed MD residents on the OITE (145.8 vs 138.8, p < 0.001). The mean scores achieved by DO and MD residents during PGY-2 (153.2 vs 153.2), 3 (176.2 vs 175.2), and 4 (182.0 vs 183.7) did not differ (p = 0.997, 0.440, and 0.149, respectively). However, for PGY-5, the mean scores for MD residents (188.6) were higher than those of DO residents (183.5, p < 0.001). Both groups had trends of improvement seen throughout PGY 1 to 5 years, with both groups showing an increase in average PGY scores when compared to each preceding PGY. This study provides evidence that DO and MD orthopedic surgery residents perform similarly on the OITE within PGY 2 to 4, thus displaying equivalencies in orthopedic knowledge within the majority of PGYs. Program directors at allopathic and osteopathic orthopedic residency programs should take this into account when considering applicants for residency.

Analysis of Questions in Sections of the Orthopaedic In-Training Examination: A Scoping Review.

The purpose of this review was to assess all available studies that analyzed the types of questions in individual sections of the Orthopaedic In-Training Examination, which may be used as a reference for residents studying for their examination. Following the Providing Innovative Service Models and Assessment extension for Scoping Reviews guidelines, a systematic review was conducted on studies that report on sections or question categories of the Orthopaedic In-Training Examination using PubMed, MEDLINE, and Web of Science databases. Two reviewers and an arbitrator reviewed and extracted relevant data from 20 included studies which made up the systematic review. All 20 studies in the review reported the mean number of questions per section, with the highest coming from musculoskeletal trauma (18.9% to 19.0%). 18 studies reported the Buckwalter taxonomic classification; 42.0% of questions were T1, 18.2% were T2, and 39.5% were T3 with a wide range from section to section. Primary sources were nearly three times more likely to be cited when compared with textbook sources. There were 12 journals that were commonly cited with the most being the Journal of Bone and Joint Surgery: American Volume (17/18). This study accurately portrays the characteristics of each section of the Orthopaedic In-Training Examination over the past 10 years. These data suggest that orthopaedic residents may be inclined to focus on musculoskeletal trauma, topics related to clinical management, and primary journal sources for studying. In addition, residency programs may choose to focus on higher yield sources or material to prepare their residents for the examination.

Evaluating ChatGPT Performance on the Orthopaedic In-Training Examination.

Artificial intelligence (AI) holds potential in improving medical education and healthcare delivery. ChatGPT is a state-of-the-art natural language processing AI model which has shown impressive capabilities, scoring in the top percentiles on numerous standardized examinations, including the Uniform Bar Exam and Scholastic Aptitude Test. The goal of this study was to evaluate ChatGPT performance on the Orthopaedic In-Training Examination (OITE), an assessment of medical knowledge for orthopedic residents. OITE 2020, 2021, and 2022 questions without images were inputted into ChatGPT version 3.5 and version 4 (GPT-4) with zero prompting. The performance of ChatGPT was evaluated as a percentage of correct responses and compared with the national average of orthopedic surgery residents at each postgraduate year (PGY) level. ChatGPT was asked to provide a source for its answer, which was categorized as being a journal article, book, or website, and if the source could be verified. Impact factor for the journal cited was also recorded. ChatGPT answered 196 of 360 answers correctly (54.3%), corresponding to a PGY-1 level. ChatGPT cited a verifiable source in 47.2% of questions, with an average median journal impact factor of 5.4. GPT-4 answered 265 of 360 questions correctly (73.6%), corresponding to the average performance of a PGY-5 and exceeding the corresponding passing score for the American Board of Orthopaedic Surgery Part I Examination of 67%. GPT-4 cited a verifiable source in 87.9% of questions, with an average median journal impact factor of 5.2. ChatGPT performed above the average PGY-1 level and GPT-4 performed better than the average PGY-5 level, showing major improvement. Further investigation is needed to determine how successive versions of ChatGPT would perform and how to optimize this technology to improve medical education. AI has the potential to aid in medical education and healthcare delivery.

ChatGPT Performs at the Level of a Third-Year Orthopaedic Surgery Resident on the Orthopaedic In-Training Examination.

Publicly available AI language models such as ChatGPT have demonstrated utility in text generation and even problem-solving when provided with clear instructions. Amidst this transformative shift, the aim of this study is to assess ChatGPT's performance on the orthopaedic surgery in-training examination (OITE). All 213 OITE 2021 web-based questions were retrieved from the AAOS-ResStudy website (https://www.aaos.org/education/examinations/ResStudy). Two independent reviewers copied and pasted the questions and response options into ChatGPT Plus (version 4.0) and recorded the generated answers. All media-containing questions were flagged and carefully examined. Twelve OITE media-containing questions that relied purely on images (clinical pictures, radiographs, MRIs, CT scans) and could not be rationalized from the clinical presentation were excluded. Cohen's Kappa coefficient was used to examine the agreement of ChatGPT-generated responses between reviewers. Descriptive statistics were used to summarize the performance (% correct) of ChatGPT Plus. The 2021 norm table was used to compare ChatGPT Plus' performance on the OITE to national orthopaedic surgery residents in that same year. A total of 201 questions were evaluated by ChatGPT Plus. Excellent agreement was observed between raters for the 201 ChatGPT-generated responses, with a Cohen's Kappa coefficient of 0.947. 45.8% (92/201) were media-containing questions. ChatGPT had an average overall score of 61.2% (123/201). Its score was 64.2% (70/109) on non-media questions. When compared to the performance of all national orthopaedic surgery residents in 2021, ChatGPT Plus performed at the level of an average PGY3. ChatGPT Plus is able to pass the OITE with an overall score of 61.2%, ranking at the level of a third-year orthopaedic surgery resident. It provided logical reasoning and justifications that may help residents improve their understanding of OITE cases and general orthopaedic principles. Further studies are still needed to examine their efficacy and impact on long-term learning and OITE/ABOS performance.