How Has Statistical Testing in Orthopedics Changed Over Time? An Assessment of High Impact Journals Over 25 Years

Abstract

It is important for physicians to be familiar with statistical techniques commonly used in published medical research. Statistical errors in medical literature are common, and there is a reported lack of understanding regarding statistical knowledge necessary for data interpretation and journal reading. As study design has become increasingly complex, peer-reviewed literature poorly addresses and explains the most common statistical methods utilized across leading orthopedic journals. Articles from 5 leading general and subspecialty orthopedic journals were compiled from 3 distinct time periods. After exclusions were applied, 9521 remained, and a random 5% sampling of these articles, balanced across journals and years, was conducted yielding 437 articles after additional exclusions. Information regarding the number of statistical tests used, power/sample size calculation, type of statistical tests used, level of evidence (LOE), study type, and study design was collected. The mean number of statistical tests across all 5 orthopedic journals increased from 1.39 to 2.29 by 2018 (p = 0.007). The percentage of articles that reported power/sample size analyses was not found to differ by year, but the value has increased from 2.6% in 1994 to 21.6% in 2018 (p = 0.081). The most commonly used statistical test was the t-test which was present in 20.5% of articles, followed by chi-square test (13%), Mann-Whitney analysis (12.6%) and analysis of variance (ANOVA, 9.6%). The mean number of tests was generally greater in articles from higher impact factor journals (p = 0.013). Studies with a LOE of I used the highest mean number of statistical tests (3.23) compared to studies with lower LOE ratings (range 1.66-2.69, p < 0.001). Randomized control trials used the highest mean number of statistical test (3.31), while case series used the lowest mean number of tests (1.57, p < 0.001). The mean number of statistical tests used per article has increased over the past 25 years with the t-test, chi-square test, Mann-Whitney analysis, and ANOVA being the most used statistical tests in leading orthopedic journals. Despite an increase in statistical tests it should be noted that there was a paucity in advance statistical testing within the orthopedic literature. This study displays important trends in data analysis and can serve as a guide to help clinicians and trainees better understand the statistics used in literature as well as identifying deficits within the literature that should be addressed to help progress the field of orthopedics.