Paper 35: The Sport Publication Observational Research Tool (SPORT): An Objective Tool to Score the Methodological Quality of Observational Clinical Sports Medicine Research
Objectives:There has been a significant increase in the number of systematic reviews and meta-analyses published within the sports medicine literature over the last decade. An important aspect of conducting a systematic review or meta-analysis is the evaluation of the methodologic quality and bias of the studies included in these reviews. Several methodologic quality assessment tools have been developed for the evaluation of primary scientific research; however, these tools may not be relevant or specific to observational sports medicine research, a discipline that carries numerous unique nuances and biases. The objective of this effort was to develop a tool – the “Sport Publication Observational Research Tool (SPORT)” – to assess and score the methodological quality of observational sports medicine research.Methods:The SPORT Score was developed through a modified Delphi process. The Delphi panel included members from the Herodicus Society and The FORUM – two collegial groups of orthopaedic sports medicine experts from across the world. All active members of the Herodicus Society (n=117) and The Forum (n=80) were invited to participate. Four rounds of questions were developed to build consensus regarding the content and scoring system of the novel tool. Based on an a priori power analysis, 55 randomly chosen observational clinical sports medicine research studies were selected by an independent librarian and scored twice by 4 reviewers of varying levels of training. Two-way random-effects intraclass correlation coefficients (ICC) were calculated for interrater reliability (using average measures for agreement) and intrarater reliability (using single measures for consistency) for SPORT subscores and total score among reviewers. The distribution and percentiles for total SPORT Score across the 55 studies were assessed. The newly developed SPORT Score was also compared to another commonly utilized methodologic quality score, the methodologic index for non-randomized studies (MINORS).Results:Forty (40/117; 34.2%) Herodicus Society members and 11 (11/80; 13.8%) FORUM members agreed to participate in the Delphi process. One participant dropped out after Round 2. Therefore, the completion rate for Rounds 1 through 4 were 100%, 100%, 98.0%, and 98.0%, respectively. The majority (n=47; 94%) of participants completed a sports medicine fellowship and were still practicing full-time (n=43; 86%) in academic centers (n=35; 70%). The vast majority had experience taking care of collegiate (n=47; 94%) and professional (n=39; 78%) athletes. Almost all (n=49; 98%) served as reviewers for journals and more than half (n=32; 64%) served on an editorial board. Half (n=25; 50%) authored over 100 peer-reviewed publications. Additionally, nearly half (n=23; 46%) received National Institutes of Health (NIH) funding and almost all (n=47; 94%) received other non-NIH funding for research. In the final round, The SPORT Score received 94% consensus approval. There were 4 major categories comprised of 19 subscores that included factors related to bias and methodologic quality such as “study design”, “sampling methods”, “selection criteria”, “power analysis”, “generalizability”, etc.Mean time to complete a SPORT score for each study was 6 minutes and 19 seconds (379.4 ± 173.4 seconds). However, time to completion varied significantly by level of training in that the “medical student” reviewer was significantly slower (9 minutes and 33 seconds [573.2 ± 188.4 seconds]) than the “junior resident” (5 minutes and 8 seconds [308.2 ± 107.9 seconds]), “chief resident” (4 minutes and 27 seconds [266.4 ± 72.2 seconds]), and “fellow/early career attending” (6 minutes and 8 seconds [368.1 ± 111.1 seconds]) at scoring studies. [F(3,213) = 62.7; p<0.001] (Figure 1).One subscore, regarding “peer review”, demonstrated unacceptable interrater reliability and was thus removed. The remaining 18 subscores had ICC ranges of 0.599-0.955 for interrater and 0.530-1.000 for intrarater reliability. Total SPORT Score ICC was 0.967 for interrater reliability and 0.966 for intrarater reliability, demonstrating almost perfect agreement for both. The final SPORT is presented in Table 1. (Table 1) The median total SPORT score across the 55 studies was 20.0 (IQR 5.5; range: 7.5 to 26.5). Scores followed a skewed distribution towards lower scores (W=0.924; p=0.002; N=55) (Figure 2). Percentiles and suggested scoring categories are shown in Table 2. (Table 2) There was a moderate, significant correlation between SPORT Scores and MINORS for the 55 studies evaluated [r(53) = 0.575, p <0.001].Conclusions:An objective tool to assess the methodological quality of observational clinical sports medicine research was successfully developed through a Delphi approach with numerous field experts through 4 rounds of questions. The tool demonstrated near perfect interrater and intrarater agreement and there was moderate correlation with a previously developed quality assessment tool. This tool can help researchers assess the quality of observational clinical sports medicine research studies, particularly those included in systematic reviews and meta-analyses.
- Research Article
48
- 10.1177/03635465221131281
- Dec 1, 2022
- The American Journal of Sports Medicine
Background: Female sports participation continues to rise; however, inequalities between male and female athletes still exist in many areas and may extend into medical research. Purpose: The purpose of this study was to (1) compare the number of published studies evaluating male versus female athletes in various sports and (2) identify which co-ed sports currently underrepresent female athletes in the sports medicine literature. Study Design: Systematic review; Level of evidence, 4. Methods: All nonreview research studies published from 2017 to 2021 in 6 top sports medicine journals were considered for inclusion. Sports medicine studies were included that isolated athletes, reported study outcomes specific to male and/or female patients, provided study outcomes for specific sports, and evaluated ≤3 different sports. The total number of studies reporting on male and/or female athletes were compared for all sports, and odds ratios (ORs) were calculated. Comparisons of study design, level of sports participation, outcomes assessed, and study quality were also made according to participant sex. Results: Overall, 669 studies were included the systematic review. Most studies isolated male athletes (70.7%), while 8.8% isolated female athletes and 20.5% included male and female athletes. Female athletes were more frequently studied in softball and volleyball, while male athletes were more commonly researched in baseball, soccer, American football, basketball, rugby, hockey, and Australian football. Notably, male athletes were largely favored in baseball/softball (91% vs 5%; OR = 18.2), rugby (72% vs 5%; OR = 14.4), soccer (65% vs 15%; OR = 4.3), and basketball (58% vs 18%; OR = 3.2). Conclusion: Sports medicine research has favored the evaluation of male athletes in most sports, including the majority of co-ed sports. Potential reasons for this inequality of research evaluation include availability of public data and database data, financial and promotional incentives, a high percentage of male sports medicine clinicians and researchers, and sex biases in sport. While the causes of these differences are multifaceted, researchers should consider both sexes for study inclusion whenever possible, and journals should support a more balanced representation of research publications regarding male and female athletes.
- Research Article
482
- 10.1152/jappl.1971.31.2.203
- Aug 1, 1971
- Journal of Applied Physiology
Diet, muscle glycogen, and endurance performance.
- Research Article
4
- 10.1177/03635465221128909
- Nov 23, 2022
- The American Journal of Sports Medicine
Sex differences in sports medicine are well documented. However, no studies to date have reviewed the rate at which sex is reported and analyzed in the athlete-specific orthopaedic sports medicine literature. To determine the rates of reporting and analyzing patient sex in athlete-specific sports medicine literature. Systematic review; Level of evidence, 4. Articles published by the 3 journals of the AOSSM (American Journal of Sports Medicine [AJSM], Orthopaedic Journal of Sports Medicine, and Sports Health: A Multidisciplinary Approach) between 2017 and 2021 were considered for inclusion. Original sports medicine research studies that isolated athletes were included. Studies that isolated sports that are predominantly single sex at the college and/or professional levels (football, baseball, softball, and wrestling) were excluded. Of the 5140 publications screened, 559 met the inclusion criteria. In total, 93.9% of all studies reported patient sex, and 34.7% of all studies analyzed patient sex. However, 143 studies only included males and 50 studies only included females (n = 193). When excluding these single-sex studies, analysis of the remaining 366 studies found that the rate of sex-specific analysis increased to 53.0%. Rates of reporting patient sex did not significantly differ by journal or by year. Similarly, rates of analyzing patient sex did not differ by year, but Sports Health analyzed sex the most frequently, and AJSM analyzed sex the least frequently (P = .002). Studies that isolated college (84.1%), youth (66.7%), or recreational (52.6%) athletes analyzed sex at or above the overall rate of 53.0%, but studies of elite athletes (35.7%) tended to analyze sex less frequently. Patient sex is well reported in the athlete-specific sports medicine literature (93.9% of included studies reported sex), demonstrating that most studies include sex as a demographic variable. However, patient sex was analyzed only in 53.0% of studies that included both male and female patients. Given that athlete-specific sex differences are known to exist within the field of sports medicine, many studies that could benefit from using patient sex as a variable for analysis likely fail to do so.
- Research Article
1
- 10.1177/23259671241261679
- Oct 1, 2024
- Orthopaedic journal of sports medicine
Race- and ethnicity-based differences in treatment access and outcomes have been reported in the orthopaedic sports medicine literature. However, the rate at which race and ethnicity are reported and incorporated into the statistical analysis of sports medicine studies remains unclear. To determine the rate at which race and ethnicity are reported and analyzed in athlete-specific sports medicine literature. Systematic review; Level of evidence, 4. Using PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, articles published by the 3 journals of the American Orthopaedic Society for Sports Medicine between 2017 and 2021 were considered for inclusion. Original sports medicine research studies that focused on athletes were included. Outcome measures included reporting and analysis of patient demographics (age, sex, race, ethnicity). Studies that included demographic variables in a multivariate analysis or that performed a race-/ethnicity-based stratified analyses were considered to have analyzed that variable. Studies that reported and/or analyzed patient demographics were examined. Chi-square tests were performed to determine statistical significance. A total of 5140 publications were screened, and 842 met the inclusion criteria. Age and sex were well reported (84.1% and 87.0%, respectively), while race (3.8%) and ethnicity (2.0%) were poorly reported. There was no difference in rates of reporting age, sex, race, or ethnicity between the American Journal of Sports Medicine (AJSM), the Orthopaedic Journal of Sports Medicine (OJSM), or Sports Health: A Multidisciplinary Approach (Sports Health). The rate of analysis was also calculated as a percentage of the studies that reported that variable. Of the studies that reported age, 38.5% analyzed age. Using this method, 26.2% of studies analyzed sex, 40.6% analyzed race, and 17.6% analyzed ethnicity. Although there was no difference in the overall rate at which studies from the 3 journals analyzed ethnicity, Sports Health studies analyzed age (P = .044), sex (P = .001), and race (P = .027) more frequently than studies published in AJSM and OJSM. Of the studies that analyzed race, most of those studies (8/13, 61.5%) found significant race-based differences in reported outcomes. This systematic review demonstrated that race and ethnicity are poorly reported and analyzed in athlete-specific sports medicine literature, despite the fact that a majority of studies analyzing race found significant differences between racial groups. Improved reporting of race and ethnicity can determine whether race- and ethnicity-based differences exist in patient interventions to ameliorate disparities in patient outcomes.
- Research Article
53
- 10.5435/jaaos-d-17-00636
- Apr 1, 2019
- Journal of the American Academy of Orthopaedic Surgeons
Comparative trials evaluating categorical outcomes have important implications on surgical decision making. The purpose of this study was to examine the statistical stability of sports medicine research. Comparative clinical sports medicine research studies involving anterior cruciate ligament, meniscus, and knee instability were reviewed in two journals between 2006 and 2016. The statistical stability for each study outcome was determined by the number of event reversals required to change the P value to either greater or less than 0.05. The number of patients lost to follow-up was also determined. Of the 1,505 studies screened, 102 studies were included for analysis, 40 of which were randomized controlled trials. There were 339 total outcome events, with 98 significant and 241 not significant. The Fragility Index, or the median number of events required to change the statistical significance of the overall study, was five (interquartile range, 3 to 8) or 5.4% of the total study population. In addition, the average number of patients lost to follow-up was 7.9, which is greater than the number needed to change the significance of each study arm and the entire study population. Results in the comparative sports medicine literature may not be as stable as previously thought, with only a small percentage of outcome events needed to change study significance. Outcomes research based on a single discreet P value cutoff may be misleading.
- Abstract
- 10.1177/2325967123s00347
- Jul 1, 2023
- Orthopaedic Journal of Sports Medicine
Objectives:Female sport participation has steadily increased over the past several decades; however, inequalities still exist regarding participation rates, social norms, and available resources. It is possible that inequalities between male and female athletes extend beyond the performance of sport and into medical research. Therefore, the purposes of this systematic review were to 1) compare the number of published studies evaluating male vs. female athletes in various sports, and 2) identify which co-ed sports currently under-represent female athletes in the sports medicine literature.Methods:All non-review research studies published from 2017-2021 in six top sports medicine journals were considered for inclusion. Only sports medicine studies that isolated athletes, reported study outcomes specific to male and/or female patients, provided study outcomes for specific sport(s), and evaluated three or fewer different sports, were included. The total number of studies reporting on male and/or female athletes were compared for all sports, and odds ratios (OR) were calculated. Comparisons of study design, level of sport participation, outcomes assessed, and study quality were also made based on subject sex.Results:Overall, 669 studies were included the systematic review. Most of the included studies isolated male athletes (70.7%), while 8.8% isolated female athletes and 20.5% included both male and female athletes. Female athletes were more frequently studied in softball and volleyball, while male athletes were more commonly researched in baseball, soccer, American football, basketball, rugby, hockey, and Australian football. Notably, male athletes were largely favored in baseball/softball (91% vs. 5%, OR=18.2), rugby (72% vs. 5%, OR=14.4), soccer (65% vs. 15%, OR=4.3), and basketball (58% vs. 18%, OR=3.2).Conclusions:Sports medicine research has favored the evaluation of male athletes in most sports, including the majority of co-ed sports. Potential reasons for this inequality of research evaluation include availability of public and database data, financial and promotional incentive, a high percentage of sports medicine clinicians and researchers being male, and sex biases in sport. While the causes of these differences are multi-faceted, researchers should consider both sexes for study inclusion whenever possible and journals should support a more balanced representation of research publications regarding male and female athletes.
- Research Article
1668
- 10.1152/jappl.1955.8.1.73
- Jul 1, 1955
- Journal of Applied Physiology
Maximal oxygen intake as an objective measure of cardio-respiratory performance.
- Research Article
22
- 10.1177/0363546515580290
- Apr 21, 2015
- The American journal of sports medicine
Background: The number of systematic reviews published in the orthopaedic literature has increased, and these reviews can help guide clinical decision making. However, the quality of these reviews can affect the reader’s ability to use the data to arrive at accurate conclusions and make clinical decisions. Purpose: To evaluate the methodological and reporting quality of systematic reviews and meta-analyses in the sports medicine literature to determine whether such reviews should be used to guide treatment decisions. The hypothesis was that many systematic reviews in the orthopaedic sports medicine literature may not follow the appropriate reporting guidelines or methodological criteria recommended for systematic reviews. Study Design: Systematic review. Methods: All clinical sports medicine systematic reviews and meta-analyses from 2009 to 2013 published in The American Journal of Sports Medicine (AJSM), The Journal of Bone and Joint Surgery (JBJS), Arthroscopy, Sports Health, and Knee Surgery, Sports Traumatology, Arthroscopy (KSSTA) were reviewed and evaluated for level of evidence according to the guidelines from the Oxford Centre for Evidence-Based Medicine, for reporting quality according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement, and for methodological quality according to the Assessment of Multiple Systematic Reviews (AMSTAR) tool. Analysis was performed by year and journal of publication, and the levels of evidence included in the systematic reviews were also analyzed. Results: A total of 200 systematic reviews and meta-analyses were identified over the study period. Of these, 53% included evidence levels 4 and 5 in their analyses, with just 32% including evidence levels 1 and 2 only. There were significant differences in the proportion of articles with high levels of evidence (P < .001) and low levels of evidence (P = .005) by journal. The average PRISMA score was 87% and the average AMSTAR score was 73% among all journals. The average AMSTAR and PRISMA scores were significantly different by journal (P = .002 and .001, respectively) and by year (P = .046 and .019, respectively). Arthroscopy, AJSM, and JBJS all scored higher than Sports Health and KSSTA on the PRISMA and AMSTAR. The average PRISMA score by year varied from 85% to 89%, and the average AMSTAR score varied from 70% to 76%. Conclusion: Systematic reviews and meta-analyses in orthopaedics sports medicine literature relied on evidence levels 4 and 5 in 53% of studies over the 5-year study period. Overall, PRISMA and AMSTAR scores are high and may be better than those in other disciplines. Readers need to be conscious of potential shortcomings when reading systematic reviews and using them in practice.
- Research Article
10
- 10.1016/j.asmr.2022.08.001
- Sep 6, 2022
- Arthroscopy, Sports Medicine, and Rehabilitation
New Opportunities in Assessing Return to Performance in the Elite Athlete: Unifying Sports Medicine, Data Analytics, and Sports Science
- Supplementary Content
7
- 10.1136/bjsm.2003.010553
- Sep 23, 2004
- British Journal of Sports Medicine
Objective: To examine the evidence base of sports medicine research and assess how relevant and applicable it is to everyday practice. Methods: Original research articles, short reports, and case reports...
- Research Article
1
- 10.1016/j.jclinepi.2024.111609
- Nov 12, 2024
- Journal of Clinical Epidemiology
Methodological quality assessment tools for diagnosis and prognosis research: overview and guidance
- Abstract
- 10.1177/2325967119s00069
- Mar 1, 2019
- Orthopaedic Journal of Sports Medicine
BACKGROUND: Variability that exists amongst the radiographic measurement parameters associated with tibial spine fractures may have direct consequences when comparing, reporting, or treating these injuries. In developing data collection of tibial spine fractures amongst multiple centers, it is important to establish reliability in radiographic parameters. Therefore, we designed a study to validate the classification and a proposed cohort of measurements of tibial spine fractures amongst multiple institutions to assist with standardizing fracture classification and treatment decisions. METHODS: Radiographic assessment of de-identified acute tibial spine fractures was performed by members of the Pediatric Research in Sports Medicine (PRISM) Tibial Spine Research Interest Group. A descriptive Powerpoint presentation was provided to each reviewer demonstrating specific measurements and classification prior to review. Reviewers were also asked to provide treatment recommendations. DICOM files were provided to the surgeon through a web-based shared drive and reviewers were required to use the same imaging software. There were 40 patients included, determined through power analysis performed based on previous reliability studies and the number of participants. Assuming the intraclass correlation coefficient (ICC) will be .85 and 95% confidence interval to be 0.2, the sample size of 40 will achieve the desired 95% confidence. Data will be reviewed using both kappa and ICC reliability measures due to both categorical and continuous data points. RESULTS: A majority of radiographic measures demonstrated moderate ICC including posterior-proximal displacement (0.378), length and height of tibial spine fracture (0.466 and 0.535, respectively), and superior displacement of medial fragment (0.420). Good ICC was seen with superior displacement of the anterior tibial spine fragment (0.734). Poor correlation was seen with the measurements for anterior displacement, posterior sagittal displacement, and roof inclination angle. Classifying tibial spine fractures according to the historical Meyer & McKeever Classification demonstrated fair agreement (kappa = 0.347). 18 of 40 (45%) fracture patterns were classified by reviewers in three or more different classifications types while only 1 fracture pattern (Type 1) was agreed upon by all reviewers. A majority of reviewers recommended arthroscopic treatment with suture for more fracture patterns. However, there was fair agreement with the initial treatment regarding operative versus closed reduction (kappa = 0.328). CONCLUSION: Measurement of superior displacement of anterior tibial spine fracture on the lateral images is the only radiographic assessment with good correlation or agreement amongst a group of surgeons in a Tibial Spine Research Group. Classification of tibial spine fractures did not demonstrate acceptable agreement. Further studies and classification methodology is needed to standardize fracture patterns and thereby study outcomes based on pattern and treatment.
- Research Article
12
- 10.3928/01477447-20120822-30
- Sep 1, 2012
- Orthopedics
Animal models are commonly used for translational research despite evidence that the methodology of these studies is often inconsistent and substandard. This study describes the characteristics and impact of published research using animal models in the American Journal of Sports Medicine (AJSM). Peer-reviewed articles published in the AJSM between January 1990 and January 2010 using animal models were identified using MEDLINE. The articles were reviewed for funding source, anesthesia used, animal used, study type, study location, outcome measures, number of animals, duration of animal survival, main topic being studied, and positive or negative treatment effect. The impact factor of the studies published between 2005 and 2010 was calculated. Two hundred fifty-seven articles, or 6% (257/4278) of the total publications during the 20-year period, were analyzed. The impact factor increased from 1.83 in 2005 to 3.9 in 2010. The most common animals used were rabbits (24%) and pigs (16%). The anterior cruciate ligament was studied in 34% of the articles, and a pig model was used for 31% of these studies. Eighty-six percent of the studies had a positive treatment effect. This study shows that animal models used in sports medicine research lack uniformity in their methods and suggests that a publication bias may exist for animal research in the sports medicine literature.
- Research Article
217
- 10.1152/jappl.1967.23.2.259
- Aug 1, 1967
- Journal of Applied Physiology
Maximal performance at altitude and on return from altitude in conditioned runners.
- Research Article
17
- 10.61838/kman.intjssh.7.2.4
- Jan 1, 2024
- International Journal of Sport Studies for Health
Objective: Research in sports medicine and exercise science has experienced significant growth over recent years. With this expansion, there has been a concomitant rise in ethical challenges specific to these disciplines. While various ethical guidelines exist for numerous scientific fields, a comprehensive set tailored specifically for sports medicine and exercise science is lacking. Aiming to bridge this gap, this paper proposes a comprehensive, updated set of ethical guidelines specifically targeted at researchers in sports medicine and exercise science, providing them with a thorough framework to ensure research integrity. Methods: A collaborative approach was adopted, involving contributions from a diverse group of international experts in the field. A thorough review of existing ethical guidelines was conducted, followed by the identification and detailed examination of 15 specific ethical topics relevant to the discipline. Each topic was discussed in terms of its definition, consequences, and preventive measures. Results: The research in sports medicine and exercise science has grown significantly, bringing to the fore ethical challenges unique to these disciplines. Our comprehensive review identifies 15 key ethical challenges: plagiarism, data falsification, role of artificial intelligence chatbots in academic writing, overstating results, excessive/strategic self-citation, duplicate publications, non-disclosure of conflicts of interest, image manipulation, misuse of peer review, ghost and gift authorship, inadequate data retention, data fabrication, falsification of IRB approvals, lack of informed consent, and unethical human or animal experimentation. For each identified challenge, we propose practical solutions and best practices, enriched by the diverse perspectives of our collaborative international expert panel. This endeavor aims to offer a foundational set of ethical guidelines tailored to the nuanced needs of sports medicine and exercise science, ensuring research integrity and promoting ethical responsibility across these vital fields. Conclusion: This article represents a seminal contribution to the establishment of essential ethical guidelines specifically designed for the fields of sports medicine and exercise science. This article charts a clear course for researchers, clinicians, and policymakers by integrating these ethical principles at the heart of our scholarly and clinical activities. Consequently, it envisions a future where the principles of research integrity and ethical responsibility consistently inform every scientific discovery and every clinical engagement.
- Research Article
- 10.1177/23259671251369017
- Nov 1, 2025
- Orthopaedic Journal of Sports Medicine
- Discussion
- 10.1177/23259671251374875
- Oct 31, 2025
- Orthopaedic Journal of Sports Medicine
- Discussion
- 10.1177/23259671251384868
- Oct 31, 2025
- Orthopaedic Journal of Sports Medicine
- Discussion
- 10.1177/23259671251384864
- Oct 31, 2025
- Orthopaedic Journal of Sports Medicine
- Research Article
- 10.1177/23259671251387337
- Oct 30, 2025
- Orthopaedic Journal of Sports Medicine
- Research Article
- 10.1177/23259671251380865
- Oct 23, 2025
- Orthopaedic Journal of Sports Medicine
- Research Article
- 10.1177/23259671251385117
- Oct 22, 2025
- Orthopaedic Journal of Sports Medicine
- Research Article
- 10.1177/23259671251383147
- Oct 22, 2025
- Orthopaedic Journal of Sports Medicine
- Research Article
- 10.1177/23259671251378751
- Oct 22, 2025
- Orthopaedic Journal of Sports Medicine
- Research Article
- 10.1177/23259671251385115
- Oct 20, 2025
- Orthopaedic Journal of Sports Medicine
- Ask R Discovery
- Chat PDF
AI summaries and top papers from 250M+ research sources.