Abstract
To evaluate whether scientific abstracts selected for podium presentation at the Orthopaedic Trauma Association (OTA) Annual Meeting differ based on the program committee size and/or the proportion of abstracts each committee member evaluates. Abstract scores from the Orthopaedic Trauma Association program committee from 2010 through 2016 were obtained. All members (range, 8-9) reviewed each clinical abstract (range, 506-778) each year in a blinded fashion. The 90 top-scoring abstracts were considered "accepted" for this study. To determine the effect of reducing the committee size, all possible combinations of reviewers for each possible committee size were modeled. To determine the effect of reducing the number of abstracts each member reviewed, we used Monte Carlo simulation with 100 cycles to generate possible combinations of 1-9 reviewers for each abstract. Mean percent agreement with the actual selection was the primary outcome. The mean percent agreement progressively declined from 90.2% with 1 less committee member to 56.7% with only a single reviewer. For each reduction in the number of committee members, 4.4% agreement was lost. If all committee members were retained but the number of reviewers per abstract was reduced from 8 to 1, the mean percent agreement declined from 88.8% to 43.0%. Each reduction in reviewers per abstract reduced the mean percent agreement 6.3%. The findings inform program committees striving to balance the trade-off between an acceptable reduction in agreement, given a reduction in the program committee size or the proportion of abstracts each committee member evaluates.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.