Abstract
ObjectivesTo assess the language used by systematic review authors to emphasize that statistically nonsignificant results show meaningful differences. To determine whether the magnitude of these treatment effects was distinct from nonsignificant results that authors interpreted as not different. Study Design and SettingWe screened Cochrane reviews published between 2017 and 2022 for statistically nonsignificant effect estimates that authors presented as meaningful differences. We classified interpretations qualitatively and assessed them quantitatively by calculating the areas under the curve of the portions of confidence intervals exceeding the null or a minimal important difference, indicating one intervention's greater effect. ResultsIn 2,337 reviews, we detected 139 cases where authors emphasized meaningful differences in nonsignificant results. Authors commonly used qualifying words to express uncertainty (66.9%). Sometimes (26.6%), they made absolute claims about one intervention's greater benefit or harm without acknowledging statistical uncertainty. The areas under the curve analyses indicated that some authors may overstate the importance of nonsignificant differences, whereas others may overlook meaningful differences in nonsignificant effect estimates. ConclusionNuanced interpretations of statistically nonsignificant results were rare in Cochrane reviews. Our study highlights the need for a more nuanced approach by systematic review authors when interpreting statistically nonsignificant effect estimates.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.