Potential effects of lowering the threshold of statistical significance in the field of chronic rhinosinusitis – A meta-research on published randomized controlled trials over last decade

Abstract

ObjectiveTo evaluate the impact of change in p-value threshold from 0.05 to 0.005, on published Randomized Controlled Trials (RCTs) over the last ten years in the field of chronic rhinosinusitis. MethodsA search of the PubMed database from 1st January 2011 to 31st December 2020 was conducted to include all RCTs that used p-value to determine the effects of an intervention. Data extracted included p-values, type of intervention, publishing journal with indexing, registration, funding, and multi- or single center status. The proportion of primary endpoints having p < 0.005 was determined first, followed by those with p > 0.005 but <0.05. Logistic regression analysis was used to determine if any trial characteristic was associated with reporting of significant p-values. ResultsIn total, 168 primary endpoints were identified from 123 RCTs. On analysis, 80 had a p-value <0.05, i.e., statistically significant on conventional parameter. Out of these, 53.75% had a p-value <0.005, which would retain significance under the proposed threshold, and the remaining 46.25% to be reclassified as “suggestive”. None of the trial characteristics were contributing to reporting of conventional or proposed p-values on logistic regression analysis. ConclusionLowering the p-value threshold would render 46.25% of a decade of published RCTs results (in the field of CRS) to be reclassified as merely “suggestive” and not significant. Trial characteristics were not found contributing to reporting of p-value <0.005 or even <0.05. Level of evidenceII.