MP44-13 A SECOND-LOOK AT REPORTED STATISTICS: CHALLENGES IN REPLICATING REPORTED P-VALUES IN PEDIATRIC UROLOGY LITERATURE

Abstract

You have accessJournal of UrologyPediatric Urology II (MP44)1 Sep 2021MP44-13 A SECOND-LOOK AT REPORTED STATISTICS: CHALLENGES IN REPLICATING REPORTED P-VALUES IN PEDIATRIC UROLOGY LITERATURE Erik Drysdale, Lauren Erdman, Mandy Rickard, Marta Skreta, Jin Kyu Kim, Daniel T. Keefe, Joana Dos Santos, Michael Chua, and Armando J. Lorenzo Erik DrysdaleErik Drysdale More articles by this author , Lauren ErdmanLauren Erdman More articles by this author , Mandy RickardMandy Rickard More articles by this author , Marta SkretaMarta Skreta More articles by this author , Jin Kyu KimJin Kyu Kim More articles by this author , Daniel T. KeefeDaniel T. Keefe More articles by this author , Joana Dos SantosJoana Dos Santos More articles by this author , Michael ChuaMichael Chua More articles by this author , and Armando J. LorenzoArmando J. Lorenzo More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000002065.13AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: p-values are the accepted metric for reporting statistically significant findings in science. Ensuring that p-values are robust, reproducible, and generated from the proper tests is essential for ensuring accurate scientific inference. We explored this issue by attempting to verify reported p-values from binary outcome tests and explore the reproducibility of p-value reporting in the hydronephrosis (HN) and bowel and bladder dysfunction (BBD) literature. METHODS: We analyzed 254 papers that were included in previously reported projects conducted regarding HN (n=129) and BBD (n=125). All studies reviewed had at least 2 groups being compared and reported a significant difference. We extracted group counts for the two primary groups from the sampled papers and calculated the Fisher’s exact test p-value. RESULTS: We found that 38 of the 254 (BBD=15, HN=23, 15%) papers sampled had insignificant p-values using our approach, but were reported as significant at <5%. Of these 38 papers, 16 had results that had results that could not be found significant through any attempted approach, 12 could achieve significance through a chi-square test without a Yates continuity correction, and the remaining 10 could be accounted for by differing methodologies (e.g. more than two groups or different test procedures). CONCLUSIONS: A meaningful number studies from pediatric urology appear to report p-values that are difficult to reproduce. A combination of reproducible code, sensitivity analyses, as well as arms-length reproduction attempts can help to reduce non-robust results. Source of Funding: none © 2021 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 206Issue Supplement 3September 2021Page: e797-e798 Advertisement Copyright & Permissions© 2021 by American Urological Association Education and Research, Inc.MetricsAuthor Information Erik Drysdale More articles by this author Lauren Erdman More articles by this author Mandy Rickard More articles by this author Marta Skreta More articles by this author Jin Kyu Kim More articles by this author Daniel T. Keefe More articles by this author Joana Dos Santos More articles by this author Michael Chua More articles by this author Armando J. Lorenzo More articles by this author Expand All Advertisement Loading ...