Common Statistical Errors in Scientific Investigations: A Simple Guide to Avoid Unfounded Decisions.

Abstract

During my experience as an author, peer reviewer, and editor during COVID-19, I have encountered - and committed - various errors related to the interpretation and use of statistical measures and tests. Primarily concerning health sciences such as epidemiology, infodemiology, and public health, the evidence used to inform a conclusion carries an extremely high weight as it translates into decisions made to preserve the population's well-being. Therefore, the aforementioned evidence must be reliable. This short guide discusses the most common and dangerous mistakes I have experienced during my scientific journey. Real and invented examples have been proposed and analyzed in detail, showing possible interpretations, both correct and incorrect, and their consequences. Such a framework makes it clear that a statistical test alone cannot answer any scientific questions. Indeed, the interpretation of results and the verification of assumptions and test eligibility - subject to the author's evaluation - are crucial components of the integrity of the scientific investigation. Before using a test or adopting a measure, we must ask ourselves the following fundamental questions: Are there valid reasons to explore my research question? Am I sure my approach can fully and adequately answer my research question? Am I sure that my model's assumptions - basic and hidden - are sufficiently satisfied? How could violating those assumptions affect the validity of the results and stakeholders? Is the effect size relevant regardless of statistical significance?