Leakage and the reproducibility crisis in machine-learning-based science

Abstract

Machine-learning (ML) methods have gained prominence in the quantitative sciences. However, there aremany known methodological pitfalls, including data leakage, in ML-based science. We systematically investigate reproducibility issues in ML-based science. Through a survey of literature in fields that haveadopted ML methods, we find 17 fields where leakage has been found, collectively affecting 294 papersand, in some cases, leading to wildly overoptimistic conclusions. Based on our survey, we introduce a detailed taxonomy of eight types of leakage, ranging from textbook errors to open research problems. We propose that researchers test for each type of leakage by filling out model info sheets, which weintroduce. Finally, we conduct a reproducibility study of civil war prediction, where complex ML models are believed to vastly outperform traditional statistical models such as logistic regression (LR). When theerrors are corrected, complex ML models do not perform substantively better than decades-old LR models.