On Reproducibility of Deep Convolutional Neural Networks Approaches

Abstract

Nowadays, Machine Learning techniques are more and more pervasive in several application fields. In order to perform an evaluation as reliable as possible, it is necessary to consider the reproducibility of these models both at training and inference time. With the introduction of Deep Learning (DL), the assessment of reproducibility became a critical issue due to heuristic considerations made at training time that, although improving the optimization performances of such complex models, can result in non-deterministic outcomes and, therefore, not reproducible models. The aim of this paper is to quantitatively highlight the reproducibility problem of DL approaches, proposing to overcome it by using statistical considerations. We show that, even if the models generated by using several times the same data show differences in the inference phase, the obtained results are not statistically different. In particular, this short paper analyzes, as a case study, our ICPR2018 DL based approach for the breast segmentation in DCE-MRI, demonstrating the reproducibility of the reported results.