Objective: This study investigates three different methods for provisioning computational environments in scientific research, focusing on the level of reproducibility offered by each one. The methods are manual setup, container-based, and one based on Infrastructure-as-Code principles, the Environment Code-First (ECF) framework. Method: The methodology adopted for this research comprises a comparative study design evaluating three distinct approaches to computational environment provisioning: manual setup, container-based setup, and the ECF framework. The study involved reproducing a research experiment using the Inception-V3 model to identify canine hip dysplasia from X-ray images across different computational setups. Data was collected through performance metrics such as reproducibility rate, consistency of results, and ease of setup. Results and Discussion: The results revealed that while offering complete control over the environment, the manual setup needed to be more consistent and more accessible to reproduce, leading to variability in the results. The container-based method improved reproducibility but required manual intervention to set up the container infrastructure. The ECF-based approach, however, demonstrated superior performance by fully automating the environment provisioning process, ensuring 100% reproducibility and consistency across different platforms. Research Implications: The practical and theoretical implications of this research are discussed, providing insights into how the results can be applied to advance practices in computational research and scientific reproducibility. These implications could encompass the broader adoption of IaC tools in scientific experiments, potentially leading to more reliable and reproducible research outcomes. Originality/Value: This study contributes to the literature by highlighting the innovative application of the Infrastructure-as-Code approach to achieving reproducibility in scientific research. The potential impact of adopting IaC tools on improving the reliability and consistency of research outcomes evidences the relevance and value of this research.
Read full abstract