Empirical investigation of the influence of continuous integration bad practices on software quality

Abstract

Continuous Integration (CI) is a powerful tool to leverage software development in a safe, reliable, and efficient way. However, in day-to-day of software development, bad habits can arise in working with CI, which can make CI not reach its full potential in projects. These bad habits we call CI bad practices. This paper presented an exploratory study on closed-source projects to investigate how CI bad practices can affect software quality. We observe (1) the impact on the quality internal attributes after the implantation of CI, (2) the evolution of software quality indicators over time, and (3) the bad practices considered most harmful to the software quality by the development teams. Our results mean that projects affected by CI bad practices will not necessarily have their quality impaired. However, this does not mean that quality will be maintained or improved over time. Our analysis of the quality indicators has allowed us to observe that, generally, they have remained stable, and the number of quality issues reported has not decreased considerably either. Finally, the quality indicator most affected by the CI bad practices was maintainability, followed by reliability and security. In addition, the vast majority of the bad practices analyzed were classified as having a medium level of effort for resolution. In this sense, we also classify bad practices according to the level of effort/impact on the quality that can help prioritize their resolution.