Long-Term Evaluation of Technical Debt in Open-Source Software

Abstract

Background: A consistent body of research and practice have identified that technical debt provides valuable and actionable insight into the design and implementation deficiencies of complex software systems. Existing software tools enable characterizing and measuring the amount of technical debt at selective granularity levels; by providing a computational model, they enable stakeholders to measure and ultimately control this phenomenon. Aims: In this paper we aim to study the evolution and characteristics of technical debt in open-source software. For this, we carry out a longitudinal study that covers the entire development history of several complex applications. The goal is to improve our understanding of how the amount and composition of technical debt changes in evolving software. We also study how new technical debt is introduced in software, as well as identify how developers handle its accumulation over the long term. Method: We carried out our evaluation using three complex, open-source Java applications. All 110 released versions, covering more than 10 years of development history for each application were analyzed using SonarQube. We studied how the amount, composition and history of technical debt changed during development, compared our results across the studied applications and present our most important findings. Results: For each application, we identified key versions during which large amounts of technical debt were added, removed or both. This had significantly more impact when compared to the lines of code or class count increases that generally occurred during development. However, within each version, we found high correlation between file lines of code and technical debt. We observed that the Pareto principle was satisfied for the studied applications, as 20% of issue types generated around 80% of total technical debt. Interestingly, there was a large degree of overlap between the issues that generated most of the debt across the studied applications. Conclusions: Early application versions showed greater fluctuation in the amount of existing technical debt. We found application size to be an unreliable predictor for the quantity of technical debt. Most debt was introduced in applications as part of milestone releases that expanded their feature set; likewise, we identified releases where extensive refactoring significantly reduced the level of debt. We also discovered that technical debt issues persist for a long time in source code, and their removal did not appear to be prioritized according to type or severity.