Long-Term Evaluation of Technical Debt in Open-Source Software

Abstract

Existing software tools enable characterizing and measuring the amount of technical debt at selective granularity levels. In this paper we aim to study the evolution and characteristics of technical debt in open-source software. We carry out a longitudinal study that covers the entire development history of several complex applications. We study how technical debt is introduced in software, as well as identify how developers handle its accumulation over the long term. We carried out our evaluation using three complex, open-source Java applications. All 110 released versions, covering more than 10 years of development history for each application were analyzed using SonarQube. We studied how the amount, composition and history of technical debt changed during development, compared our results across the studied applications and present our most important findings. For each application, we identified key versions during which large amounts of technical debt were added, removed or both. This had significantly more impact when compared to the lines of code or class count increases that generally occurred during development. Within each version, we found high correlation between file lines of code and technical debt. We observed that the Pareto principle was satisfied for the studied applications, as 20% of issue types generated around 80% of total technical debt. Early application versions showed greater fluctuation in the amount of existing technical debt. Application size appeared to be an unreliable predictor for the quantity of technical debt. Most debt was introduced in applications as part of milestone releases that expanded their feature set. We also discovered that technical debt issues persist for a long time in source code, and their removal did not appear to be prioritized according to type or severity.