Technical Debt Identification Research Articles

Including validation of process control systems’ engineering into the technical debt classification

The Technical Debt (TD) concept depicts a situation in which a technical compromise is selected despite a better solution available. The associated risks to TD may be critical if TD is overlooked or not fixed properly. For instance, the product might not be used safely (in the healthcare supplies industry) or be contaminated (in the food & beverage sector) due to a design fault not adequately addressed. This work extends the TD classification for process automation systems that need to undergo a validation process according to Good Automated Manufacturing Practice (GAMP), which is required for pharmaceutical production or food processing, for example. The approach analysed industrial engineering documents to identify such TD types and sub-types. Three selected TD use cases confirmed by industry experts are reported. The presented meta-analysis approach on engineering documents can be employed as a TD identification method for such process control systems. The TD classification for process automation was thereby enlarged for an especially critical type of machine that may harm the health of many humans.

Local and Global Explainability for Technical Debt Identification

Local and Global Explainability for Technical Debt Identification

Identification and Management of Technical Debt

Technical debt is a concept used to describe the lack of good practices during software development, leading to several problems and costs. Identification and management strategies can help reduce these difficulties. In a previous study, Alves et al. (2016) analyzed the research landscape of such strategies from 2010 to 2014. This paper replicates and updates their study to explore the evolution of technical debt identification and management research landscape over a decade, including literature from 2010 until 2022. We analyzed 117 papers from the ACM Digital Library, IEEE Xplore, Science Direct, and Springer Link. Newly suggested strategies include automatically identifying admitted debt in comments, commits, and source code. Between 2015 and 2022, more empirical evaluations have been performed, and the general research focus has changed to a more holistic approach. Therefore, the research area evolved and reached a new level of maturity compared to previous results from Alves et al. (2016). Not only are code aspects considered for technical debt, but other aspects have also been investigated (e.g., models for the development process).

LEAP – A Process for Identifying Potential Technical Debt in Iterative System Development

AbstractSystems engineering has seen a rise in the use of iterative methods to design and develop both hardware and software systems which allow for system refinement to be responsive to user needs. However, focusing on items with high value to the user can result in technical debt, where technical compromises made for short‐term gain impact the long‐term health of the system. Current methods for identifying technical debt focus on finding existing technical debt items within a system and not on proactive identification of technical debt during the iterative system planning process. This paper presents a novel technique to identify technologies that impact the ability of the system to satisfy the needs of its stakeholders. The method is used to evaluate different choices of technological implementations in both the temporal and the functional dimensions to reduce the risk of incurring technical debt which prevents the successful delivery of the system.

Technical Debt Prioritization in Telecommunication Applications: Why the Actual Refactoring Deviates from the Plan and How to Remediate It? Case Study in the COVID Era

This paper focuses on application of a technical debt prioritisation technique in telecommunication software managing a fleet of devices for a video surveillance system. Technical debt for this application was gathered, categorised and prioritised according to the Continuous Debt Valuation Approach (CoDVA), previously proposed by the authors. The following research question was posed: Is prioritising technical debt reduction based on CoDVA effective (i.e., executed as per plan, bringing tangible benefits)? The outbreak of COVID-19 pandemic caused unprecedented disturbance to the engineering organisations worldwide, therefore the technical debt identification phase had to be adapted to cope with a switch to forced working-from-home mode. This was achieved by applying the Wisdom of Crowds method, ensuring broad participation of engineers, and providing a fairly complete picture of the accrued technical debt. Nevertheless, the actual technical debt reduction activities did not follow exactly the expected guidelines. The three main causes of this phenomenon were discovered: continuous refactoring approach, sizing of technical debt items, and the broadened scope of refactoring activities. Therefore, as a result of this case study we propose to adopt a specific broadened definition of technical debt and follow a few rules for defining its scope and granularity.

A framework for conditional statement technical debt identification and description

Technical Debt occurs when development teams favour short-term operability over long-term stability. Since this places software maintainability at risk, technical debt requires early attention to avoid paying for accumulated interest. Most of the existing work focuses on detecting technical debt using code comments, known as Self-Admitted Technical Debt (SATD). However, there are many cases where technical debt instances are not explicitly acknowledged but deeply hidden in the code. In this paper, we propose a framework that caters for the absence of SATD comments in code. Our Self-Admitted Technical Debt Identification and Description (SATDID) framework determines if technical debt should be self-admitted for an input code fragment. If that is the case, SATDID will automatically generate the appropriate descriptive SATD comment that can be attached with the code. While our approach is applicable in principle to any type of code fragments, we focus in this study on technical debt hidden in conditional statements, one of the most TD-carrying parts of code. We explore and evaluate different implementations of SATDID. The evaluation results demonstrate the applicability and effectiveness of our framework over multiple benchmarks. Comparing with the results from the benchmarks, our approach provides at least 21.35, 59.36, 31.78, and 583.33% improvements in terms of Precision, Recall, F-1, and Bleu-4 scores, respectively. In addition, we conduct a human evaluation to the SATD comments generated by SATDID. In 1-5 and 0–5 scales for Acceptability and Understandability, the total means achieved by our approach are 3.128 and 3.172, respectively.

Using deep temporal convolutional networks to just-in-time forecast technical debt principal

Technical debt is a widely used metaphor to summarize all the consequences of poorly written code. Managing technical debt is important for software developers to allow adequate planning for software maintenance and improvement activities, such as refactoring and preventing system degradation. Several studies in the literature investigate the identification of technical debt and its consequences. This work aims to explore a deep learning approach to just-in-time predict the impact on technical debt when changes are performed on the source code. In this way the developer can work better, trying to improve the quality of the code that is being modified. Knowing what the TD trend will be in just-in-time source code with the change made is the key to avoiding a project taking a long time to remediate or improve. The model exploits the knowledge of quality and ad-hoc process metrics evolution over time. To validate the approach, a large dataset, including metrics evaluated from commits of ten Java software projects, was built. The results obtained show the effectiveness of the proposed approach in predicting the Technical Debt accumulation within the source code.

The Gap between the Admitted and the Measured Technical Debt: An Empirical Study

Technical debt is a well understood and used concept in IT development. The metaphor, rooted in the financial world, captures the amount of work that development teams owe to a product. Every time developers take a shortcut within development, the technical debt accumulates. Technical debt identification can be accomplished via manual reporting on the technical debt items, which is called self-admitted technical debt. Several specialised methods and tools have also emerged that promise to measure the technical debt. Based on experience in the community, the impression emerged that the measured technical debt is of a significantly different amount than the self-admitted debt. In this context, we decided to perform empirical research on the possible gap between the two. We investigated 14 production-grade software products while determining the amount of accumulated technical debt via (a) a self-admitting procedure and (b) measuring the debt. The outcomes show clearly the significant difference in the technical debt reported by the two methods. We urge development and quality-assurance teams not to rely on technical debt measurement alone. The tools demonstrated their strength in identifying low-level code technical debt items that violate a set of predefined rules. However, developers should have additional insight into violations, based on the interconnected source code and its relation to the domain and higher-level design decisions.

DebtFree: minimizing labeling cost in self-admitted technical debt identification using semi-supervised learning

Keeping track of and managing Self-Admitted Technical Debts (SATDs) is important for maintaining a healthy software project. Current active-learning SATD recognition tool involves manual inspection of 24% of the test comments on average to reach 90% of the recall. Among all the test comments, about 5% are SATDs. The human experts are then required to read almost a quintuple of the SATD comments which indicates the inefficiency of the tool. Plus, human experts are still prone to error: 95% of the false-positive labels from previous work were actually true positives. To solve the above problems, we propose DebtFree, a two-mode framework based on unsupervised learning for identifying SATDs. In mode1, when the existing training data is unlabeled, DebtFree starts with an unsupervised learner to automatically pseudo-label the programming comments in the training data. In contrast, in mode2 where labels are available with the corresponding training data, DebtFree starts with a pre-processor that identifies the highly prone SATDs from the test dataset. Then, our machine learning model is employed to assist human experts in manually identifying the remaining SATDs. Our experiments on 10 software projects show that both models yield statistically significant improvement in effectiveness over the state-of-the-art automated and semi-automated models. Specifically, DebtFree can reduce the labeling effort by 99% in mode1 (unlabeled training data), and up to 63% in mode2 (labeled training data) while improving the current active learner’s F1 relatively to almost 100%.

Prevalence, common causes and effects of technical debt: Results from a family of surveys with the IT industry

Context:The technical debt (TD) metaphor describes actions made during various stages of software development that lead to a more costly future regarding system maintenance and evolution. According to recent studies, on average 25% of development effort is spent, i.e. wasted, on TD caused issues in software development organizations. However, further research is needed to investigate the relations between various software development activities and TD. Objective:The objective of this study is twofold. First, to get empirical insight on the understanding and the use of the TD concept in the IT industry. Second, to contribute towards precise conceptualization of the TD concept through analysis of causes and effects. Method:In order to address the research objective a family of surveys was designed as a part of an international initiative that congregates researchers from 12 countries—InsighTD. At country level, national teams ran survey replications with industry practitioners from the respective countries. Results:In total 653 valid responses were collected from 6 countries. Regarding the prevalence of the TD concept 22% of practitioners have only theoretical knowledge about it, and 47% have some practical experiences with TD identification or management. Further analysis indicated that senior practitioners who work in larger organizations, larger teams, and on larger systems are more likely to be experienced with TD management. Time pressure or deadlinewas the single most cited cause of TD. Regarding the effects of TD: delivery delay, low maintainability, and rework were the most cited. Conclusion:InsighTD is the first family of surveys on technical debt in software engineering. It provided a methodological framework that allowed multiple replication teams to conduct research activities and to contribute to a single dataset. Future work will focus on more specific aspects of TD management.

Machine Learning for Technical Debt Identification

Technical Debt (TD) is a successful metaphor in conveying the consequences of software inefficiencies and their elimination to both technical and non-technical stakeholders, primarily due to its monetary nature. The identification and quantification of TD rely heavily on the use of a small handful of sophisticated tools that check for violations of certain predefined rules, usually through static analysis. Different tools result in divergent TD estimates calling into question the reliability of findings derived by a single tool. To alleviate this issue we use 18 metrics pertaining to source code, repository activity, issue tracking, refactorings, duplication and commenting rates of each class as features for statistical and Machine Learning models, so as to classify them as High-TD or not. As a benchmark we exploit 18.857 classes obtained from 25 Java projects, whose high levels of TD has been confirmed by three leading tools. The findings indicate that it is feasible to identify TD issues with sufficient accuracy and reasonable effort: a subset of superior classifiers achieved an F2-measure score of approximately 0.79 with an associated Module Inspection ratio of approximately 0.10. Based on the results a tool prototype for automatically assessing the TD of Java projects has been implemented.

Evaluating the agreement among technical debt measurement tools: building an empirical benchmark of technical debt liabilities

Software teams are often asked to deliver new features within strict deadlines leading developers to deliberately or inadvertently serve “not quite right code” compromising software quality and maintainability. This non-ideal state of software is efficiently captured by the Technical Debt (TD) metaphor, which reflects the additional effort that has to be spent to maintain software. Although several tools are available for assessing TD, each tool essentially checks software against a particular ruleset. The use of different rulesets can often be beneficial as it leads to the identification of a wider set of problems; however, for the common usage scenario where developers or researchers rely on a single tool, diverse estimates of TD and the identification of different mitigation actions limits the credibility and applicability of the findings. The objective of this study is two-fold: First, we evaluate the degree of agreement among leading TD assessment tools. Second, we propose a framework to capture the diversity of the examined tools with the aim of identifying few “reference assessments” (or class/file profiles) representing characteristic cases of classes/files with respect to their level of TD. By extracting sets of classes/files exhibiting similarity to a selected profile (e.g., that of high TD levels in all employed tools) we establish a basis that can be used either for prioritization of maintenance activities or for training more sophisticated TD identification techniques. The proposed framework is illustrated through a case study on fifty (50) open source projects and two programming languages (Java and JavaScript) employing three leading TD tools.

Comparing Maintainability Index, SIG Method, and SQALE for Technical Debt Identification

There are many definitions of software Technical Debt (TD) that were proposed over time. While many techniques to measure TD emerged in recent times, there is still not a clear understanding about how different techniques compare when applied to software projects. The goal of this paper is to shed some light on this aspect, by comparing three techniques about TD identification that were proposed over time: (i) the Maintainability Index (MI), (ii) SIG TD models, and (iii) SQALE analysis. Considering 20 open source Python libraries, we compare the TD measurements time series in terms of trends and evolution according to different sets of releases (major, minor, and micro), to see if the perception of practitioners about TD evolution could be impacted. While all methods report generally growing trends of TD over time, there are different patterns. SQALE reports more periods of steady states compared to MI and SIG TD. MI is the method that reports more repayments of TD compared to the other methods. SIG TD and MI are the models that show more similarity in the way TD evolves, while SQALE and MI are less similar. The implications are that each method gives slightly a different perception about TD evolution.

Identifying self-admitted technical debt through code comment analysis with a contextualized vocabulary

Abstract Context Previous work has shown that one can explore code comments to detect Self-Admitted Technical Debt (SATD) using a contextualized vocabulary. However, current detection strategies still return a large number of false positives items. Moreover, those strategies do not allow the automatic identification of the type of debt of the identified items. Objective This work applies, evaluates, and improves a set of contextualized patterns we built to detect self-admitted technical debt using code comment analysis. We refer to this set of patterns as the self-admitted technical debt identification vocabulary. Method We carry out three empirical studies. Firstly, 23 participants analyze the patterns of a previously defined contextualized vocabulary and register their level of importance in identifying SATD items. Secondly, we perform a qualitative analysis to investigate the relation between each pattern and types of debt. Finally, we perform a feasibility study using a new vocabulary, improved based on the results of the previous empirical studies, to automatically identify self-admitted technical debt items, and types of debt, that exist in three open source projects. Results More than half of the new patterns were considered decisive or very decisive to detect technical debt items. The new vocabulary was able to find items associated to code, design, defect, documentation, and requirement debt. Thus, the result of the work is an improved vocabulary that considers the level of importance of each pattern and the relationship between patterns and debt types to support the identification and classification of SATD items. Conclusion The studies allowed us to improve a vocabulary to identify self-admitted technical debt items through code comments analysis. The results show that the use of pattern-based code comment analysis can contribute to improve existing methods, or create new ones, for automatically identifying and classifying technical debt items.

Automating Change-Level Self-Admitted Technical Debt Determination

Technical debt (TD) is a metaphor to describe the situation where developers introduce suboptimal solutions during software development to achieve short-term goals that may affect the long-term software quality. Prior studies proposed different techniques to identify TD, such as identifying TD through code smells or by analyzing source code comments. Technical debt identified using comments is known as Self-Admitted Technical Debt (SATD) and refers to TD that is introduced intentionally. Compared with TD identified by code metrics or code smells, SATD is more reliable since it is admitted by developers using comments. Thus far, all of the state-of-the-art approaches identify SATD at the file-level. In essence, they identify whether a file has SATD or not. However, all of the SATD is introduced through software changes. Previous studies that identify SATD at the file-level in isolation cannot describe the TD context related to multiple files. Therefore, it is beneficial to identify the SATD once a change is being made. We refer to this type of TD identification as “Change-level SATD Determination”, which determines whether or not a change introduces SATD. Identifying SATD at the change-level can help to manage and control TD by understanding the TD context through tracing the introducing changes. To build a change-level SATD Determination model, we first identify TD from source code comments in source code files of all versions. Second, we label the changes that first introduce the SATD comments as TD-introducing changes. Third, we build the determination model by extracting 25 features from software changes that are divided into three dimensions, namely diffusion, history and message, respectively. To evaluate the effectiveness of our proposed model, we perform an empirical study on 7 open source projects containing a total of 100,011 software changes. The experimental results show that our model achieves a promising and better performance than four baselines in terms of AUC and cost-effectiveness (i.e., percentage of TD-introducing changes identified when inspecting 20 percent of changed LOC). On average across the 7 experimental projects, our model achieves AUC of 0.82, cost-effectiveness of 0.80, which is a significant improvement over the comparison baselines used. In addition, we found that “Diffusion” is the most discriminative dimension among the three dimensions of features for determining TD-introducing changes.

Understanding automated and human-based technical debt identification approaches-a two-phase study

ContextThe technical debt (TD) concept inspires the development of useful methods and tools that support TD identification and management. However, there is a lack of evidence on how different TD identification tools could be complementary and, also, how human-based identification compares with them.ObjectiveTo understand how to effectively elicit TD from humans, to investigate several types of tools for TD identification, and to understand the developers’ point of view about TD indicators and items reported by tools.MethodWe asked developers to identify TD items from a real software project. We also collected the output of three tools to automatically identify TD and compared the results in terms of their locations in the source code. Then, we collected developers’ opinions on the identification process through a focus group.ResultsAggregation seems to be an appropriate way to combine TD reported by developers. The tools used cannot help in identifying many important TD types, so involving humans is necessary. Developers reported that the tools would help them to identify TD faster or more accurately and that project priorities and current development activities are important to be considered together, along with the values of principal and interest, when deciding to pay off a debt.ConclusionThis work contributes to the TD landscape, which depicts an understanding between different TD types and how they are best discovered.

VisminerTD: a tool for automatic identification and interactive monitoring of the evolution of technical debt items

Technical debt (TD) contextualizes problems faced during software evolution considering the tasks that are not carried out adequately during software development. Software TD is a type of debt that brings a short-term benefit, but which may have to be paid with interest later on in the software development life cycle. Its presence brings risks to the project and can reduce its quality. It is worthwhile to have automatic mechanisms to monitor it, as TD monitoring requires the analysis of large amounts of complex data. Therefore, the combination of software metrics and code comment analysis, in the identification, and information visualization techniques, in monitoring, present themselves as a promising strategy to manage TD. This work presents VisminerTD, a tool that allows the automatic identification and interactive monitoring of the evolution of TD items by combining software metrics, code comment analysis, and information visualization. To evaluate its applicability, a feasibility study was carried out considering JUnit 4 and Apache Ant software projects. The results indicated that VisminerTD can support software development teams in monitoring TD items. In addition, a second case study was performed to assess the feasibility of the proposed tool regarding its usefulness, ease of use, and self-predicted future use. The results provided positive evidence on the use of the proposed tool, indicating (i) that it can be useful in supporting TD Identification and TD monitoring activities and (ii) that it can bring gains in terms of comprehensiveness and efficacy when evaluating the desirable time to identify and monitor different types of debt. Given the current scenario characterized by limited options of tools that combine different information to support automatic identification and monitoring of the evolution of TD items in software projects, VisminerTD can approximate the state-of-the-art and the state-of-the-practice in the TD area, contributing to a wider dissemination of the concept.

Identification and management of technical debt: A systematic mapping study

ContextThe technical debt metaphor describes the effect of immature artifacts on software maintenance that bring a short-term benefit to the project in terms of increased productivity and lower cost, but that may have to be paid off with interest later. Much research has been performed to propose mechanisms to identify debt and decide the most appropriate moment to pay it off. It is important to investigate the current state of the art in order to provide both researchers and practitioners with information that enables further research activities as well as technical debt management in practice. ObjectiveThis paper has the following goals: to characterize the types of technical debt, identify indicators that can be used to find technical debt, identify management strategies, understand the maturity level of each proposal, and identify what visualization techniques have been proposed to support technical debt identification and management activities. MethodA systematic mapping study was performed based on a set of three research questions. In total, 100 studies, dated from 2010 to 2014, were evaluated. ResultsWe proposed an initial taxonomy of technical debt types, created a list of indicators that have been proposed to identify technical debt, identified the existing management strategies, and analyzed the current state of art on technical debt, identifying topics where new research efforts can be invested. ConclusionThe results of this mapping study can help to identify points that still require further investigation in technical debt research.

Supporting Technical Debt Cataloging with TD-Tracker Tool

Technical debt (TD) is an emergent area that has stimulated academic concern. Managers must have information about debt in order to balance time-to-market advantages and issues of TD. In addition, managers must have information about TD to plan payments. Development tasks such as designing, coding, and testing generate different sorts of TD, each one with specific information. Moreover, literature review pointed out a gap in identifying and accurately cataloging technical debt. It is possible to find tools that can identify technical debt, but there is not a described solution that supports cataloging all types of debt. This paper presents an approach to create an integrated catalog of technical debts from different software development tasks. The approach allows tabulating and managing TD properties in order to support managers in the decision process. It also allows managers to track TD. The approach is implemented by TD-Tracker tool, which can integrate different TD identification tools and import identified debts. We present integrations between TD-Tracker and two external tools, used to identify potential technical debts. As part of the approach, we describe how to map the relationship between TD-Tracker and the external tools. We also show how to manage external information within TD-Tracker.

Comparing four approaches for technical debt identification

Software systems accumulate technical debt (TD) when short-term goals in software development are traded for long-term goals (e.g., quick-and-dirty implementation to reach a release date versus a w...