Invalid bug reports complicate the software aging situation

Abstract

Symptoms of software aging include performance degradation and failure occurrence increasing when software systems run for a period of time. Therefore, software aging is closely related to system performance. Understanding and analyzing performance issues in the software system is critical to mastering software aging information. Instead of focusing on normal valid bug reports (VBRs), this paper advocates the usage of invalid bug reports (IBRs) to capture software aging signals. We use performance bugs that are highly related to software aging as an example to construct a binary classification model for bug report classification. We conduct a rigorous evaluation of the constructed models via different performance measures (i.e., recall, precision, F1-score, AUC). Then, the model is used to predict the performance bug reports (PBRs) in IBRs, and a manual analysis of the prediction results is conducted to identify aging-related bug reports (ABRs). The final results show that the ratio of PBRs in IBRs ranges from 4.9 to 42.18% for the two real open-source projects HDFS and HBase when considering five different classifiers. Among these five classifiers, Support Vector Machine (SVM) classifier can achieve the best performance. The ratios of PBRs in IBRs by using this classifier are 11.1% and 15.35% for these two datasets and the performances in terms of F1-score are 85% and 74%. Further analysis of the predicted PBRs of IBRs in the project HDFS is conducted through a manual user case study; some surprising findings revealing the relationship between IBRs, PBRs, and ABRs are presented: (1) Around 50% of the PBRs in IBRs are related to software aging; (2) components that undertake major tasks are more prone to aging problems; (3) more than 50% ARBs lead to timeout, 33% ARBs are caused by improper control of memory or threats, and 29% ARBs are caused by inappropriate management of file operation or disk usage; (4) hard to reproduce is the major reason that ARBs are usually closed as invalid because many aging-related bugs would temporarily disappear by restarting the system.