Crowdsourced test report prioritization considering bug severity

Abstract

In crowdsourced testing, a large number of test reports will be generated in a short time. How to efficiently inspect these reports becomes one of the critical steps in the testing process. In recent years, many automated techniques like clustering, classification, and prioritization have emerged to provide an automated inspection order over test reports. Even though these methods have achieved good performance, they did not consider the priority to image and text information. Simultaneously, existing prioritization approaches only focus on the rate of detecting faults but ignore the severity of the faults. In fact, bug severity is a vital indicator that the users provide to flag the criticality of a bug, so developers can then use it to set their priority for the resolution process. For these reasons, this paper presents a novel prioritization approach for crowdsourcing test reports. It extracts features from text and screenshot information of the test reports, uses the hash technique to index test reports, and finally designs a prioritization algorithm. To validate our approach, we conducted experiments on six industrial projects. The results and the hypotheses analysis show that our approach can detect all faults faster in a limited time and can prioritize reports that have higher severity faults compared with the existing methods.