Software process evaluation: a machine learning framework with application to defect management process

Abstract

Software process evaluation is important to improve software development and the quality of software products in a software organization. Conventional approaches based on manual qualitative evaluations (e.g., artifacts inspection) are deficient in the sense that (i) they are time-consuming, (ii) they usually suffer from the authority constraints, and (iii) they are often subjective. To overcome these limitations, this paper presents a novel semi-automated approach to software process evaluation using machine learning techniques. In this study, we mainly focus on the procedure aspect of software processes, and formulate the problem as a sequence (with additional information, e.g., time, roles, etc.) classification task, which is solved by applying machine learning algorithms. Based on the framework, we define a new quantitative indicator to evaluate the execution of a software process more objectively. To validate the efficacy of our approach, we apply it to evaluate the execution of a defect management (DM) process in nine real industrial software projects. Our empirical results show that our approach is effective and promising in providing a more objective and quantitative measurement for the DM process evaluation task. Furthermore, we conduct a comprehensive empirical study to compare our proposed machine learning approach with an existing conventional approach (i.e., artifacts inspection). Finally, we analyze the advantages and disadvantages of both approaches in detail.