Process mining application in software process assessment

Abstract

Nowadays, our daily life heavily depends on software. Software is everywhere, from appliances in our homes, to safety-critical systems such as medical equipment. The failure of these software-intensive systems results in high financial losses, environmental or property damages, or even loss of life. Therefore the quality of software is recognized as a critical factor in software development. In order to achieve high quality of software products, reduce the costs of their development and deliver the product within specified time, activities for creating these products must be specified and executed. This set of activities is called the software development process. The software development process aims at creating complex and quality software products. Based on the premise that good quality software development process leads to good quality software products, the need for continuous software process improvement is recognized. Software process improvement is a cyclic activity involving assessing the current software development process in an organization, setting improvement goals, planning and executing improvements, analyzing and evaluating improvement results. A starting point for the software process improvement is software process assessment, which creates a baseline for improvement. The information gathered during the software process assessment must be objective and provide a realistic picture of the software development process. Currently, such process assessment is based on the information that is gathered based on interviews, brainstorm sessions, presentations, questionnaires and document reviews. These information gathering techniques are suitable in a stable environment with a limited problem space. However, software processes are complex entities, involving heterogeneous resources. Furthermore, software processes are often not explicitly modeled, and manuals to support the development work contain abstract guidelines and procedures. The differences between the actual executed (i.e. real) process and the documented process were already recognized by the research community. Therefore, another approach to gather process information for software process assessment is needed. In this thesis, we present a structural approach for choosing a more suitable information gathering strategy based on taking the characteristics of software processes into account. We performed a structured literature review in order to identify characteristics of software processes. Based on these characteristics, we identified the strategies 1) synthesis from characteristics of software processes and 2) experimentation with a discovered software process as more appropriate. Furthermore, we identified process mining as a set of techniques associated with these strategies and therefore suitable for software process assessment. In order to investigate the feasibility of process mining and its added value for software process assessment, we performed series of case studies on the practical application of software process mining on data from an industrial company. The case studies showed that process mining can provide useful and non-trivial process insights. Based on an in-depth study of a widely used software process improvement framework, namely the Capability Maturity Model Integration (CMMI), and its assessment method, a structured approach to use process mining for software process assessment was designed. First, we identified processes to be analyzed by process mining techniques. CMMI describes process areas to which an organization should map its software development. These process areas are satisfied by applying practices specific to a given process area. We identified specific practices, which can characterize software development processes and we proposed criteria to identify the practices for which process mining is feasible. These criteria are based on the notion of uncertainty. Next, we identified process aspects (such as compliance to the specified process model or timeliness of a software development process) essential for software process improvement as defined by CMMI. These process aspects determine which process information must be collected for the software process assessment and subsequent improvement. After that, we identified process-mining techniques that can provide this process information. As a result, we identified the parts of CMMI where collecting an objective evidence for process assessment by means of asking and deriving can be replaced by an application of identified process mining techniques. Although we applied process mining in the context of CMMI, our approach can also be applied to other software process improvement frameworks. Subsequently, the availability of the data required for process mining was studied. First, we identified data requirements for process mining techniques. Based on this, an approach to evaluate the suitability of software development support tools as data sources for process mining was developed. To test this approach, we considered software configuration management tools as possible data sources for process mining and selected the data logs of four commonly used software configuration tools for further study. We compared these data logs to the process mining data requirements and evaluated the suitability of a given tool as a data source for process mining.