Keynote talk: To use or not to use? The metrics to measure software quality (developers' view)

Abstract

Summary form only given, as follows. The complete presentation was not made available for publication as part of the conference proceedings. The research related to software product metrics is becoming increasingly important and widespread. One of the most popular research topics in this area is the development of different software quality models that are based on product metrics. These quality models can be used to assist software project managers in making decisions e.g. if the most “critical” parts of the code have been identified by using an error-proneness model, the testing resources can be concentrated on these parts of the code. As another example, the quality models based on complexity and coupling metrics may assist in estimating the cost of software changes (impact analysis). Many empirical studies have been conducted to validate these quality models. The main aim of these empirical investigations has been to identify which metrics are the best predictors to estimate the changes of software quality. The examination of the applicability of the quality models to different application domains and development platforms is also a very interesting research area. Besides the intensive research activities, software product metrics have also been integrated into the practical development processes of many software companies. This is partly due to the fact that generally a top-level management prefers to make its decisions relying on measurable data. Our industrial experiences show that the people responsible for software quality and top-level project managers are also interested in applying quality monitoring tools based on product metrics. Obviously, the advantages of using such a tool should be clearly defined, but it should be noted as well that its use in the daily development process may require significant additional resources. On the other hand, according to our experiences, convincing developers about the usefulness of software product metrics can be very difficult. When analysing concrete metrics, the developers’ first reaction is usually looking for examples that justify the substantial deviation from the proposed metric value. Developers can hardly accept that there is a correlation between the metrics and the quality of the software developed by them. Consequently, we conducted a study where we asked the opinion of developers working in different fields and on different platforms about software product metrics. The experts participating in the study possessed different development skills (young developers and senior ones with many years of experience). The participants worked on C/C++, C#, Java and SQL platforms. Since some of the developers worked in open-source projects, open-source related questions were also included in the study. Besides the questions related to size, complexity, and coupling metrics, we asked the developers’ opinion about coding rule violations and copy-paste programming. We asked every developer to mark a so-called reference system (a system whose development process he had actively participated in). These reference systems have been analyzed with the Columbus tool, and we calculated metrics, rule violations and clones. Besides the general questions, in the study we included questions that were related to the metrics derived from the given reference system. The answers were evaluated on the basis of different aspects. We examined to what extent the developers’ professional skills, the development platforms and the specialities of a given application domain can affect the answers.