What’s up with software metrics? – A preliminary mapping study

Abstract

Background Many papers are published on the topic of software metrics but it is difficult to assess the current status of metrics research. Aim This paper aims to identify trends in influential software metrics papers and assess the possibility of using secondary studies to integrate research results. Method Search facilities in the SCOPUS tool were used to identify the most cited papers in the years 2000–2005 inclusive. Less cited papers were also selected from 2005. The selected papers were classified according factors such as to main topic, goal and type (empirical or theoretical or mixed). Papers classified as “Evaluation studies” were assessed to investigate the extent to which results could be synthesized. Results Compared with less cited papers, the most cited papers were more frequently journal papers, and empirical validation or data analysis studies. However, there were problems with some empirical validation studies. For example, they sometimes attempted to evaluate theoretically invalid metrics and fail to appreciate the importance of the context in which data are collected. Conclusions This paper, together with other similar papers, confirms that there is a large body of research related to software metrics. However, software metrics researchers may need to refine their empirical methodology before they can answer useful empirical questions.