Copula-based software metrics aggregation

Maria Ulan,Morgan Ericsson,Anna Wingkvist,Welf Löwe

doi:10.1007/s11219-021-09568-9

Abstract

A quality model is a conceptual decomposition of an abstract notion of quality into relevant, possibly conflicting characteristics and further into measurable metrics. For quality assessment and decision making, metrics values are aggregated to characteristics and ultimately to quality scores. Aggregation has often been problematic as quality models do not provide the semantics of aggregation. This makes it hard to formally reason about metrics, characteristics, and quality. We argue that aggregation needs to be interpretable and mathematically well defined in order to assess, to compare, and to improve quality. To address this challenge, we propose a probabilistic approach to aggregation and define quality scores based on joint distributions of absolute metrics values. To evaluate the proposed approach and its implementation under realistic conditions, we conduct empirical studies on bug prediction of ca. 5000 software classes, maintainability of ca. 15000 open-source software systems, and on the information quality of ca. 100000 real-world technical documents. We found that our approach is feasible, accurate, and scalable in performance.

Highlights

Software quality assessment helps to improve software and its development based on qualitative and quantitative data
I.e., how one can combine different software metrics into a single measure, that preserves properties of original data with a minimum of human supervision
To demonstrate that the proposed approach is valuable, it should be shown that an aggregate metrics based on copulas leads to accurate (R4), sensitive (R5), and scaling (R6) assessments of quality

Summary

Introduction

Software quality assessment helps to improve software and its development based on qualitative and quantitative data. To get a complete picture of (sub-) characteristics and of quality, several metrics need to be assessed and aggregated. Quality assessment tools often use different statistical assumptions about the metrics distributions and their independence and aggregation often performed in an ad-hoc manner. There is a need to define the aggregation of metrics in quality models formally, based on accepted mathematical theories. This way, quality scores, the result of metrics aggregation, become uniquely interpretable and provide a common basis for decision making. The interpretation of aggregation should be aligned with intuitive reasoning, expert knowledge, and common sense, and based on mathematical soundness to remove uncertainty, subjectivity, and degrees of freedom for quality assessment tool providers. There are several possible quality models, depending on the different goals

Objectives

Results

Discussion

Conclusion