Bayesian probability distributions for assessing measurement of subjective software attributes

Abstract

In order to validate objective indirect measures of software attributes, it is vital to demonstrate sufficiency of the representation condition. However, when a direct measurement on an ordinal scale is undertaken by human-raters, a significant degree of subjectivity may exist. Consequently, demonstrating that an objective indirect measure represents the attribute is difficult. It is difficult because subjectivity during direct measurement will lead to miss-classification. Further, during a demonstration the objective indirect measure itself cannot be assumed to give the ‘true’ measurement values. Modular cohesion is an attribute measured on an ordinal scale that exhibits subjectivity during direct measurement. By reference to cohesion classification data Bayesian inference probability distributions are constructed that represent error due to subjectivity during direct measurement. Using these probability distributions, an approach to demonstrate sufficiency of the representation condition for an objective indirect measure is proffered. In addition, Bayesian probability distributions can be used to provide informative estimates of the predictive capability of prediction systems for subjective attributes.