Evaluating the practical use of different measurement scales in requirements prioritisation

Abstract

The importance of prioritising requirements is widely recognised. A number of different techniques for prioritising requirements have been proposed, some based on an ordinal scale, others on a ratio scale. Some measurement scales provide more information than others, i.e. the ratio scale is richer than the ordinal scale. This paper aims to investigate the differences between the scales used in prioritisation. This is important since techniques using a richer scale tend to be more time-consuming and complex to use. Thus, there is a trade-off between simple techniques only providing ranks and complex techniques providing information about the relative distance between requirements priorities. The paper suggests an approach to measure the skewness of the ratio distribution and a way to use the cost-value approach on ordinal scale data. Four different empirical data sets were used to verify the suggested approaches. The skewness measure seems feasible to determine in which cases the ratio scale is valuable. It indicates that some of our subjects tend to use the extreme values of the scale while others are more modest. The cost-value approach based on ordinal scale data also seems feasible. The requirements selection decisions based on ordinal scale data agree substantially with the decisions based on ratio scale data.