Rank reversals in multi-criteria decision analysis with statistical modelling of ratio-scale pairwise comparisons

Abstract

In multi-criteria decision analysis, the overall performance of decision alternatives is evaluated with respect to several, generally conflicting decision criteria. One approach to perform the multi-criteria decision analysis is to use ratio-scale pairwise comparisons concerning the performance of decision alternatives and the importance of decision criteria. In this approach, a classical problem has been the phenomenon of rank reversals. In particular, when a new decision alternative is added to a decision problem, and while the assessments concerning the original decision alternatives remain unchanged, the new alternative may cause rank reversals between the utility estimates of the original decision alternatives. This paper studies the connections between rank reversals and the potential inconsistency of the utility assessments in the case of ratio-scale pairwise comparisons data. The analysis was carried out by recently developed statistical modelling techniques so that the inconsistency of the assessments was measured according to statistical estimation theory. Several type of decision problems were analysed and the results showed that rank reversals caused by inconsistency are natural and acceptable. On the other hand, rank reversals caused by the traditional arithmetic-mean aggregation rule are not in line with the ratio-scale measurement of utilities, whereas geometric-mean aggregation does not cause undesired rank reversals.