New results on inconsistency indices and their relationship with the quality of priority vector estimation

Abstract

The article is devoted to the problem of inconsistency in the pairwise comparisons based prioritization methodology. The issue of “inconsistency” in this context has gained much attention in recent years. The literature provides us with a number of different “inconsistency” indices suggested for measuring the inconsistency of the pairwise comparison matrix (PCM). The latter is understood as a deviation of the PCM from the consistent case – a notion that is formally defined in this theory. However the usage of the indices is justified only by some heuristics. It is still unclear what they really “measure”. What is even more important and still not known is the relationship between their values and the “consistency” of the decision maker's judgments on the one hand, and the prioritization results upon the other.In this paper we argue that it is necessary to distinguish between the three following tasks: the “measuring” of the “PCM inconsistency”, the PCM-based “measuring” of the consistency of the decision maker's judgments and, finally, the “measuring” of the usefulness of the PCM as a source of information for estimation of the priority vector (PV). We present examples showing that improving the consistency of PCM may lead to poorer PV estimation results, and that such a situation may occur quite naturally. Next we focus on the third of the above tasks, which is very important one in multi-criteria decision making. For the first time in literature, with the help of Monte Carlo simulations, we analyze the performance of the most common inconsistency indices as indicators of the final PV estimates quality. We consider two types of PV estimation errors and examine their distributions as well as their relationship with the indices values. The new results presented here allow for a more profound interpretation of the well-known inconsistency characteristics. Moreover, based on the analysis, we also introduce a new inconsistency index. In comparison with the other ones, the new index manifests significantly higher correlation with PV estimation errors. This fact also enables us to propose a novel PCM acceptance approach that is supported by the classical statistical methodology.