Reliability of ranking-based decision methods: a new perspective from the alternatives’ supremacy

Abstract

Due to the immediacy of communications, the interconnection of different data sources and the large volume of information available in digital environments, rankings have become one of the most used tools in the decision making (DM) process. When choosing an option, the decision maker not only considers the positions of the different alternatives into the ranking, but also usually checks the intensity values associated with them. Therefore, it is very important that methods used to build rankings adequately represent the preferences of users. These issues, known as order and intensity preservation conditions, have been studied for the well-known multi-criteria decision analysis (MCDA) called analytical hierarchical process (AHP) and extended to reconstruction methods of AHP matrices by defining a measure that considers only the order preservation condition. In this article, a measure, complementary to the latter, that allows establishing the difference between the predominance of the alternatives between two rankings is defined. To do this, the relations of predominance between the alternatives for each ranking are analyzed, and then the comparison between these relations is made by defining a bounded supremacy-based measure called supremacy difference index $$ sdi_{m} $$ . The $$ sdi_{m} $$ behavior is compared to conventional distance measures that are not bounded, and it is used to compare three reconstruction methods of AHP from the supremacy point of view and, finally, how the $$ sdi_{m} $$ usage can be extended for the evaluation of any ranking-based method is discussed.