Evidence-Based Aggregation and Ranking in an Ordinal Scale

Abstract

A method has been developed for aggregating information from experts and then ranking these aggregated estimates in the case when the estimates are presented on an ordinal scale with an indication of their confidence. The method is based on the blurring of point estimates in accordance with the degrees of their confidence and the formation of the so-called bodies of evidence. The aggregation is performed based on the conjunctive rules of combination of the Dempster – Shafer theory, and the ranking is performed by calculating (possibly interval) medians of the evidence bodies corresponding to the pignistic probabilities. The method is illustrated by the example of ranking the scores of reviewers in a conference management system such as EasyChair.