Are multi-criteria decision-making tools useful? An experimental comparative study of three methods

Abstract

Many decision makers still question the usefulness of multi-criteria decision-making methods and prefer to rely on intuitive decisions. In this study we evaluated a number of multi-criteria decision-making tools for their usefulness using incentive-based experiments, which is a novel approach in operations research but common in psychology and experimental economics. In this experiment the participants were asked to compare five coffee shops to win a voucher for their best-rated shop. We found that, although the usefulness of different multi-criteria decision-making tools varied to some extent, all the tools were found to be useful in the sense that, when they decided to change their ranking, they followed the recommendation of the multi-criteria decision-making tool. Moreover, the level of inconsistency in the judgements provided had no significant effect on the usefulness of these tools.