Heuristics for prioritizing pair-wise elicitation questions with additive multi-attribute value models

Abstract

Additive value models are widely used in Multiple Criteria Decision Analysis. Direct elicitation of the value model preference parameters can impose excessive cognitive burden on the decision maker. Indirect techniques that employ pair-wise questions have been proposed for lowering the elicitation effort. In all practically relevant problems, more than a single question needs to be answered for arriving at a sufficiently precise outcome. The selection and ordering of questions affects the number of answers required for ranking the decision alternatives. However, evaluating all possible questions and answers is intractable due to the search space being, in the worst case, of factorial size. This paper develops heuristics for prioritizing pair-wise elicitation questions based on (1) necessary preference relations, (2) extreme ranks attained by the alternatives, (3) pair-wise preference indices, and (4) rank acceptability indices. We also introduce three metrics for assessing quality of a question prioritization heuristic. Numerical results allow us to identify a subset of heuristics that score well on our metrics in a variety of problem settings. This conclusion was validated in a real-world experiment where 101 subjects answered pair-wise questions to rank 10 mobile phone packages evaluated in terms of four criteria.