User acceptance of multi-criteria decision support systems: The impact of preference elicitation techniques

Abstract

Previous research indicates that decision makers are often reluctant to use potentially beneficial multi-criteria decision support systems (MCDSS). Prior research has not examined the specific impact of preference elicitation techniques on user acceptance of MCDSS. The present research begins to fill this gap by examining the effect on users’ MCDSS evaluations of two commonly used preference elicitation techniques, absolute measurement and pairwise comparisons, while holding constant all other aspects of the MCDSS and decision making task. Experimental results ( N = 153) indicate that users consider MCDSS with pairwise comparisons to be higher in decisional conflict, more effortful, less accurate, and overall less desirable to use than MCDSS with absolute measurements. Thus, any potential normative superiority of a preference elicitation technique must be balanced against its potentially adverse effects on user acceptance of the MCDSS within which it is employed. We present a research agenda for exploring the tradeoffs between objective validity and user acceptance in the design of decision analysis tools.