The metric quality of full-profile judgments and the number-of-attribute-levels effect in conjoint analysis

Abstract

Several studies have documented the existence of a positive association between the number of levels used for an attribute and the attribute's estimated influence (relative importance) on preference judgments in conjoint analysis. We propose that this number-of-levels effect is contingent upon the (lack of) metricity in the preference judgments. We use magnitude estimation to test this idea. We find that rating scale and magnitude estimation data provide comparable average effects. We then use the magnitude estimation data to categorize respondents into “metric” and “nonmetric” groups, and we find that the number-of-levels effect is substantially smaller for respondents who do satisfy the criteria for metric quality than for respondents who do not.