A Comparison of Different Methods for the Elicitation of Attribute Weights: Structural Modeling, Process Tracing, and Self-Reports

Abstract

This study investigates whether structural modeling, process tracing, and self-reports are able to provide similar information about attribute weights in multiattribute evaluation processes. In three experiments subjects had to evaluate a large number of profiles of fictitious persons described on a number of attributes. The experiments differed in type of judgment task, type of subjects, and number of attributes. Subject attribute weights were derived in all cases by fitting a statistical model (statistical weights), by analyzing verbal protocols (verbal protocol weights), and by directly asking the subject how important the attributes are for the judgments (subjective weights). Correspondence between the three sets of weights is examined in two ways: by computing the correlation between three sets of weights and by calculating how adequately the different sets of weights, applied in a linear model, can predict the subject′s judgments. The first method appears to be inappropriate for investigating correspondence. The correlations are rather unstable because of the small number of attributes, and apart from that, they tend to underestimate real correspondence when the weights in the respective sets are approximately equal. The second method shows that the three sets of weights are about equally adequate in predicting the actual subject judgments. It is concluded that this method convincingly demonstrates that the three different ways of eliciting attribute weights yield similar results.