Bipolarity and/or duality of social survey measurement scales and the question-order effect

Abstract

When measuring (complex) attitudes within a social survey, researchers often use balanced lists of positive and negative items. The purpose of the present research is to investigate: (a) whether a specific order of measurement scale items can lead to the bipolar (single-dimensional) concept (attitude) being recognised as a dual (bi-dimensional) concept and vice-versa; and (b) whether item order can affect the consistency (metric characteristics) of a measurement scale. An experiment on a group of social science students was conducted: students were randomly split into three subgroups and three different version of a questionnaire (with three differing item orders) were applied. A multi-group confirmatory factor analysis (‘CFA’) and a single group CFA for each item order separately were applied. The final conclusion of the experiment is that there is no general rule about how and when respondents form separate (dual) or unidimensional (continuous) representations of measured concepts. Item-order effects are possible, but they are not as important as one would expect. The results of the experiment also suggest that other factors should be taken into account: the content of the measured concept and the cognitive sophistication of the respondents.