Question Format, Response Effort, and Response Quality

Abstract

In empirical social research, using questions with an agreement/disagreement scale, also known as agree/disagree (A/D) questions, is a very popular technique for measuring attitudes. Major national and international surveys, such as the American National Election Study (ANES), the Eurobarometer, and the International Social Survey Program (ISSP) make use of the A/D question format. Methodological considerations, however, suggest that such questions require effortful cognitive processing and are prone to response bias. Therefore, many survey scientists recommend the use of item-specific (IS) questions, which usually consist of an interrogative request for an answer and tailored response categories so that they seem to imply less response effort. In this thesis, I propose an explanatory approach to explain differences between the A/D and IS question format in terms of response effort and response quality. Furthermore, I provide supporting evidence for this approach from five experimental studies employing different methods and techniques, such as eye tracking. The overall results indicate that IS questions are associated with higher response effort, but they are more robust against response bias, such as scale direction effects, and, thus, produce better response quality. All in all, the findings suggest substantial differences between the presumed cognitive complexity of question formats and the response effort expended in responding.