Asking for Numbers and Quantities: Visual Design Effects in Paper&Pencil Surveys

Abstract

When reporting frequencies and quantities in self-administered surveys using open-ended questions, respondents make use of estimation strategies in order to provide a reasonable response. Accordingly, respondents do not only report pure numbers using digits, instead they extend their response to alphanumeric information and explicit estimates (e.g. “about 10–20 cigarettes per day”). Such responses are not only of reduced quality, but they also require a lot of editing and cleaning in the post-field phase of a survey. While previous methodological work on frequency questions has focused on question wording, reference periods and other text elements that have traditionally been considered significant for high quality frequency responses, in this article we test the impact of various graphical characteristics of the response field in order to reduce the number of alphanumeric elaborations and explicit estimates. The field-experimental studies reported are based on the assumption that respondents interpret formal characteristics of frequency questions as meaningful information throughout the question answer process. The findings reported will help survey designers reduce the proportion of estimated responses.