Questionnaire design, return rates, and response favorableness in an employee attitude questionnaire.

Abstract

Topic order and location of demographic items were systematically varied in an employee attitude survey distributed to 1,188 office personnel of a single organization. With six questionnaire topic areas arranged in the order that matched employee representatives' perceptions of employee priorities, returns were higher (96%) than with any of five random orders (average 78%). Returns were higher with demographic items placed at the end (85%) than at the beginning (77%). Global attitudes and three of six topic attitudes varied with topic order and were more positive with the prioritized topic order than with the five random orders. The largest order effect was noted in employees' attitudes toward pay (p < .001), which was the most important topic. Methodological and practical implications are discussed. In this experiment, we examined two design features of an employee attitude survey—topic order and demographic item location. Criteria were questionnaire return rates and response favorableness, including favorableness of global attitudes and attitudes toward specific topics. Understanding such effects is important for both scientific and practical reasons. Earlier research on return rates in attitude surveys has investigated effects of several features of questionnaire design and administration (see reviews by Dillman, 1978; Kanuk & Berenson, 1975). Little previous research has examined the effects of item order in surveys. Kraut, Wolfson, and Rothenberg (1975) circulated two versions of a questionnaire in which the order of two 23-item sections (from a total of 168 items) was manipulated and analyzed. Favorableness of responses was not influenced by whether the items appeared in the first half or the last half of the survey. Bradburn and Mason (1964) varied the order of questions in face-to-face interviews but found no differences in responses. Martin and McConnell (1970) examined the effects of order of presentation on return rates. Two questionnaires were given, one more difficult than the other; higher return rates were obtained when the easier one was given first. None of these studies considered the possibility that there might be differences between the return rates of questionnaires whose topics are arranged logically, based on their relevance to