Response Format Effects in Questions about Norms: Implications for the Reliability and Validity of the Normative Approach

Abstract

Several studies have demonstrated that altering the format (question wording or presentation format) can alter responses obtained from normative questions. This study extends that research with two experiments. In a study of McKenzie River boaters in Oregon ( n = 225), respondents received either a semiopen (fill in the blank) or closed (scale with anchored points) format of questions about acceptable waiting time at boat launches, percent of time spent in sight of other boaters, and number of groups met per day. In the second study, at Okefenokee National Wildlife Refuge, 774 respondents received either the semiopen format or one of two closed formats (anchored with 30 or 60), asking for the acceptable number of motorboats and canoes. In both cases, norm prevalence was significantly higher in the closed format (46 to 78%) than in the semiopen format (27 to 60%), and the average increase from one format to the other was 23%. This suggests either that respondents are more likely to guess at a number when given a closed response frame or that the closed format provides important context that elicits valid responses. Examination of the mean values obtained under the different response formats suggest that both processes may be operating, but the latter is more likely. It appears that closed formats, like photographs, offer a cognitively easier and more meaningful task.