Agreement by design: The effect of visual harmony on responses to surveys

Abstract

AbstractThe present study is one of the first to empirically examine how the visual harmony of a questionnaire can lead to measurement bias. Researchers often employ questionnaires with Likert scales to measure constructs. In this note, we examine how the design of the survey instrument, specifically, its visual harmony, can impair measurement accuracy. Two studies investigate effects of visual harmony in surveys on responses to Likert scales using paper and pencil surveys. Applying an established customer relationship management model, Study 1 employs a survey of female visitors to a grocery store (n = 115). Switching to a product and brand innovation context, Study 2 employs a survey of male and female members of a consumer panel (n = 180) to examine responses to a new e‐scooter. Across studies, results indicate that assessing important consumer response constructs through visually more harmonious surveys can lead to more positive response patterns, lower scale reliability, and questionable validity, especially with females. Although these effects do not occur uniformly across measures and samples, they occur regardless of consumers' past experience with completing questionnaires, their familiarity with questionnaire design, and the naturalness and elaborateness of the visual design. Relating specific elements (e.g., text boxes, type font, shapes, and images) and relational properties of design (e.g., balance, symmetry, and coherence) to consumers' overall perception of harmony aids marketers and researchers in achieving intermediate levels to obtain realistic, reliable, and valid results.