Logos and Inserts can Reduce Survey Return Rates: An Experiment in California

Abstract

Survey methodologists seek techniques that make the mail they send more likely to be opened, completed, and returned than missed, ignored, or thrown away. Traditional design wisdom recommends against using color and logos, which can make the delivery envelope look like marketing, fundraising, and other “junk mail.” Yet there have been few experimental tests of this assertion. Theoretically, a distinctive graphic representing a recognizable and beloved sponsor, combined with a prominent motivational message should serve dual roles of a) distinguishing surveys from marketing, advertising, fundraising, and b) encourage the respondent to participate. In this experiment, a random half of envelopes had a sponsor logo on the outside and a motivational insert affixed to the cover letter. The response task was a one-page screener form asking for basic household information and a telephone number for a subsequent telephone survey. The experiment used a random sample of addresses from the USPS Delivery Sequence File (DSF) in two California communities with hard-to-survey characteristics. The logo envelope received about a 1.3 percentage point lower return rate in the first mailing in which it was used, but not in the second mailing. Further, that difference was only significant in one of the two communities (a 2.8 point difference in that community v. 0.7 points in the other). Traditional advice about avoiding logos in mail surveys seems sound, but effects are not universal. Comparing experimental and control conditions at their first use, forms returned from logo/insert packets were less likely to report adults age 41 years old and over (70.8% v. 78.2% in non-logo/non-insert mailings). In their second use, logo/insert packets were more likely to reporting at least one adult 18-40 in the household (76.8% v. 66.9% for non-logo/non-insert mailings). Suggestions for future research are discussed.