Should we use email for farm surveys? A comparative study of email and postal survey response rate and non-response bias

Abstract

Many agricultural researchers are now turning away from the traditional postal surveys to email surveys of farmers – an option that is increasingly viable as digitalisation continues to permeate rural areas. However, email surveys often result in considerably lower response rates. This raises questions about the potential of email surveys to experience non-response bias, where the survey methodology excludes particular sectors of the general population and thus results in responses that do not represent the wider population. In this paper we address the issue of whether agricultural researchers should move from postal surveys to email surveys by comparing the results of two applications of the Norwegian national Trends survey – one to 3000 farmers via email and one to 3000 farmers via standard mail. The postal survey achieved a response rate of 41.1% – almost double that of the email survey at 21.4%. However, analysis of the returns suggested this had not led to greater non-response bias in the email survey. While respondents to the email survey were younger, better educated and more likely to be part-time farmers, comparing the entire survey revealed very few significant differences between the two samples. Where the difference was significant (in particular, attitudes towards technology), the scalar difference was so small that using different survey methods would not have led to different conclusions. Although there was no evidence that the low response rate compromised the email survey, we conclude that postal surveys may still be preferable because (a) there is less scope for non-response bias, and (b) having to double the gross survey size to achieve a sufficient sample size may create additional survey fatigue in the long term. We discuss the applicability of the findings to farm surveys in other countries.