Thou shalt not cheat: how to reduce internet use in web surveys on political knowledge

Abstract

By means of a split-ballot survey experiment, we study whether a normative instruction not to use the internet when answering political knowledge questions reduces cheating in web surveys. The knowledge questions refer to basic facts about the European Union and the data come from the Italian National Election Study web panel carried out in Italy before the 2014 European Election. Our analysis shows that a simple normative instruction significantly reduces cheating. We also show that reducing cheating is important to achieve a correct assessment of reliability of knowledge scales, while a decrease of cheating leaves unaltered the knowledge gap between lower and higher educated respondents. These results invite caution when including political knowledge questions in an online survey. Our advice is to include a normative instruction not to search the internet to reduce cheating and obtain more genuine answers. More generally, we conclude by stressing the need to consider the implications of online data collection when building questionnaires for public opinion research.