The Mode Effect in Mixed-Mode Surveys

Abstract

Web surveys can suffer from their nonrandom nature (coverage error) and low response rate (nonresponse error). Therefore, web surveys should be supported by mail survey to eliminate these problems. However, using different survey methods together may introduce another problem: the mode effect. This experimental study investigated the mode effect between two survey modes. A randomly selected group of 1,500 teachers were assigned to two experimental groups, one of which received mail surveys, while the other received web surveys. Nonrespondents in both groups were followed up with the opposite mode. Overall, results show that there is no mode effect between mail surveys and web surveys on psychometric quality of the rating scales and data quality (item nonresponse rate) of the survey except regarding respondents’ age and unit-response rate. Our findings indicate that web surveys had a substantially lower unit-response rate than mail surveys and that web survey respondents are significantly younger than mail survey respondents.