Causes and consequences of non-random drop-outs for citizen science projects: lessons from the North American amphibian monitoring program

Abstract

Citizen science holds great promise for collecting useful environmental data over large spatial scales. However, statistical issues that arise in the analysis of citizen science data may be relatively unfamiliar to scientists accustomed to data collected with traditional research methods. In particular, citizen science projects are often designed with standard randomization procedures, but volunteers may drop-out of a project in a highly non-random manner. For example, if volunteers are less likely to continue monitoring sites that are highly urbanized or polluted, these sites will be under-represented in analyses, and observed patterns could be biased accordingly. We tested for non-random drop-outs in the context of the North American Amphibian Monitoring Program (NAAMP), a road-based, citizen-science survey of calling frogs and toads. We found that discontinuation of survey routes by NAAMP volunteers was associated with high traffic volume, high noise levels, and low forest cover along these routes. The absolute increase in probability of dropping out of the program that was associated with these factors was often low (e.g., 2–10%), but much larger increases in drop-out probabilities (e.g., 40–70%) were predicted when traffic or noise were particularly high or when multiple factors were considered simultaneously. In addition, analysis of amphibian count data suggested that relatively low counts of amphibian and low species richness were also associated with increased probability that survey routes would be discontinued. Together, these non-random drop-outs led to the decreased representation of highly urbanized sites in our data set, and may have altered the estimated relationships between explanatory variables (e.g., traffic, forest cover) and amphibian species richness. Our results, therefore, suggest that citizen science projects need to be designed after careful consideration of the factors that promote retention of volunteers and the effects that non-random drop-outs may have on the data they generate. Stratification that takes non-random drop-outs into account may be necessary to ensure adequate representation of some kinds of survey sites in citizen science projects.