Nonparametric Bootstrap and Small Area Estimation to Mitigate Bias in Crowdsourced Data

Abstract

Open and crowdsourced data are becoming prominent in social sciences research. Crowdsourcing projects harness information from large numbers of citizens who voluntarily participate in one collaborative project, and allow new insights into people's attitudes and perceptions. However, these data may be affected by a series of biases that limit their representativeness (i.e. self‐selection bias, unequal participation, underrepresentation of certain areas and times). In this chapter we present a two‐step method aimed to produce reliable small area estimates from crowdsourced data when no auxiliary information is available at the individual level. A nonparametric bootstrap, aimed to compute pseudo‐sampling weights and bootstrap weighted estimates, is followed by an area‐level model‐based small area estimation approach, which borrows strength from related areas based on a set of covariates, to improve the small area estimates. To assess the method, this research team conducted a simulation study and an application to safety perceptions in greater London. The simulation study results show that the area‐level model‐based small area estimator under the nonparametric bootstrap improved (in bias and variability) the small area estimates in the majority of areas. The application produced estimates of safety perceptions at a small geographical level in greater London from Place Pulse 2.0 data. In the application, estimates are validated externally by comparing these to survey estimates. Further simulation experiments and applications are needed to examine whether this method also improves the small area estimates when the sample biases are larger, smaller, or show different distributions. A measure of reliability also needs to be developed to estimate the error of the small area estimates under the nonparametric bootstrap.