Evaluating citizen science data: A citizen observatory to measure cyclists’ waiting times

Abstract

Citizen science allows mobility researchers and mobility planners to collect data at a larger scale and at lower costs compared to traditional data collection methods. Citizen observatories (COs) are especially interesting because public authorities are often involved in the setup and the participation barrier for citizens is reduced by relying on smartphones. Citizen science and COs both foster public participation in scientific research. An evaluation framework can increase the quality of and the trust in citizen science data. Although considered important by existing evaluation frameworks for citizen science, detailed instructions on how to evaluate data accuracy are lacking. This is especially the case in domains where citizen science has not been widely used, such as mobility. This paper therefore presents a framework to evaluate the data quality of a CO for mobility. The framework includes representativeness, accuracy, reliability, and validity. The framework is demonstrated by applying it to a CO in Ghent, Belgium, in which cyclists used their smartphones to collect their perceived and objective waiting times at signalised intersections. Although the representativeness of the data could be improved, the data was found to be accurate. Data showed that participants on average spent 4% of their trip time waiting at signalised intersections. Furthermore, no clear overestimation or underestimation of the waiting time was found. The main limitation of the evaluation framework is the focus on data quality, which is only one aspect of citizen science.