Determining the Accuracy of Crowdsourced Tweet Verification for Auroral Research

Nathan A Case,Nick Lalone,Andrea Tapia,Elizabeth A Macdonald,Sean Mccloat

doi:10.5334/cstp.52

Abstract

The Aurorasaurus project harnesses volunteer crowdsourcing to identify sightings of an aurora (the “northern/southern lights”) posted by citizen scientists on Twitter. Previous studies have demonstrated that aurora sightings can be mined from Twitter with the caveat that there is a large background level of non-sighting tweets, especially during periods of low auroral activity. Aurorasaurus attempts to mitigate this, and thus increase the quality of its Twitter sighting data, by using volunteers to sift through a pre-filtered list of geolocated tweets to verify real-time aurora sightings. In this study, the current implementation of this crowdsourced verification system, including the process of geolocating tweets, is described and its accuracy (which, overall, is found to be 68.4%) is determined. The findings suggest that citizen science volunteers are able to accurately filter out unrelated, spam-like, Twitter data but struggle when filtering out somewhat related, yet undesired, data. The citizen scientists particularly struggle with determining the real-time nature of the sightings, so care must be taken when relying on crowdsourced identification.

Highlights

The citizen science project Aurorasaurus (MacDonald et al 2015) has two main goals: Improving the “nowcasting” of a visible aurora and the ability to accurately model both the size and strength of an aurora
Like many citizen science projects, Aurorasaurus is h eavily reliant upon a community of volunteers for providing data and for validating/classifying data
To complement the aurora sightings reported directly to the project, Aurorasaurus systematically searches for o bservations of an aurora posted on Twitter, using the Twitter Search API and several rudimentary filters

Summary

Introduction

The citizen science project Aurorasaurus (MacDonald et al 2015) has two main goals: Improving the “nowcasting” of a visible aurora (commonly known as the “northern/ southern lights”) and the ability to accurately model both the size and strength of an aurora. Previous studies have shown that Twitter users, who post short updates (of a maximum 140 characters in length) known as “tweets,” will often share details about the conditions around them This is especially true for largescale events such as earthquakes (Earle et al 2010; Crooks et al 2013), influenza outbreaks (Culotta 2010; Lampos et al 2010), and service outages (Motoyama et al 2010). Case et al (2015a) showed that Twitter can be a useful source of data for studying the aurora by comparing the number of tweets relating to an aurora with auroral activity (or, to common auroral activity indices) These authors noted that Twitter data are noisy and that many tweets containing aurora-related keywords (e.g., “aurora” and “northern lights”) are not sightings. Often such tweets are about a person or place or the desire to witness an aurora

Results

Discussion

Conclusion