Crowdsourcing listening tests by using polling stations

Abstract

Listening tests are used in various sound-related research areas. Such tests generally require complex and controlled protocols that involve consequent manpower, demanding preparation and a limited number of participants. Furthermore, big data analytics can hardly be applied using such data gathering schemes. Several options exist to overcome such limitations: web-based listening tests, mobile phone-based applications or even mobile laboratory units; each of them has specific advantages and disadvantages. In this communication, the relevance of using polling stations as a tool for crowdsourcing perceptive evaluations of sounds and involving the general public into this process is under consideration. Such a polling station was installed for popularization and research purposes in a touring science exhibition related to sound. Visitors could listen to various sounds with headsets and provide their feedback concerning different topics (including perceived sound quality of vehicles, quality of speech and audio coding technologies and perception of warning sound alarms). A first series of over five thousand votes was collected over a six- week period during the first presentation of the exhibition in Sherbrooke, Quebec, Canada. The results of crowdsourced responses were compared to laboratory results previously obtained in controlled laboratory conditions with a limited number of participants (<20), and evaluation scores were found to be comparable. The presentation will illustrate the setup of the polling station, discuss the results and draw perspectives linked to this exploratory study.