Influence of environmental background noise on speech quality assessments task in crowdsourcing microtask platform

Abstract

It is important to realize in which environments a speech quality evaluation test can be carried out to achieve reliable results outside the laboratory. We report on our current activity on using microphone signals for evaluating environmental conditions in crowdtesting. In order to analyze the impact of environmental noise, a two-phase experiment is conducted using a mobile crowdsourcing platform. A speech quality assessment task is used with stimuli from the SwissQual 501 database from the ITU-T Rec. P.863 competition (kindly provided by SwissQual AG). The first phase of the experiment is conducted in a laboratory, in which participants are assigned to either silent or background noise simulation (with Cafeteria and Road Noise) study groups. In each case 5 seconds of environmental sound are recorded by the participants before and after performing the task. Next, the same participants were asked to perform the same task in a real living room, cafeteria, or any other environment (and proofing it by taking photos) and again to record the environmental noise. Environments are labelled based on their influence on the result of speech quality assessments. Later, different features like loudness, noisiness, etc. will be extracted from the recorded audio files and used to predict if the environment is suitable for performing a crowdtesting speech quality assessment.