A 3D computer game for testing perception of acoustic detail in speech

Abstract

This paper presents an experimental framework for acoustic perception studies, including an example application which tests attention to fine phonetic detail in natural speech. Traditional perception experiments in speech research do not employ natural testing scenarios, and they are notorious for being perceived as boring and repetitive by test subjects. This contribution presents a solution to the problem, by employing a computer game in which perception of the acoustic aspect to be tested is crucial for success in the game. The game is designed as a first person shooter. The player moves around within a virtual 3D environment and reacts to agents which belong to two different categories, each of which is associated with one response key. The two categories are initially distinguished by visual and acoustic cues, and the acoustic cues are designed to differ only in the perceptual aspect to be tested. Gradually, visual cues are removed. Thus, players with better perception of the aspect score higher in the game. A strong benefit of this framework is the high involvement and motivation of the subjects to perform well in the game. Practical and theoretical challenges encountered with the implementation of an attention test within the game are discussed.This paper presents an experimental framework for acoustic perception studies, including an example application which tests attention to fine phonetic detail in natural speech. Traditional perception experiments in speech research do not employ natural testing scenarios, and they are notorious for being perceived as boring and repetitive by test subjects. This contribution presents a solution to the problem, by employing a computer game in which perception of the acoustic aspect to be tested is crucial for success in the game. The game is designed as a first person shooter. The player moves around within a virtual 3D environment and reacts to agents which belong to two different categories, each of which is associated with one response key. The two categories are initially distinguished by visual and acoustic cues, and the acoustic cues are designed to differ only in the perceptual aspect to be tested. Gradually, visual cues are removed. Thus, players with better perception of the aspect score higher in t...