Data-driven methods in a model-driven world: Making sense of auditory classification images for continuous speech

Abstract

Some of my recent work has focused on the development of Auditory Bubbles, a method for generating classification images that show the acoustic features underlying different percepts of continuous speech. Auditory Bubbles work by applying randomly shaped spectrotemporal filters to the modulation power spectrum of sentence-level speech stimuli. A reverse correlation is then used to relate the filter patterns to a behavioral outcome (e.g., keyword recognition). This technique is data driven—i.e., a computational approach (regression) is taken to estimate the relation between stochastically varying acoustic-speech features and listener responses (the classification image). For a complex stimulus such as continuous speech, this relation may be determined by multiple underlying factors—e.g., task demands, listener characteristics, response strategies, and perceptual acuity. As such, it can be difficult to interpret classification images, which (blindly) reflect the combined influence of these underlying factors. In this talk, I present the classification images from multiple speech experiments comparing (a) different stimuli (single- versus multi-talker), (b) different tasks (intelligibility versus emotion perception), and (c) different listener groups (persons with versus without hearing loss, musicians versus non-musicians) and discuss preliminary efforts to constrain the interpretation of classification images by appealing to the tenets of signal detection theory.