Evaluation of an automated speech-controlled listening test with spontaneous and read responses

Abstract

A method for an automated system for speech audiometry is introduced and evaluated using pre-recorded responses as well as spontaneous utterances produced by listeners during a real measurement. A hearing test is performed under the use of automatic speech recognition (ASR) based on the matrix sentence test, which is used clinically for diagnostics and fitting of hearing devices as well as in psychoacoustic research. The test measures the speech reception threshold (SRT), i.e., the signal-to-noise ratio at which the subject achieves 50% word recognition rate. A major disadvantage of current testing procedures is the requirement of a human expert supervising the test and logging the listener’s responses. An automated system reduces the required resources and therefore provides a tool for frequent assessment of the SRT, which can contribute to an early diagnosis of hearing loss. The accuracy of the ASR-based SRT measurement is compared to results obtained with a human supervisor. To this end, two databases are used that contain either well-controlled read utterances that resemble typical responses during SRT measurements produced by 17 speakers, or spontaneous responses collected during real SRT measurements using ASR. Twenty normal-hearing and seven slightly to moderate hearing-impaired subjects participated in the collection of this spontaneous speech. In order to assess the SRT accuracy for read speech, two simulation schemes are proposed that employ Monte Carlo tests to simulate a listener’s profile and corresponding responses, which are validated with the real measurement data. We show that ASR deletion rates of 0.9% and insertion rates of 2.9% for matrix text words are sufficiently low to obtain accurate SRT measurements in the range of 0.5 dB SNR. This is comparable to the test-retest accuracy obtained by human supervisors. While ASR errors are overestimated when using the controlled speech material in comparison to spontaneous speech, this error type has minimal effect on SRT estimation. Hence, the use of pre-recorded, read speech material is sufficient when evaluating the accuracy of speech-controlled, automated listening tests.