Speech visualization and its application for the hearing impaired

Abstract

A speech‐visualization system has been developed as an extension of research on a color display system for connected speech [Watanabe et al., IEEE Trans. Acoust. Speech Signal Process. ASSP‐33, 164–173 (1985)]. In this newly developed system, normalized spectrographic patterns are overlapped as a change of luminance on a whole pattern that represents voiced segments by colors. The spectrographic patterns are represented with high luminance in unvoiced portions, and with low luminance in voiced ones so as not to disturb the color image. This extension allows unvoiced plosives as well as fricatives and affricates to be clearly visualized by the sharp contrast between their black‐and‐white patterns and the vivid colors of the voiced components. Moreover, in order to generate a visually clear image, a cleaner, which eliminates unnecessary patterns such as aspirates that are often observed at the end of utterances, is introduced. Finally, two applications of this system are described. One is a speech training system in which a personal computer, with a database, controls the speech‐visualization system and provides a hearing aid. The other is a telecommunication system that utilizes a public telephone channel. [This work was supported by TAF.]