Speaker-independent spontaneous speech dialog system TOSBURG II using multimodal response and speech response cancellation

Abstract

This paper describes the speaker-independent spontaneous speech dialog system TOSBURG II, where the user can input the speech interrupting the speech response from the system. In the speech dialog system TOSBURG, which was previously constructed experimentally, spontaneous utterance dialog was realized by a continuous speech understanding based on the key word and the multimodal response. TOSBURG II aims at dialog that is more flexible and interactive. The speech response component superposed on the input speech signal is cancelled by the active control technique, making it possible for the user to interrupt and input during the system speech response. To improve the precision of speech recognition, noise immunity learning, which is useful in detecting the keyword in the noise, is extended so that it can also be applied to the speech input after the speech response cancellation, where the residual component still remains. To evaluate the effectiveness of speech response cancellation and noise immunity learning, an experiment is executed by superposing the speech data (obtained by the utterance of 350 sentences containing 49 key words by five adult males) on the system speech response. The key word detection rate, false alarm rate, and sentence recognition rate are improved from 58.8 percent, 24.1 FA/H/W (false alarms/hour/word), 27.8 percent, to 89.6 percent, 15.4 FA/H/W, 58.4 percent, respectively. A dialog experiment is also conducted using the constructed realtime system, and the usefulness of the proposed system is verified.