Multimodal dialogue system MultiksDial

Abstract

AbstractThis paper describes the multimodal dialogue system MultiksDial, which focuses on the speech input/output. In this system, the speech recognition unit and the touch panel are used as the means of input, and the Text‐To‐Speech (TTS) synthesizer and the display are used as the means of output. In other words, the features of the system are that both the input and the output are multimodal. As the auxiliary means of input, a photoelectric sensor is used to realize a smooth dialogue by monitoring the behavior of the user and by guiding the operation. An information guide system is implemented on MultiksDial, and the operability of the user interface is evaluated.The following characteristics of the speech input are demonstrated as the result of the comparison experiment of the input mode. The direct indication by the speech input can execute the operation faster than the touch input, where the stepwise indication is required. Another point is that the beginner can execute? smooth dialogue by guiding the operation by the synthesized speech. By those results, it is verified that the multimodal implementation of the dialogue channel is useful in realizing an efficient dialogue between the user and the system.