Multimodal spoken dialog system using state estimation by body motion

Abstract

Spoken dialog systems are presently used widely. However, some users avoid using them because of poor usability and unattractiveness. In this study, we develop a system that captures the user's movement and estimates the user state. This function is incorporated into the existing spoken dialog system to build a multimodal dialog system. In the experiments, the recognition performance of body motion is evaluated with five subjects. The results show that most actions could be recognized in real time with an accuracy of over 60%. The motivation for use is expected to be improved by the proposed system.