Multimodal intention understanding method based on NHMM

Abstract

This paper proposes a multimodal intention understanding method based on NHMM. Due to the declining expression ability of the elderly, single-mode input during human-computer interaction may reduce the accuracy of robot intention understanding and cannot meet the requirements of helping the elderly. Therefore, in the aspect of intention understanding, this paper uses multi-modal input of voice, gesture and posture to obtain the user’s multi-dimensional information as much as possible. Each modal information is processed to obtain three intention sets, and then fused to obtain the fused intention set, By combining the improvement of the core formula of hidden Markov model (HMM) with Bayes, the intention with the highest probability in the fusion intention set is calculated as the result of intention understanding. The results show that the accuracy of this method is 97.7% in the state of user natural interaction, and compared with the other four intention understanding methods, this method is more effective for intention extraction under fuzzy input.