Virtual Agent Learning for NUI/NUX-Based Games

Abstract

Natural User Interface (NUI) embodies the concept of direct interaction with computers using the human body instead of using existing (intermediary) input devices. Further, Natural User eXperience (NUX) incorporates the concept of more natural and intuitive feedback in accordance with the user’s command. This paper proposes an approach that enables virtual agents to interact with users playing games on the basis of NUI. Users have a more immersive gaming experience as a result of the NUX-based interactive virtual agent and the use of NUIs in games. To enable the virtual agent to learn automatically via interaction, we generate a virtual human using imitation learning. Then we apply Q-learning to enable the virtual agent to learn in a short time. The results of two experimental scenarios confirm that our proposed method is sufficiently able to instruct a virtual agent to support a virtual human.