Developing a Multimodal Entertainment Tool with Intuitive Navigation, Hands-Free Control, and Avatar Features, to Increase User Interactivity

Abstract

Advances in user interactivity in digital entertainment offer the potential to engage users beyond traditional passive and solitary experiences. Indeed, a high level of interactivity is inherent in tasks involving cooking, auto repair, and home improvement, all of which require users to complete multiple, detailed, and interdependent steps. Such tasks also require access to visual and audio instructions while a user's hands are engaged in a primary, physical task, and are often conducted in unique locations, e.g., kitchen or garage. This effort describes the design of an interactive, multimodal digital entertainment user experience for an ‘edutainment’ cooking show. The prototype wireframes incorporate three novel features, identified through requirements gathering and iterative design, of an interactive recipe map for hierarchical content navigation, voice command hands-free control, and avatars to further engage users. The overall design and three features were evaluated via usability testing in real kitchen settings in the conduct of actual cooking with real ingredients with a diverse range of seven participants. The results illustrate that the hierarchical navigational feature, alongside interactive voice communication, were effective at reducing users' cognitive load, and streamlined necessary information in order to support task completion.