Rethinking User Interaction with Smart Environments—A Comparative Study of Four Interaction Modalities

Abstract

Smart environments, comprised of networked embedded devices, improve the lives of their users by providing them with a variety of assistive services that traditional built environments are incapable of supporting. However, as the number of connected devices in smart environments continue to increase, so does the level of complexity involved in interacting with these environments. Traditional human-computer interaction techniques are not always well-suited for smart environments and this poses some unique usability challenges. To facilitate interactions within such technology-rich smart environments, new models and interaction interfaces need to be developed. In this paper we propose a multi-modal approach to smart environment interaction and explore two novel interaction interfaces: gesture-based interface and mixed-reality-based interface. We also conducted a user study to compare the learnability, efficiency and memorability of these new interfaces to two more commonly used interfaces: voice-based interface and a smartphone GUI-based interface. Our user study experiment involved four light control tasks that subjects were asked to complete using the four interaction interfaces. Study subjects found different interaction techniques to be more suitable for different tasks based on the type, complexity and context of the task. Our analysis of the study results and subject feedback suggest that a multi-modal approach is preferable to a uni-modal approach for interacting with smart environments. We suggest that novel interaction techniques be further explored in order to develop efficient multi-modal approaches along with the widely used techniques.