Multisensory HCI Design with Smell and Taste for Environmental Health Communication

Abstract

In the field of Multisensory Human-Computer Interaction (HCI) Design, integration of chemical senses has been increasingly explored. Nevertheless, smell and taste senses are still underrepresented in HCI. This underrepresentation limits Environmental Health Communication, which should incorporate those senses. The exploratory study presented in this paper describes Multisensory HCI design process with the chemical senses as mediators of environmental health communication. To do so we formulated design hypothesis and divided our research design process into two phases: the chemical sense experience for cross-sensory analogies research and the digital media experience for meaning and communication research. Thereby, we describe the conceptualization, design and evaluation of our design project Earthsensum. Our approach accomplished symbolic displaying of smell and taste and led us to a new multi-sensorial interaction system using Mobile Virtual Reality (MVR) and Mobile Augmented Reality (MAR). The results of our findings confirmed our design hypothesis, and showed that the purposed interaction system lead not only to a better understanding of smell and taste perception, as well as of environmental challenges. We discuss our results in the context of the HCI design strategies for chemical senses inclusion.