Touch and Speech: Multimodal Interaction for Elderly Persons

Abstract

AbstractThis paper reports our work on the development and evaluation of a multimodal interactive guidance system for navigating elderly persons in hospital environments. A list of design guidelines has been proposed and implemented in our system, addressing the needs of designing a multimodal interfaces for elderly persons. Meanwhile, the central component of an interactive system, the dialogue manager, has been developed according to a unified dialogue modelling method, which combines the conventional recursive transition network based generalized dialogue models and the classic agent-based dialogue theory, and supported by a formal language based development toolkit. In order to evaluate the minutely developed multimodal interactive system, the touch and speech input modalities of the current system were evaluated by an elaborated experimental study with altogether 31 elderly. The overall positive results on the effectiveness, efficiency and user satisfaction of both modalities confirm our proposed guidelines, approaches and frameworks on interactive system development. Despite the slightly different results, there is no significant evidence for one preferred modality. Thus, further study of their combination is considered necessary.KeywordsMultimodal interactionElderly-centered system designHuman-computer interactionSpoken dialogue systemsFormal methods