Formal framework and necessary properties of the fusion of input modes in user interfaces

Abstract

Multiple input devices are increasingly used in user interfaces to make human-computer communication more efficient and effective. Interface designers have not only to decide on which input modes should be supported, but also how to fuse them into a single representation format that can be processed by the underlying application system. Drawing appropriate decisions requires, however, a sufficient understanding of the properties of fusion itself. While others have informally characterized input fusion as a transformation between information types, the purpose of the paper is to explore fusion by means of formal process modelling. That is, fusion processes are defined in a formal framework which supports proof of the existence of necessary properties following directly from the process definitions. The presented approach can be applied to analyse and compare fusion processes in existing systems, as well as an aid for interface designers, who have to verify the behaviour of their systems.