Abstract
The design and evaluation of multimodal interaction is difficult. For designers in industry, developing multimodal interaction systems is a big challenge. Although past researches have presented various methodologies, they have addressed only specific cases of multimodality and failed to generalise their methodologies to a range of applications. In this paper, we present a usability framework for the design and evaluation of multimodal interaction. First, in the early phase of multimodality design, elementary multimodal commands are elicited using traditional usability techniques. Second, based on the CARE (Complementarity, Assignment, Redundancy, and Equivalence) properties and the FSM (Finite State Machine) formalism, the original set of elementary commands is automatically expanded to form a more comprehensive set of multimodal commands. Third, this new set of multimodal commands is evaluated in two ways: user-testing and errorrobustness evaluation. This framework acts as a structured and general methodology both for designing and evaluating multimodal interaction. We expect that it will help designers to produce more usable multimodal systems.
Highlights
The design and evaluation of multimodal interaction is difficult
Because it is not possible for designers to manually verify the whole set of multimodal commands, our usability framework relies on the FSM formalism to automate the verification process
We presented a usability framework to design and evaluate multimodal interactions
Summary
The design and evaluation of multimodal interaction is difficult. The appropriateness of multimodal choice is influenced by many factors such as the nature of the tasks and the various user contexts. Comprehensive multimodal corpora, which reflect diverse user circumstances and could help selecting appropriate combinations of modalities, are not yet available [11]. As multimodal systems tend to utilise natural channels of communication, they are widely exposed to human errors and ambiguity. Building error-robust multimodal interaction models, which enable mutual disambiguation, is not trivial. We present a usability framework for the design and evaluation of multimodal interaction, which provides designers with a practical methodology to overcome the above issues. We describe how FSMs (Finite State Machines) can help us model various multimodal commands.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
