A quality framework for multimodal interaction in educational environments

Abstract

Designing gesture-based collaborative and interactive systems using new technologies has shown noticeable potential for educational purposes and could enhance the memorability of digital contents. In this paper, we present a quality framework and a new combination of modalities to recognise different types of human gestures. The outcome can be used in future classroom designs by replacing expensive smartboards which are physically limited to a fixed place. We also propose a framework based on Norman's theory of action for interactive collaborative systems. This framework has been tested by a combination of Microsoft Kinect and a smartphone's built-in gyroscope together with two types of interfaces in an educational environment. The results proved the improvement of participant's interactivity and participation by changes in five pedagogical factors in different groups of the study and specifically in higher ages.