CREAME: CReation of Educative Affordable Multi-surface Environments

Abstract

Collaborative serious games have a positive impact on behavior and learning, but the majority are still being developed for traditional technological platforms, e.g., video consoles and desktop/laptop computers, which have been deemed suboptimal for children by several studies. Instead, the use of handheld devices such as tablets and smartphones presents several advantages: they are affordable, very widespread, and mobile---which enables physical activity and being able to engage in a game without requiring users to gather around a fixed, dedicated, location. Plus, combining several of these devices and coordinating interactions across them in what is called a Multi-Display Environment (MDE) brings on additional benefits to collaboration like higher scalability, awareness, parallelism, and fluidity of the interaction. How to interact with these multi-tablet environments is therefore a critical issue. Mobile devices are designed to be interacted mainly via touch, which is very straightforward but usually limited to the small area of the displays, which can lead to the occlusion of the screen and the underuse of the peripheral space. For this reason, this thesis focuses on the exploration of another interaction mechanism that can complement touch: tangible around-device interactions. Tangible interactions are based on the manipulation of physical objects, which have an added value in childhood education as they resonate with traditional learning manipulatives and enable the exploration of the physical world. On the other hand, the exploitation of the space surrounding the displays has several potential benefits for collaborative-learning activities: reduced on-screen occlusion (which may increase workspace awareness), the use of tangible objects as containers of digital information that can be seamlessly moved across devices, and the identification of a given student through the encoding of their ID in a tangible manipulator (which facilitates the tracking of their actions and progress throughout the game). This thesis describes two different approaches to build collaborative-learning games for MDEs using tangible around-device interactions. One, called MarkAirs, is a mid-air optical solution relying on no additional hardware besides the tablets except for several cardboard printed cards. The other, Tangibot, introduces a tangible-mediated robot and other physical props in the environment and is based on RFID technology. Both interactions are respectively evaluated, and it is observed that MarkAirs is usable and undemanding both for adults and for children, and that fine-grained gestures above the tablets can be successfully conducted with it. Also, when applied to collaborative games, it can help reduce screen occlusion and interference among the different users' actions, which is a problem that may arise in such settings when only touch interactions are available. A collaborative learning game with MarkAirs is evaluated with primary school children, revealing this mechanism as capable of creating collaborative learning experiences and presenting an added value in user experience, although not in performance. With respect to Tangibot, we show how collaboratively controlling a mobile robot with tangible paddles and achieving certain precision with it is feasible for children from 3 years of age, and even for elderly people with mild cognitive impairment. Furthermore, it provides a fun experience for children and maintains them in a constant state of flow.