Multi-touch Tabletop Research Articles

Computerized collaboration scripts and real-time intergroup competition for enhancing student collaboration and learning with multi-touch tabletop displays

Computerized collaboration scripts and real-time intergroup competition for enhancing student collaboration and learning with multi-touch tabletop displays

Visual Signifier for Large Multi-Touch Display to Support Interaction in a Virtual Museum Interface

The signifier is regarded as a crucial part of interface design since it ensures that the user can manage the device appropriately and understand the interaction that is taking place. Useful signifiers keep users’ attention on learning, but poorly designed signifiers can disrupt learning by slowing progress and making it harder to use the interface. The problem is that prior research identified the qualities of signifiers, but their attributes in terms of being visually apparent in broad interaction areas were not well recognized. Implementing the signifier without sufficient visual features such as a picture, figure or gesture may interfere with the user’s ability to navigate the surface, particularly when dealing with domains that demand “leisure exploration,” such as those in culture and heritage, and notably the museum application. As technology has evolved and expanded, adopting a multi-touch tabletop as a medium of viewing should be advantageous in conserving cultural heritage. As technology advances and improves, employing a multi-touch tabletop as a public viewing medium should be advantageous in maintaining cultural heritage. Some visual elements should be incorporated into the signifier to produce a conspicuous presentation and make it easier for users to identify. In this study, a preliminary study, a card sorting survey, and a high-fidelity experiment were used to investigate users’ experience, perspective, and interpretation of the visual signifier of the museum interface for large displays. This work offered a set of integrated visual signifiers on a big multi-touch display that makes a substantial contribution to supporting navigation and interaction on a large display, therefore aiding comprehension of the exhibited information visualization.

Immersive Spring Morning in the Han Palace: Learning Traditional Chinese Art Via Virtual Reality and Multi-Touch Tabletop

ABSTRACT We examine the user experience, learning effectiveness, learning behaviors, and learning motivation of 54 young adults in pairs using an immersive virtual reality (IVR) environment and a multi-touch tabletop (MTT) to gain a cultural-historical understanding of the traditional Chinese paintings, Spring Morning in the Han Palace and The Night Revels of Han Xizai respectively. We used mixed methods of knowledge assessments, questionnaires, observation, and interviews to collect and analyze data. The results reveal the IVR environment significantly increased the learning effectiveness and motivation compared to the MTT system, particularly in tasks related to recall of details or spatiality. The IVR design incorporates five learning scenes, and as part of this aspect, the qualitative results indicate users in the IVR condition (1) developed a variety of exploratory and embodied learning strategies; (2) quickly switched their attention to the central learning content, and (3) had equitable interactions in collaborative learning. This research contributes to the potential of IVR learning effectiveness and associated design considerations to learn traditional cultural heritage.

STEAM Education Using Sericulture <i>Ukiyo-e</i>: Object-Based Learning through Original Artworks Collected at a Science University Museum in Japan

Societies across the world face the need to increase the number of science, technology, engineering, and mathematics (STEM) personnel who could resolve issues concerning sustainability. The body of scholarly literature on STEM education has grown significantly since the beginning of the 21st century. STEAM education incorporates arts with STEM subjects. This pedagogical strategy is increasingly adopted in formal/informal education settings to foster scientific thinking and creativity through multisensory artistic activities. Science museums and centers house discrete items such as multi-touch tabletops that allow visitors to handle objects and engage in experiential activities. This study postulates that science university museums can contribute to STEAM education. It illustrates the effectiveness of object-based learning (OBL) in STEAM education imparted through original artwork housed in science university museums. This study’s application of OBL at university museums establishes OBL within cross-disciplinarySTEAM-based educational systems in Japan through the use of original artwork such as sericulture ukiyo-e. Students majoring in agriculture or technology participated in the workshop designed for this study, directly engaged with discrete multisensory sericulture ukiyo-e, and were encouraged to think from diverse perspectives. Finally, the studentsdebated the sericulture ukiyo-e and organized an exhibition. This study concludes that the academic value of STEAM education is enhanced through the technique of fostering problem-solving in students using interactions with and active discussions about artwork collections and objects.

Zooming in Time\u2014Exploring Students\u2019 Interpretations of a Dynamic Tree of Life

Central to evolution is the concept of a common ancestry from which all life has emerged over immense time scales, but learning and teaching temporal aspects of evolution remain challenging. This study investigated students’ interpretation of evolutionary time when engaging with a multi-touch tabletop application called DeepTree, a dynamic visualization of a phylogenetic tree. Specifically, we explored how interactive finger-based zooming (zooming “in” and “out”) influenced students’ interpretation of evolutionary time, and how temporal information and relationships were conceptualized during interaction. Transcript analysis of videotaped interview data from ten secondary school students while they interacted with DeepTree revealed that zooming was interpreted in two ways: as spatially orientated (movement within the tree itself), or as time-orientated (movement in time). Identified misinterpretations included perceiving an implicit coherent timeline along the y-axis of the tree, that the zooming time duration in the virtual tree was linearly correlated to real time, and that more branch nodes correspond to a longer time. Sources for erroneous interpretations may lie in transferring everyday sensory experiences (e.g., physical movements and observing tree growth) to understanding abstract evolution concepts. Apart from estimating the occurrence of dinosaurs, DeepTree was associated with an improvement in interpretation of relative order of evolutionary events. Although highly promising, zooming interaction in DeepTree does not facilitate an intuitive understanding of evolutionary time. However, the opportunity to combine visual and bodily action in emerging technologies such as Deep Tree suggests a high pedagogical potential of further development of zooming features for optimal scientific understanding.

Impact of different interaction protocols on group communication, satisfaction and learning outcomes of primary school children when using multitouch tabletops

Using information and communication technology (ICT) in childhood education is becoming more relevant as research shows that it can be used to foster children's academic and non-academic skills. ICT can help build environments where children can communicate and collaborate, but creating effective learning environments is not trivial. It is thus necessary to study which configuration is the most appropriate for encouraging collaboration. In this work, we present how two different ways of interacting with a multitouch tabletop, taking turns without having to agree on the answer and working simultaneously but having to agree on the answer, affect group communication and children's satisfaction. We have carried out four different learning experiments involving 180 children between 6 and 11 years of age who had to solve math problems in groups of three and four at a multitouch tabletop. Our results suggest that turn-based interaction makes students communicate more with each other when solving activities in groups. In addition, children's satisfaction is high when they perform activities at a multitouch tabletop, but learning outcomes seems to not be impacted by the way of interacting with the device. Thus, while multitouch tabletops can be used to create collaborative learning environments, it is the way in which students interact with the device that may impact group communication.

Recognizing Unintentional Touch on Interactive Tabletop

A multi-touch interactive tabletop is designed to embody the benefits of a digital computer within the familiar surface of a physical tabletop. However, the nature of current multi-touch tabletops to detect and react to all forms of touch, including unintentional touches, impedes users from acting naturally on them. In our research, we leverage gaze direction, head orientation and screen contact data to identify and filter out unintentional touches, so that users can take full advantage of the physical properties of an interactive tabletop, e.g., resting hands or leaning on the tabletop during the interaction. To achieve this, we first conducted a user study to identify behavioral pattern differences (gaze, head and touch) between completing usual tasks on digital versus physical tabletops. We then compiled our findings into five types of spatiotemporal features, and train a machine learning model to recognize unintentional touches with an F1 score of 91.3%, outperforming the state-of-the-art model by 4.3%. Finally we evaluated our algorithm in a real-time filtering system. A user study shows that our algorithm is stable and the improved tabletop effectively screens out unintentional touches, and provide more relaxing and natural user experience. By linking their gaze and head behavior to their touch behavior, our work sheds light on the possibility of future tabletop technology to improve the understanding of users' input intention.

DEDOS: An Authoring Toolkit to Create Educational Multimedia Activities for Multiple Devices

Information and Communication Technologies offer new possibilities for teachers to enhance their teaching methods. The increasing use of personal computers, tablets, interactive whiteboards, or even multitouch tabletops in the classrooms seems to attract the interest of the students. However, there are not many tools that allow teachers to create multimedia activities for all these technologies in an effortless way. Most of current authoring tools either focus on creating content for only one device or they do not fully exploit the benefits of rich content for designing engaging educational activities. In this paper, we present an authoring toolkit composed by two applications: DEDOS-Editor, which allow teachers to design their own learning activities, and DEDOS-Web, which allows the students to perform those activities adapting them to multiple devices. To test both tools, we have performed two evaluations. One with teachers to test the authoring tools and a second one with primary school students to test if the activities designed with this tool enhance their learning process. Results show that DEDOS-Editor is an easy to learn authoring tool which helps teachers to complement their learning methods while DEDOS-Web is flexible enough to create several learning scenarios from just one set of activities, factors which lead to achieving positive learning outcomes.

Multi-Touch Tabletop System Using Infrared Image Recognition for User Position Identification

A tabletop system can facilitate multi-user collaboration in a variety of settings, including small meetings, group work, and education and training exercises. The ability to identify the users touching the table and their positions can promote collaborative work among participants, so methods have been studied that involve attaching sensors to the table, chairs, or to the users themselves. An effective method of recognizing user actions without placing a burden on the user would be some type of visual process, so the development of a method that processes multi-touch gestures by visual means is desired. This paper describes the development of a multi-touch tabletop system using infrared image recognition for user position identification and presents the results of touch-gesture recognition experiments and a system-usability evaluation. Using an inexpensive FTIR touch panel and infrared light, this system picks up the touch areas and the shadow area of the user’s hand by an infrared camera to establish an association between the hand and table touch points and estimate the position of the user touching the table. The multi-touch gestures prepared for this system include an operation to change the direction of an object to face the user and a copy operation in which two users generate duplicates of an object. The system-usability evaluation revealed that prior learning was easy and that system operations could be easily performed.

Comparing a Single-Touch Whiteboard and a Multi-Touch Tabletop for Collaboration in School Museum Visits

This paper explores two important classes of large screen displays, single-touch whiteboards and multi-touch tabletops, for the context of collaborative learning by school groups at a museum. To do this, we designed MuseWork, as a worksheet activity, with two phases: first, students explore the museum, individually or in pairs, guided by our tablet worksheet app; then, in groups, they collaborate to create a poster at a large-screen display, using our device-customised MuseWork interfaces. Our goal was to gain insights about the implications for engagement and collaboration when groups use these devices; single-touch whiteboards are important as they are widely available in classrooms and multi-touch tabletops are an emerging technology. Our research questions asked: 1) whether MuseWork enabled groups to complete the collaborative task at both devices and 2) how the whiteboard and tabletop each affect key aspects of collaboration. We report a between-subjects study of 67 students, aged 10--14 years, from 2 schools, in 12 groups. Our results, based on qualitative and quantitative data, indicate the MuseWork interface for each device proved effective, with groups completing the activity and satisfied with the result and the experience (RQ1). Comparisons of groups using each device (RQ2) give new insights in terms of the products of the collaborative activity, and the strategies groups spontaneously developed for group co-ordination and device use. Our contributions are insights from the first in-the-field study of children collaborating at single-touch interactive whiteboards and multi-touch tabletops.

„It’s ComforTable“: Ansätze zur Erforschung von Interaktionen an Tabletop- Displays im Ausstellungsraum

The persuasive dimension of design can be revealedby a detailed view on objects in spaceThis paper discusses the concepts of interactional architecture and interactional space as a base for analyzing interaction on multi-touch tabletops. It aims at showing how to develop questions for a future analysis of interactive actions through the analysis of the architecture of space. Therefore, the present paper contains a report of a data-session, in which the architecture of the tabletop was investigated. It will be shown that the tabletop’s design suggests various ways of use. Only the combination of an analysis of architecture and of interaction in a real environment may eventually lead to a better understanding of multi-user interaction. Such an in-the-wild study can result in ideas on the user-centered design of interfaces.

A Comparative Study of Map Exploration Interfaces for Multi-Touch Tabletops

ABSTRACTWith the advent of multi-touch tabletops, software developers are creating innovative interfaces for applications such as interactive map systems. In this article, we summarized three representative interfaces from prevalent electronic map applications and literatures—gesture based, widget based, and hybrid, which supported both gestures and widgets—and then compared the effectiveness and the efficiency of these three interfaces by experiments. The results show that the hybrid interface had a better performance, requiring less task-completion time and operations, and causing less arm fatigue. The widget-based interface had the least distance traversed by fingers, but it required many tapping operations. The gesture-based interface was easier to learn compared to the widget-based interface, but the distance traversed during finger movements was the longest. Based on the findings from our experiments, we discuss the implications for the design of map exploration interfaces on multi-touch tabletops.

Making L2 learners’ reasoning skills visible: The potential of Computer Supported Collaborative Learning Environments

This paper explores the use of Computer Supported Collaborative Learning Environments (CSCLE) as multimodal spaces for promoting critical thinking for English as Second Language Learning (L2) education from multiple perspectives (Technology, Thinking Skills and Interaction). The exploration focuses on the use of a multitouch tabletop, and an accompanying application called Digital Mysteries, as affordances in CSCLE’s for making reasoning skill-based thinking visible for L2 learning in Higher Education.Despite the worldwide promotion of teaching thinking in L2 education, it is not always easy for teachers to identify the types of thinking skills being targeted in L2 pedagogical tasks. To the authors’ knowledge, little empirical interactional evidence is available to demonstrate critical thinking in L2 learner talk during group work. This paper examines interactions among three groups of Chinese English Language learners at a higher education institution in a CSCLE. Video data were collected of students’ thinking-in-action whilst engaging in multimodal interactions in the environment. Results show that new technologies can provide innovative and empirically driven ways in which L2 learners’ thinking is externalised and how critical reasoning can be tracked, promoted, evaluated and self-regulated. The findings suggest that collaborations in a CSCLE can support the completion of tasks embedding high levels of cognitive complexity by L2 learners with effective use of limited cognitive resources. This leads to a number of recommendations about integrating the teaching of critical thinking skills into the L2 classroom using CSCLE technologies.

The "Smart Dining Table": Automatic Behavioral Tracking of a Meal with a Multi-Touch-Computer.

Studying how humans eat in the context of a meal is important to understanding basic mechanisms of food intake regulation and can help develop new interventions for the promotion of healthy eating and prevention of obesity and eating disorders. While there are a number of methodologies available for behavioral evaluation of a meal, there is a need for new tools that can simplify data collection through automatic and online analysis. Also, there are currently no methods that leverage technology to add a dimension of interactivity to the meal table. In this study, we examined the feasibility of a new technology for automatic detection and classification of bites during a laboratory meal. We used a SUR40 multi-touch tabletop computer, powered by an infrared camera behind the screen. Tags were attached to three plates, allowing their positions to be tracked, and the saturation (a measure of the infrared intensity) in the surrounding region was measured. A Kinect camera was used to record the meals for manual verification and provide gesture detection for when the bites were taken. Bite detections triggered classification of the source plate by the SUR40 based on saturation flux in the preceding time window. Five healthy subjects (aged 20–40 years, one female) were tested, providing a total sample of 320 bites. Sensitivity, defined as the number of correctly detected bites out of the number of actual bites, was 67.5%. Classification accuracy, defined as the number of correctly classified bites out of those detected, was 82.4%. Due to the poor sensitivity, a second experiment was designed using a single plate and a Myo armband containing a nine-axis accelerometer as an alternative method for bite detection. The same subjects were tested (sample: 195 bites). Using a simple threshold on the pitch reading of the magnetometer, the Myo data achieved 86.1% sensitivity vs. 60.5% with the Kinect. Further, the precision of positive predictive value was 72.1% for the Myo vs. 42.8% for the Kinect. We conclude that the SUR40 + Myo combination is feasible for automatic detection and classification of bites with adequate accuracy for a range of applications.

RoboJockey: Designing an Entertainment Experience with Robots.

The RoboJockey entertainment system consists of a multitouch tabletop interface for multiuser collaboration. RoboJockey enables a user to choreograph a mobile robot or a humanoid robot by using a simple visual language. With RoboJockey, a user can coordinate the mobile robot's actions with a combination of back, forward, and rotating movements and coordinate the humanoid robot's actions with a combination of arm and leg movements. Every action is automatically performed to background music. RoboJockey was demonstrated to the public during two pilot studies, and the authors observed users' behavior. Here, they report the results of their observations and discuss the RoboJockey entertainment experience.

Using Multitouch Collaboration Technology to Enhance Social Interaction of Children with High-Functioning Autism

ABSTRACTAims. Children with high-functioning Autism Spectrum Disorder (HFASD) have major difficulties in social communication skills, which may impact their performance and participation in everyday life. The goal of this study was to examine whether the StoryTable, an intervention paradigm based on a collaborative narrative, multitouch tabletop interface, enhanced social interaction for children with HFASD, and to determine whether the acquired abilities were transferred to behaviors during other tasks. Methods. Fourteen boys with HFASD, aged 7–12 years, participated in a 3-week, 11-session intervention. Social interactions during two nonintervention tasks were videotaped at three points in time, one prior to the intervention (pre), a second immediately following the intervention (post) and a third three weeks after the intervention (follow-up). The video-recorded files were coded using the Friendship Observation Scale to ascertain the frequencies of positive and negative social interactions and collaborative play. Differences in these behaviors were tested for significance using nonparametric statistical tests. Results. There were significantly higher rates of positive social interactions and collaborative play, and lower rates of negative social interactions following the intervention suggesting generalization of the social skills learned during the intervention. Improvement was maintained when tested three weeks later. Conclusion. These findings provide support for the use of collaborative technology-based interventions within educational settings to enhance social interaction of children with HFASD.

“Whoa! We’re going deep in the trees!”: Patterns of collaboration around an interactive information visualization exhibit

In this paper we present a qualitative analysis of natural history museum visitor interaction around a multi-touch tabletop exhibit called DeepTree that we designed around concepts of evolution and common descent. DeepTree combines several large scientific datasets and an innovative visualization technique to display a phylogenetic tree of life consisting of over 70,000 species. After describing our design, we present a study involving pairs of children interacting with DeepTree in two natural history museums. Our analysis focuses on two questions. First, how do dyads negotiate their moment-to-moment exploration of the exhibit? Second, how do dyads develop and negotiate their understanding of evolutionary concepts? In order to address these questions we present an analytical framework that describes dyads’ exploration along two dimensions: coordination and target of action. This framework reveals four distinct patterns of interaction, which, we argue, are relevant for similar interactive designs. We conclude with a discussion of the role of design in helping visitors make sense of interactive experiences involving the visualization of large scientific datasets.

Applications of Multi-Touch Tabletop Displays and Their Challenging Issues: An Overview

Abstract Multi-touch tabletop displays provide a co-located collaborative workspace for multiple users around a physical table. They sit together and perform collaborative interaction to select and manipulate digital contents using their bare fingers. However, these systems bring a new paradigm shift in user interaction and present various challenges to design natural user interfaces respectively. The growing popularity of tabletop displays and their related issues have gained a greater attention among researchers in academia and industry. It creates a need to present an overview of multi-touch tabletop displays. This review paper attempts to present the touch enabling technologies that support in the construction of multi-touch tabletop displays. It also presents the important applications of multi-touch tabletop displays in different domains and their challenging issues in different perspectives. Finally, this paper proposes the future work.

A user experience model for tangible interfaces for children

Tangible user interfaces allow children to take advantage of their experience in the real world when interacting with digital information. In this paper we describe a model for tangible user interfaces specifically for children that focuses mainly on the user experience during interaction and on how to design interactions. This model is related to other user-centered models and has been used to design a multi-touch tabletop application for a museum. The results of a first evaluation of this museum application show that the interactions at the multi-touch table were engaging and fun, which was one of the goals of the museum application and one of the four themes of the model. Regarding a few other themes of the model results of the first observations are encouraging: the interactions seemed to be intuitive and seemed to facilitate exploration and collaboration. More systematic evaluation of the complete model by video analysis and an additional validation study is proposed.

Collaborative learning using tabletop and interactive whiteboard systems

The information and communication technologies and interactive displays highly encouraged and assisted the collaborative learning. The recent developments in interactive whiteboards and multi-touch tabletop displays technologies also support the collaborative learning in the different settings. However, there is lack of studies to assess the user experience and feedback of collaborative learning using tabletop and interactive whiteboard systems. This paper attempts to assess the user experience using experimental methods. The findings from current literature were used to formulate problem and implement the prototype. Later on the test participants experienced the system. The findings show that tabletop and interactive whiteboard systems support and enhance the collaborative learning and it can be improved through better incorporation and setting of these systems in classrooms or other places. The overall user satisfaction for using whiteboard was 3.7/5.0 and tabletop was 3.9/5.0, which shows a motivation of using both technologies in a collaborative learning environment. The findings suggest for better design of user interface for these system in order to maximise the user performance in terms of collaboration, efficiency and effectiveness during collaborative learning.