Multi-touch Surfaces Research Articles

Examining the usability of touchscreen gestures for adults with DS

This document is part of a global investigation that aims to establish best practices in usability testing of mobile applications, that fit the specific needs of persons with cognitive disabilities. The motivating factor is to improve the quality of life of people with special needs. As a first step, we want to discover what are the skills of people with DS (DS) when using a mobile device. Thanks to its direct manipulation interaction style, multi-touch technology is the ideal mechanism for learning activities with a view to the social inclusion of people with DS. This paper investigates the most common touchscreen gestures on commercial existing software. A commercial analysis was carried out to discover the touch screen gestures used by 103 free software Apps running on a mobile touch screen tablet of 11 where the applications run”. The commercial analysis showed that most applications support tap and drag operations on multi-touch technology. Additionally, the research sought to discover the adults with DS (19–54 years of age) ability to perform other gestures on multi-touch surfaces. A DS user skill study was performed to assess the ability of this user segment to interact with multi-touch surfaces. The analysis involved 53 participants, aged between 19 and 54 years, from two vocational training centers attended by people with DS in Madrid. Authors used the Gesture Games App for experimenting with multi-touch gestures such as tap, double tap, long press, drag, scale up, scale down, and one-finger rotation. Tap, double tap, and drag were the three gestures that the most participant could use. In contrast, participants had difficulty performing one-finger rotation and long press gesture. Our statistical analysis showed that the ability to perform each gesture was independent of gender, age group, and previous experience with touchscreen of a participant.

Turns vs consensus: Learning mathematics in multi-touch surfaces

• There is no significant difference between turns and consensus. • Consensus methodology enhaces 21st century skills. • There is no gender gap. • Multi-touch table have a positive impact in the knowledge. The adoption of information and communication technologies in primary education is crucial for adapting traditional classrooms to the digital era. Over time, young children are increasingly using touch screen technologies such as tablets at home and interactive whiteboards at school, either for leisure or for academic activities. However, the literature shows that there is still a gap between what is known about the benefits of using this technology and its real use in primary education settings. Most researchers have focused on the pedagogical theory behind using touch screen devices, but there are few empirical studies about how these technologies and different approaches affect students’ learning processes. This paper presents two learning experiences in a primary school in Fuenlabrada (Madrid-Spain) where primary students performed mathematics activities using a multi-touch table with two different methodologies: turns and consensus. The results show that both methodologies help students acquire meaningful learning, but there is no statistically significance between them.

A Unified Model for User Identification on Multi-Touch Surfaces: A Survey and Meta-Analysis

User identification on interactive surfaces is a desirable feature that is not inherently supported by existing technologies. We have conducted an extensive survey of existing identification techniques, which led us to formulate a unified model for user identification. We start by introducing this model that (1) classifies existing user identification approaches in five categories according to the identification technology, (2) identifies eight characteristic identification system parameters, and (3) proposes a way for visualizing the system's characteristics as points on a radar chart to allow for quick comparison and contrast between systems. This model is then used to present our survey of existing user identification approaches and visualize their characteristics, highlighting their strengths and limitations. The model also makes it possible to visually represent requirements of systems that require user identification, identify existing approaches that can meet an application's requirements, and help report on and evaluate new approaches to user identification systematically.

Catch a Star: A Fitt�s Law-Based Touch Accuracy Test on Multi Touch Surface

Objectives: To provide an alternative solution which is Catch-the-Stars that applied Fitt’s Law rules to determine the accuracy of children-multi touch interaction. Methods/Analysis: A comparison of touch accuracy between kindergarten children and university students was conducted to provide a holistic analysis for the application testing. Multi touch surface is the latest and a great way to interact with computers and mobile devices. Nevertheless, several problems were highlighted, especially on children-multitouch interaction and touch accuracy for children when they interact with multitouch devices. Findings: The results demonstrated that the size, position, and pixel of the objects, as well as the direction of the screen that is displayed, affect the accuracy to capture the target. Based on the results that were collected from kindergarten children and university students, the size of objects that is more suitable for the children and students can be determined. For children, the most suitable size of objects is medium size which is 100 pixels. Application/ Improvements: As the current technology is constantly evolving at a fast pace, multi touch surface devices have become a popular trend. Multi touch surface not only functions as an important tool but is also a representative of high-technology products in all range of education. Keywords: Catch-the-Stars, Children-Multi Touch Interaction, Fitt’s Law, Movement Time, Touch Accuracy

A pattern-based approach for developing creativity applications supported by surface computing

An activity model for applications focused on creativity.Building blocks for constructing applications designed for creativity.By relying on our building blocks, applications were implemented in short time.Our building blocks prove the model's expressivity for describing creativity sessions. Whereas creativity tasks have traditionally been supported by conventional media and tools (such as paper, pens, scissors and glue); mobile phones, tablets and other devices based on interactive surfaces are increasingly been used as additional support. Large-sized multi-touch interactive surfaces appear as an interesting alternative for supporting creativity processes and for supporting synchronous collocated collaboration. However, they have mostly been usedfor visualization and navigation purposes. Their use as authoring means, which would be essential in creativity tasks, has only begun to be explored. Applications and platforms that have been developed in this area rely on low-level primitives for implementing representation of ideas and discussions. We have identified a significant gap between the level of development tools and the abstractions required by end-user applications that aim to support creativity processes using interactive surfaces. This gap makes it difficult for developers to build applications that provide richer, more flexible support for innovators working collaboratively around interactive surfaces. Based upon a thorough analysis of existing applications and user practices in the field, we have identified the key actions and interaction patterns that take place during collaborative creativity sessions. Thus, we propose ISCALI (Innovation Solutions Centered on Activities for Large-sized Interfaces), a model that can be used both for describing and for prescribing the role of multi-touch surfaces in collaborative creativity tasks. In accordance to Activity Centered Design, ISCALI comprises three major components: activities, actions and operations. The central activities within the processes of creativity comprise generation, organization and evaluation of ideas. Each of these activities encompasses sets of actions. Finally, several operation sets achieve the goal of each of the actions. Based upon our model, we designed a general architecture for collaborative creativity applications. This architecture addresses the development gap through a proposed set ofbuilding blocks. These building blocks implement the main interaction patterns needed for stimulating creativity tasks that rely on interactive surfaces. We have implemented prototypical versions of these building blocks, referred to as TOKAs (Touch Operations for Creative Activities), and have made them available to developers. Independent developers have implemented applications that facilitate the use of various methodologies that foster creativity and synchronous collocated collaboration. These developers have taken advantage of the availability of TOKAs. The implementation and use of TOKAs demonstrate ISCALI's expressivity for describing and guiding the development of applications that support collaborative creativity on top of interactive surfaces.

Accessibility and tangible interaction in distributed workspaces based on multi-touch surfaces

Traditional interaction mechanisms in distributed digital spaces often fail to consider the intrinsic properties of action, perception, and communication among workgroups, which may affect access to the common resources used to mutually organize information. By developing suitable spatial geometries and natural interaction mechanisms, distributed spaces can become blended where the physical and virtual boundaries of local and remote spaces merge together to provide the illusion of a single unified space. In this paper, we discuss the importance of blended interaction in distributed spaces and the particular challenges faced when designing accessible technology. We illustrate this discussion through a new tangible interaction mechanism for collaborative spaces based on tabletop system technology implemented with optical frames. Our tangible elements facilitate the exchange of digital information in distributed collaborative settings by providing a physical manifestation of common digital operations. The tangibles are designed as passive elements that do not require the use of any additional hardware or external power while maintaining a high degree of accuracy.

Investigating device‐specific visual feedback for cross‐device transfer in table‐centric multisurface environments

SummaryTable‐centric multisurface environments (T‐MSEs) that combine small multitouch surfaces (eg, smartphones and tablets) with large interactive tabletops provide people with both personal and shared workspaces to support various independent and collective tasks during group activities. This paper reports on the third in a series of studies exploring how existing interaction methods for cross‐device transfer, such as the Pick‐and‐Drop (P&D) method, can be adapted to table‐centric multisurface environment settings. The study examined the use of device‐specific visual feedback to improve users' awareness of transferred content during P&D transfer. The tabletop feedback utilized the existing Surface Ghosts P&D feedback approach (ie, “ghosted” versions of transferred content were displayed in real time under the user's hand). The tablet feedback consisted of a static “Tablet Bridge” feedback showing miniature versions of transferred content along the top edge of the tablet interface. The study found that providing both types of feedback significantly improved users' transfer awareness over providing Surface Ghosts feedback alone. It also revealed that the Tablet Bridge feedback helped compensate for technical and usability issues associated with the Surface Ghosts feedback design. Lessons learned from our combined series of cross‐device transfer studies are reflected upon, and relevant design implications are discussed.

Time-sculptures of terrifying ambiguity: staging ‘inner space’ and migrating realities in Analogue’s Living Film Set

ABSTRACTThis article examines Analogue’s Living Film Set, an interactive theatre piece which uses miniature film sets, multi-touch surface technology and live video feeds to reframe my semi-remembered memories from the mid-1980s as a collective participatory experience. Drawing on new wave novelist J. G. Ballard’s notion of childhood memory as ‘time-sculptures of terrifying ambiguity’ [Ballard, J. G. 1963. “Time, Memory and Inner Space.” J. G. Ballard website (originally published in The Woman Journalist Magazine). Accessed August 6, 2015. http://www.jgballard.ca/non_fiction/jgb_time_memory_innerspace.html], I will demonstrate how my childhood town of Shepperton has been overwritten in both Ballardian literary fiction and the incursion of cinematic artifice from the neighbouring activities of Shepperton Film Studios. I argue that the ambiguity of my recollections and the contamination of my lived history with ‘prosthetic memories’ [Landsberg, Alison. 2004. Prosthetic Memory: The Transformation of American Remembrance in the Age of Mass Culture. New York, NY: Columbia UP, 20–21.] has provided a creative space to re-enact the blended hyperreality of my early childhood through the work’s intermedial form. I will conclude by examining how the shifting reality status of the media used within the performance intersects with the notion of ‘time-sculptures’ and problematises what Carol Martin [(2013). Theatre of the Real. Basingstoke: Palgrave Macmillan.] has identified as ‘theatre of the real’.

Leveraging finger identification to integrate multi-touch command selection and parameter manipulation

Identifying which fingers are touching a multi-touch surface provides a very large input space. We describe FingerCuts, an interaction technique inspired by desktop keyboard shortcuts to exploit this potential. FingerCuts enables integrated command selection and parameter manipulation, it uses feed-forward and feedback to increase discoverability, it is backward compatible with current touch input techniques, and it is adaptable for different touch device form factors. We implemented three variations of FingerCuts, each tailored to a different device form factor: tabletop, tablet, and smartphone. Qualitative and quantitative studies conducted on the tabletop suggests that with some practice, FingerCuts is expressive, easy-to-use, and increases a sense of continuous interaction flow and that interaction with FingerCuts is as fast, or faster than using a graphical user interface. A theoretical analysis of FingerCuts using the Fingerstroke-Level Model (FLM) matches our quantitative study results, justifying our use of FLM to analyse and validate the performance for the other device form factors.

A pattern-based approach for developing creativity applications supported by surface computing

Whereas creativity tasks have traditionally been supported by conventional media and tools, (such as paper, pens, scissors and glue), mobile phones, tablets and other devices based on interactive surfaces are increasingly been used as additional support. Large-sized multi-touch interactive surfaces appear as an interesting alternative for supporting collaborative creativity processes. However, they have mostly been used for visualization and navigation purposes. Their use as authoring means, which would be essential in creativity tasks, has only begun to be explored. We have identified a significant gap between the level of development tools and the abstractions required by end-user applications that aim to support creativity processes using interactive surfaces. Thus, we propose ISCALI (Innovation Solutions Centered on Activities for Large-sized Interfaces), a model that can be used both for describing and for prescribing the role of multi-touch surfaces in collaborative creativity tasks. Based upon ISCALI, we designed a general architecture for collaborative creativity applications; and we developed a set of building blocks, referred to as TOKAs (Touch Operations for Creative Activities). Applications that facilitate the use of various methodologies that foster collaborative creativity have been implemented by independent developers who have taken advantage of the availability of TOKAs. The implementation and use of TOKAs demonstrate ISCALI’s expressivity for describing and guiding the development of applications that support collaborative creativity on top of interactive surfaces.

Exploring Developmental Appropriateness of Multitouch Tables in Prekindergarten: A Video Analysis

This study explored how developmentally appropriate practices influenced the affordances of a multitouch surface in an early childhood classroom. Children, ages 4 to 5 years old, were videotaped while engaged in a storymaking application on a multitouch table. As a result of the study, we found that some challenges with the functionality undermined children's autonomous control. The researchers observed numerous instances of issues with control of the medium, including height of the table, sensitivity to multiple types of touch, and limited instructions and prompts. Implications for future research and practice are discussed.

Toward natural multi-user interaction in advanced collaborative display environments

Large high-resolution displays have become widely spread in research centers, laboratories, and public spaces during the last decade. There have been various research efforts in transformation of these displays from passive screens to interactive environments where multiple users can interact independently with multi-touch surfaces, in-air interaction or pointing devices. Such systems provide basic multi-user interaction, i.e., two or more users can interact concurrently. However, continuous user tracking and association of input events with users, which could considerably improve user experience, is still a largely unexplored topic. In this paper, we present a set of techniques enabling cue-less multi-user interaction in environments where horizontal or vertical high-resolution displays are in the role of central visualization platforms. We define the concept of cue-less multi-user interaction and set of techniques that enable unobtrusive user tracking and their association with input events through combination of a multi-touch surface and a depth sensor. Further, we introduce an open-source framework implementing techniques behind this concept and their evaluation in terms of accuracy of user association and the overall system interactivity when integrated with selected distributed rendering middleware.

Designing an accessible low-cost interactive multi-touch surface

This paper presents an accessible interactive surface specifically designed for cerebral palsy users. The work context is an educational centre for disabled users, although the system presented can be used in other contexts. The main contributions are as follows: the table leaves enough leg room to fit in wheelchairs and has a half-circle radius cut-out in the top which lets users in wheelchairs sit closer to the tabletop. Table height can be easily adjusted to children or adults. The tabletop screen angle can be easily adjusted to adapt to different body postures and allow for user comfort. The tabletop does not compromise user safety as the surface is protected with strong Plexiglas. Furthermore, an assessment of accessibility has been carried out to analyse its appropriateness to fit users with different characteristics.

Multi-touch gestures for pre-kindergarten children

The direct manipulation interaction style of multi-touch technology makes it the ideal mechanism for learning activities from pre-kindergarteners to adolescents. However, most commercial pre-kindergarten applications only support tap and drag operations. This paper investigates pre-kindergarteners׳ (2–3 years of age) ability to perform other gestures on multi-touch surfaces. We found that these infants could effectively perform additional gestures, such as one-finger rotation and two-finger scale up and down, just as well as basic gestures, despite gender and age differences. We also identified cognitive and precision issues that may have an impact on the performance and feasibility of several types of interaction (double tap, long press, scale down and two-finger rotation) and propose a set of design guidelines to mitigate the associated problems and help designers envision effective interaction mechanisms for this challenging age range.

Supporting the Loewenstein occupational therapy cognitive assessment using distributed user interfaces

Purpose: Improve the quantity and quality of information obtained from traditional Loewenstein Occupational Therapy Cognitive Assessment Battery systems to monitor the evolution of patients’ rehabilitation process as well as to compare different rehabilitation therapies. Methods: The system replaces traditional artefacts with virtual versions of them to take advantage of cutting edge interaction technology. The system is defined as a Distributed User Interface (DUI) supported by a display ecosystem, including mobile devices as well as multi-touch surfaces. Due to the heterogeneity of the devices involved in the system, the software technology is based on a client-server architecture using the Web as the software platform. Results: The system provides therapists with information that is not available (or it is very difficult to gather) using traditional technologies (i.e. response time measurements, object tracking, information storage and retrieval facilities, etc.). Conclusions: The use of DUIs allows therapists to gather information that is unavailable using traditional assessment methods as well as adapt the system to patients’ profile to increase the range of patients that are able to take this assessment.Implications for RehabilitationUsing a Distributed User Interface environment to carry out LOTCAs improves the quality of the information gathered during the rehabilitation assessment.This system captures physical data regarding patient’s interaction during the assessment to improve the rehabilitation process analysis.Allows professionals to adapt the assessment procedure to create different versions according to patients’ profile.Improves the availability of patients’ profile information to therapists to adapt the assessment procedure.

Efficacy of surface disinfectant cleaners against emerging highly resistant gram-negative bacteria.

BackgroundWorldwide, the emergence of multidrug-resistant gram-negative bacteria is a clinical problem. Surface disinfectant cleaners (SDCs) that are effective against these bacteria are needed for use in high risk areas around patients and on multi-touch surfaces. We determined the efficacy of several SDCs against clinically relevant bacterial species with and without common types of multidrug resistance.MethodsBacteria species used were ATCC strains; clinical isolates classified as antibiotic-susceptible; and multi-resistant clinical isolates from Klebsiella oxytoca, Klebsiella pneumoniae, and Serratia marcescens (all OXA-48 and KPC-2); Acinetobacter baumannii (OXA-23); Pseudomonas aeruginosa (VIM-1); and Achromobacter xylosoxidans (ATCC strain). Experiments were carried out according to EN 13727:2012 in quadruplicate under dirty conditions. The five evaluated SDCs were based on alcohol and an amphoteric substance (AAS), an oxygen-releaser (OR), surface-active substances (SAS), or surface-active-substances plus aldehydes (SASA; two formulations). Bactericidal concentrations of SDCs were determined at two different contact times. Efficacy was defined as a log10 ≥ 5 reduction in bacterial cell count.ResultsSDCs based on AAS, OR, and SAS were effective against all six species irrespective of the degree of multi-resistance. The SASA formulations were effective against the bacteria irrespective of degree of multi-resistance except for one of the four P. aeruginosa isolates (VIM-1). We found no general correlation between SDC efficacy and degree of antibiotic resistance.ConclusionsSDCs were generally effective against gram-negative bacteria with and without multidrug resistance. SDCs are therefore suitable for surface disinfection in the immediate proximity of patients. Single bacterial isolates, however, might have reduced susceptibility to selected biocidal agents.

Design and Implementation of Low-Cost Multi-Touch Surface Computing Using Image Processing

Touch sensing devices have reached the pinnacle of success in the present era thereby revolutionizing human machine interaction. Capacitive touch-screens are popular because of its portability whereas image processing based touch-screens are massive but for low-cost surface computers with the capability of both touch and object-recognition, the latter is preferred because it uses optical solutions to sense touch.Multi-touch devices are in vogue and therefore the concept of surface computing is gaining popularity since it provides an enhanced multi-touch and multi-user experience.Multi-touch denotes a set of interaction techniques that allow computer users to control multimedia applications with several fingers. The concept of gesture recognition thus comes into the picture. Our objective is to implement a cost-effective multi-touch surface computing system. The image processing algorithms used are implemented in MATLAB and the simulation of gesture recognition is carried out using Open Source Computer Vision Library (OpenCV).

Mockup Builder: 3D modeling on and above the surface

We present Mockup Builder, a semi-immersive environment for conceptual design which allows virtual mockups to be created using gestures. Our goal is to provide familiar ways for people to conceive, create and manipulate three-dimensional shapes. To this end, we developed on-and-above-the-surface interaction techniques based on asymmetric bimanual interaction for creating and editing 3D models in a stereoscopic environment. Our approach combines both hand and finger tracking in the space on and above a multi-touch surface. This combination brings forth an alternative design environment where users can seamlessly switch between interacting on the surface or above it to leverage the benefit of both interaction spaces. A formal user evaluation conducted with experienced users shows very promising avenues for further work towards providing an alternative to current user interfaces for modeling.

Applications of Multi-touch Augmented Reality System in Education and Presentation of Virtual Heritage

Applications of augmented reality have a great immersion potential for education, edutainment or cultural heritage presentations. The combination of intuitive and playful interaction with multi-faceted presentation options proved to be an effective mixture that attracts viewers to educational or cultural content. We introduce a multi-touch augmented reality system (MARS) using two display units for showing two contexts of the same object(s). One of the displays serves as primary context and hosts a multi-touch surface for user input. The second display shows a camera feed of user interacting with the primary context and replaces the primary context in real-time by the secondary context and adds the augmented reality presentation of the object(s).

Collaborative Map Exploration Using Multitouch Surfaces

The Collaborative Map Explorer defines a new learning artifact that allows students to explore different layers of information related to a geographical location on a map using distributed user interfaces in the same space at the same time. Students are able to manipulate geographical information using viewports that can be moved over a map that is projected on a shared surface (i.e. multitouch surface). Each viewport exposes different layers of information for a particular region on the map. These information layers are selected by the student using a mobile device. Consequently, the use of a distributed user interface provides students with the ability to address local and shared information at the same time in the same space in a Computer Supported Collaborative Learning Environment. The article also includes a preliminary usability evaluation of this interaction technique that exposes the potential of this solution.