Tangible Interface Research Articles

Home-Based Activities for Children with Speech Sound Disorders: Requirements for a Tangible User Interface for Internet of Things Artefacts

This paper presents the state of the art regarding the use of tangible user interfaces for internet of artefacts (IoA) targeting health applications, with a focus on speech and language therapy and related areas, targeting home-based interventions, including data security and privacy issues. Results from a systematic literature review, focus group, and a nationwide questionnaire have been used to determine the system requirements for an artefact prototype to be developed. The aim of this study was to understand what is the usual practice of clinicians and to contribute to a better intervention or post-intervention approach for children with Speech Sound Disorders (SSD). The literature review revealed that some studies proposed technological solutions while others used a social approach and/or gamified activities. We could conclude that more research is needed and that a unified method or framework to address SSD intervention or post-intervention tools is lacking. Clinicians need more and better tools to be able to quantify and qualitatively assess the activities developed at home.

Comparing the Performance and Evaluation of Computer Experts and Farmers when Operating Agricultural Robots: A Case of Tangible vs Mouse-Based UIs

Tangible user interfaces (TUIs) integrate computer systems with the physical world as they allow interaction through physical objects and the human kinesthetic system. They have been studied in various domains, both by the designing and piloting of innovative applications, as well as by comparatively analyzing TUI against GUI interaction. In the human-robot interaction (HRI) field, TUIs are considered a promising approach as they make effective use of physical object affordances, but research is rather inconclusive about whether TUIs offer improved efficiency, lower error rates, increased intuitiveness, engagement, and user satisfaction. In this study, two prototype UIs were designed and evaluated to compare a TUI with a 2D mouse-based UI for remotely operating two agricultural robots used for vineyard spraying. Two different user groups played the role of the operator: computer experts and farmers leading to an experiment with a 2×2 setup, where two different types of UIs were evaluated by two different user groups. The formulated research questions concern the efficiency, accuracy, and user evaluation for each UI and for each group individually and combined. Analysis has shown that there were no statistically significant differences comparing the alternative UIs for each group in terms of time to complete the task, even though computers experts were faster, as expected. Also, the number of collisions, as well as the percentage of unsprayed and double sprayed area, revealed no significant differences, either for user groups or UIs. The TUI received more positive evaluations in terms of user preference, and users reported lower perceived error rates, especially in the case of farmers, who were also more willing to use the TUI in their daily job.

When does self-report of pain occur?: A study of older adults.

Technologies for self-care can drive participatory health and promote independence of older adults. One self-care activity is regularly measuring and registering personal health indicators (self-reporting). Older adults may benefit from this practice, as they are more likely to have chronic health issues and have specific self-monitoring needs. However, self-reporting technologies are usually not designed specifically for them. Pain is usually measured using patient reports compiled during medical appointments, although this process may be affected by memory bias and under reporting of fluctuating pain. To address these issues, we introduced a simple tangible interface to self-report pain levels and conducted a three-hour evaluation with 24 older adults. The goal of this study was to identify whether specific activities, activity levels or pain levels trigger older adults to self-report their pain level, besides to understand how older adults would use such a device. Within the limited time frame of the experiment, the majority of our participants chose to report pain when they felt it most, not reporting lower levels of pain. No evidence was found to suggest a relationship between the reporting of pain and the activity (or activity level). Several design insights intended to improve the design of technologies are provided.

TactCube: An Intelligent Device to 'converse' with Smart Environments.

Ambient Intelligence is a vision of daily life in which intelligent devices interact with humans to make their lives easier, and the technology is invisible. Artificial Intelligence (AI) governs this smart environment and must interact with humans to best meet their needs and demands. Although voice assistants are very popular and efficient as conversational AI, under some conditions they cannot be used. Therefore, this work proposed a complementary tactile and tangible interface to converse with AI, creating new Tactile Signs. A prototype of TactCube, a wireless cube-shaped device that can interact with AI using only the tactile sense, is presented. The hypothesis is that TactCube can be manipulated with one hand and generate a sequence of numbers that can be interpreted as a new tactile language by a neural network solution. The paper describes the initial research made to define how these sequences can be generated and demonstrates how TactCube is able to do it.

Desarrollo de Herramientas de Educación Convivencial en el Siglo XXI

Ivan Illich famously coined the term «tools for conviviality» in which he refers to tools that encourage individual creativity and expression, and that can be adapted to people’s needs and situations. This paper describes the design of an interactive adaptable tool, whose applications include education and game play, and contextualizes it within Illich’s framework. The Collaborative Interactive Tabletop for Education (CITE), currently being developed at SUNY Polytechnic, allows users to manipulate digital information through the manipulation of hand-held tabletop objects. Such interfaces, known as Tangible User Interfaces (TUI), are believed to have significant advantages over traditional screen-based interfaces. For example, they have been shown to measurably increase collaborative behavior among users. However, TUIs have not been widely adopted, perhaps because most current TUI designs require some kind of specialized hardware. Our approach is to instead create a system, a set of verified instructions along with open-license software packages, for adapting any combination of standard computer hardware into a highly customizable interactive tabletop.

Construction of a Low-Cost Layered Interactive Dashboard with Capacitive Sensing

In the present work, a methodology for the low-cost crafting of an interactive layered dashboard is proposed. Our aim is that the tangible surface be constructed using domestic materials that are easily available in every household. Several tests were performed on different capacitive materials before the selection of the most suitable one for use as a capacitive touch sensor. Various calibration methods were evaluated so that the behavior of the constructed capacitive touch sensors is smooth and reliable. The layered approach is achieved by a menu of few touch buttons on the left side of the dashboard. Thus, various different layers of content can be projected over the same construction, offering extendibility and ease of use to the users. For demonstration purposes, we developed an entertaining plus an educational application of projection mapping for the pervasive and interactive projection of multimedia content to the users of the presented tangible interface. The whole design and implementation approach are thoroughly analyzed in the paper and are presented through the illustration and application of various multimedia layers over the dashboard. An evaluation of the final construction proves the feasibility of the proposed work.

VIC — A Tangible User Interface to train memory skills in children with Intellectual Disability

Memory is defined as the capability of encoding, storing, and retrieving information and is a pillar of our cognitive functions. Memory is one the most investigated processes in people with Intellectual Disability (ID), and studies have documented severe deficits in its functioning. While there are several attempts to exploit GUIs (Graphical User Interfaces) to support memory training for children with ID, limited research explores Tangible User Interfaces (TUIs) for this purpose. The paper describes the design and technology of a novel TUI named VIC (VIsual spatial Cubes for memory training), a system composed by a set of digitally enhanced cubes that emit light and sound, a sensorized board and a mobile app. VIC enables children to perform multiple, configurable memory training activities that involve block manipulation and block placements. The research is based on a vast analysis of the state of the art and on validated methods adopted in memory rehabilitation contexts. From this analysis, and from the theories underlying TUIs, we distilled a set of design principles that informed the design of the multi-sensory affordances of VIC, and can be exploited for other researchers to develop TUIs in this field. The paper also reports an exploratory study that involved 12 children with ID and 3 therapists from a specialized daycare Center. The study focused on the evaluation of the quality of the system in terms of usability, likability and potential for adoption. Although preliminary, the results suggest that our design approach was sound and VIC has the potential to become a valid tool to complement existing practices in memory training and can be expanded to support also memory assessment for children with ID.

Learning maths with a tangible user interface: Lessons learned through participatory design with children with visual impairments and their educators

Through a set of participatory design (PD) sessions with children with visual impairments and their educators, we understood current practices in maths teaching, and designed a novel system to support learning for this particular educational context. Sixteen children were engaged in 19 PD sessions to develop tangibles and auditory stimuli to represent numbers, and to explore activities to use through a tangible user interface. We describe the context and lessons learned along the PD process with children and educators, and their implications on the design. Two main outputs were derived: iCETA, a multimodal tangible user interface that allows the use of tangible blocks to represent numbers; and, Logarín, an audiogame designed for iCETA that enables mathematical training. We explored the use of iCETA and Logarín during 15 sessions with 11 children with visual impairments. Results indicate that playing Logarín is engaging and capable of promoting their mathematical abilities. This research supports evidence that PD is successful in bringing children and other stakeholders together to design a solution that fits children’s needs and promises educational impact.

Develop a Procedural Model for Designing Tangible Interaction Products in Light of Frameworks and Models of Tangible User Interfaces

Develop a Procedural Model for Designing Tangible Interaction Products in Light of Frameworks and Models of Tangible User Interfaces

Conveying Intangible Cultural Heritage in Museums with Interactive Storytelling and Projection Mapping: The Case of the Mastic Villages

Spatial Augmented Reality (SAR), as implemented with projection mapping, is part of mixed-reality technology with numerous applications in the cultural domain. In museums, interactive projection mapping has been exploited to superimpose virtual content on exhibited artefacts, offering users various hybrid ways to interact with the artefacts’ physical and digital content. For this reason, it has been widely used in the context of architectural heritage to promote culture and raise awareness about historical buildings or landscapes by visualizing significant elements they convey. This paper presents the design, development, and iterative user evaluation of an interactive projection mapping installation for the Mastic Museum on Chios island in Greece that promotes UNESCO’s intangible cultural heritage. The installation affords tangible interaction to activate the video projections presented in a storytelling manner on a 3D-printed scale model of a representative historic settlement exhibited inside the museum. The concept of this installation aims to connect the tangible and intangible cultural heritage of mastic and the related villages with narration and vivid illustrations. Three evaluation phases took place during the development at the lab and the museum, informing UX, learning, and design considerations.

Construction of a Tangible VR-Based Interactive System for Intergenerational Learning

The recent years have witnessed striking global demographic shifts. Retired elderly people often stay home, seldom communicate with their grandchildren, and fail to acquire new knowledge or pass on their experiences. In this study, digital technologies based on virtual reality (VR) with tangible user interfaces (TUIs) were introduced into the design of a novel interactive system for intergenerational learning, aimed at promoting the elderly people’s interactions with younger generations. Initially, the literature was reviewed and experts were interviewed to derive the relevant design principles. The system was constructed accordingly using gesture detection, sound sensing, and VR techniques, and was used to play animation games that simulated traditional puppetry. The system was evaluated statistically by SPSS and AMOS according to the scales of global perceptions of intergenerational communication and the elderly’s attitude via questionnaire surveys, as well as interviews with participants who had experienced the system. Based on the evaluation results and some discussions on the participants’ comments, the following conclusions about the system effectiveness were drawn: (1) intergenerational learning activities based on digital technology can attract younger generations; (2) selecting game topics familiar to the elderly in the learning process encourages them to experience technology; and (3) both generations are more likely to understand each other as a result of joint learning.

Marker-based Tangible Interfaces for Smartphone-based Virtual Reality

In this work, we describe an approach for tangible interaction for smartphone-based VR using visual markers. Smartphone-based VR represents a low barrier to entry given that many people nowadays own a smartphone device and that VR headsets for these devices are affordable and quick to set up when co

The Sonified Textiles within the Text(il)ura Performance: Cross-cultural Tangible Interfaces as Phenomenological Artifacts

This research is working with Sonified Textile Controllers. They can be described as textile sensor-based tangible interfaces. Some of them are part of a disruptive sound performance – an experimental sound art concert that has been presented in various cities in Europe and the Americas called "Text(il)ura"[1]. The three fiber-based controllers presented as part of this performance, The Shipibo Conibo-Style Textile, The Hanap Pacha Quipu, and the Unkuña of Noise, all reference the cultures of Paola Torres Nunez del Prado’s country of origin, Perú. These textile artifacts are works of art as well as devices that fall within the scope of smart textiles and that of human-computer interaction, as they allow the user to experiment with alternative ways of approaching the execution of sound art or electronic music. Referring to both aesthetically and symbolically distinct forms of expression from diverse human communities, these cross-cultural devices are characteristically hybrid and ever-changing: history, myth, craft and technology, gender, transhumanism, sound, visuals and tactility – everything is intertwined in an amalgamation of human knowledges and experiences aiming for a type of universality that does not impose one thought system over the other. The design of the interfaces proposes a different approach to tactile manipulation of electronic sound instruments, where it is common to find controllers made of metal or plastic, and where the aural and tactile sensitivity with which they are first made, transforms the performance itself. The Sonified Textiles aim to redefine musical interfaces, both conceptually and design-wise within the e-textile realm. The research uses a phenomenological framework to go beyond the mind-body dualism and to reconnect with the natural world.

Toward User Experience in ATC: Exploring Novel Interface Concepts for Air Traffic Control

AbstractAir traffic control (ATC) is a safety-critical, cooperative work domain, which faces usability challenges due to technology driven development in the past. In this work, we followed a user-centered design process to explore how novel interaction concepts increase user experience in ATC. Based on controllers’ needs we envisioned one unified interface together with three possible interaction concepts (the mouse interface, flight-dial and tangible interface) addressing different aspects of ATC. We prototypically implemented the interaction concepts and iterated each prototype based on feedback from 24 controllers. Qualitative data from these iterative formative evaluations indicated that controllers prefer interfaces that are efficient to use, minimalistic, customizable and context sensitive. A summative evaluation (N = 12) showed that the hedonic quality of all three concepts were higher compared to the system currently in use. Our results and insights can provide guidance and inspiration for the future design of ATC interfaces.

Correction to: Narrating by Doing: A Bridging Concept for Understanding and Informing the Design of Tangible Interfaces for Storytelling

Correction to: Narrating by Doing: A Bridging Concept for Understanding and Informing the Design of Tangible Interfaces for Storytelling

Tangible and Personalized DS Application Approach in Cultural Heritage: The CHATS Project

Storytelling is widely used to project cultural elements and engage people emotionally. Digital storytelling enhances the process by integrating images, music, narrative, and voice along with traditional storytelling methods. Newer visualization technologies such as Augmented Reality allow more vivid representations and further influence the way museums present their narratives. Cultural institutions aim towards integrating such technologies in order to provide a more engaging experience, which is also tailored to the user by exploiting personalization and context-awareness. This paper presents CHATS, a system for personalized digital storytelling in cultural heritage sites. Storytelling is based on a tangible interface, which adds a gamification aspect and improves interactivity for people with visual impairment. Technologies of AR and Smart Glasses are used to enhance visitors’ experience. To test CHATS, a case study was implemented and evaluated.

The Use of Tangible User Interfaces in K12 Education Settings: A Systematic Mapping Study

Tangible User Interfaces have enriched and expanded the user experience when interacting with computers and smart devices. The monopoly of graphical user interfaces has been broken thanks to the emergence of new complementary technologies that allow for new ways of interacting with computer systems, such as tangible interaction, among others. Due to the scope and number of research articles addressing the Tangible User Interface that have been published, it can now be considered an interaction mechanism that is relatively mature and integrated within society. However, while the application of tangible interfaces in different areas is described as a success, there are only a limited number of research articles about their impact on education and learning systems. As a result, it is difficult to show the actual impact of Tangible User Interface technology in K12 education settings. This study tries to fill this gap by performing a systematic mapping study that shows the current state of research on the impact of this technology in these settings, analyzing the findings and identifying the main advances and limitations of this novel technology.

Tac-Trace: A Tangible User Interface-Based Solution for Teaching Shape Concepts to Visually Impaired Children

A novel, low-cost, Arabic language tangible user interface (TUI)-based solution for teaching and reinforcing the concepts of planar shapes and their relationships to visually impaired (VI) children is presented. The solution employs a computer vision-based system to track tagged 3D-printed objects which children can manipulate to receive audio instructions and feedback in the context of various activities for learning progressively difficult shape concepts in a systematic manner. A graphical user interface (GUI) is also provided for the teachers to create and manage student accounts, track their students’ individual progress and download and print the templates and models of the tangible tokens along with their barcodes. The aim is to provide an engaging, accessible solution that would mitigate the demands on the time, efforts and financial resources of the teachers while allowing children to autonomously learn and review shape concepts at their own pace. By utilizing off-the-shelf components for setting up the TUI and providing templates and CAD models for generating the tangible tokens, we have endeavored to make the system affordable and easily reproducible. The system has been developed in consultation with teachers for VI children to ensure its compliance with current teaching and classroom practices. Initial evaluations of the system with VI children and teachers have validated its potential and yielded valuable design recommendations which would not only inform the design of future iterations of our system but also benefit other researchers seeking to build similar systems.

Cross-Model Correspondences between Chinese Character Radicals and Materials in Tangible User Interfaces

Cross-Model Correspondences between Chinese Character Radicals and Materials in Tangible User Interfaces

Group Cohesion for a Coaching System in Co-located Collaborative Environments

Technologies that support co-located collaboration must not only provide a shared workspace, but also support collaboration. From an observational study, some collaboration problems were identified in groups of people working in a system with a Tangible User Interface. Some of these problems could be identified and prevented with the support of Coaching System. This system encourages interactions between group members through Social Interventions. To develop a Coaching System, it is necessary to know the cohesion between the members of the group, in order to decide the appropriate Social Interventions. In this paper, a model is proposed to represent the social interactions that occur in a group of people when performing a task. Interactions can be analyzed to determine the degree of cohesiveness of a group and support the collaboration.