Multi-touch Gestures Research Articles

A fitting algorithm based on multi-touch gesture for rapid generation of railway line

Human-computer interaction (HCI) technology plays a critically essential role in the computer-aided design of railway line locations. However, the traditional interactive design with a mouse+keyboard cannot well meet the rapid generation requirements of the railway line during scheme discussion. This research presents a fitting algorithm for the rapid generation of railway lines by using a multi-touch gesture algorithm. The fitting method from free hand-drawing lines to satisfied railway lines is proposed. Then the interactive operation hand gestures are defined and implemented into the railway line location design. The hand-drawing lines generated by defined gestures are automatically fitted with the target horizontal line by using the inflection detection algorithm based on Euclidean Distance (ED). Meanwhile, the vertical line can be fitted by a similar algorithm with an extreme point-to-point (EPP) and chord-to-point distance accumulation (CPDA). Moreover, a real-world example verification is carried out. The multi-touch gesture algorithm is applied for the automatic fitting of the railway line. Compared with the traditional interactive methods, the consumption time of railway line generation by using the multi-touch interactive mode is decreased by about 15%. This research provides fundamental support for rapid scheme discussion of railway line generation based on natural HCI, which is well-matched with modern handheld devices, and the requirements of rapid selection as well as the quick comparison of railway line schemes in the early stage of design.

Rapid design of articulated objects

Designing articulated objects is challenging because, unlike with static objects, it requires complex decisions to be made regarding the form, parts, rig, poses, and motion. We present a novel 3D sketching system for rapidly authoring concepts of articulated objects for the early stages of design, when designers make such decisions. Compared to existing CAD software, which focuses on slowly but elaborately producing models consisting of precise surfaces and volumes, our system focuses on quickly but roughly producing models consisting of key curves through a small set of coherent pen and multi-touch gestures. We found that professional designers could easily learn and use our system and author compelling concepts in a short time, showing that 3D sketching can be extended to designing articulated objects and is generally applicable in film, animation, game, and product design.

Attentive Sequence-to-Sequence Modeling of Stroke Gestures Articulation Performance

Production time of stroke gestures is a fundamental measure of user performance with graphical user interfaces. However, production time represents an overall quantification of the user's gesture articulation process and therefore provides an incomplete picture of such process. Moreover, previous approaches assumed stroke gestures as synchronous point sequences, when most gesture-driven applications have to deal with asynchronous point sequences. Furthermore, deep generative models of human handwriting ignore the temporal information, thereby missing a key component of the user's gesture articulation process. To solve these issues, we introduce Ditto , a sequence-to-sequence deep learning model that estimates the velocity profile of any stroke gesture using spatial information only, providing thus a fine-grained estimation of the moment-by-moment behavior of the user's articulation performance. We show that this unique capability makes Ditto remarkably accurate while handling gestures of any type: unistrokes, multistrokes, and multitouch gestures. Our model, code, and associated web application are available as open source software.

A bioinspired three-dimensional integrated e-skin for multiple mechanical stimuli recognition

Spatiotemporal recognition of multiple mechanical stimuli is essential for electronic skin (e-skin), which can provide more complete and accurate interaction information to enable elaborated functions, such as gesture recognition, object manipulation, and fine tactile discrimination. However, nonspecific sensor response and performance sacrifice for integration limit the perceptual capability of the current systems. Here, we report a bioinspired e-skin that can measure strain, shear and pressure independently with direction information using three-dimensional integrated, mechanically isolated multiple sensors. Novel microstructures of collapsed nanocone clusters, hemi-ellipsoids, and wrinkles are introduced in different sensors to achieve a gauge factor of 6 with a linear working range of 80% (linearity > 0.99) for strain, a sensitivity of 0.1 N−1 for shear force, and a sensitivity of 3.78 kPa−1 for pressure, and all of these sensors possess short response times on the order of 100 ms. The independent, highly sensitive, and fast response of these sensors makes real-time recording and mapping of multiple mechanical stimuli to be achieved. Multi-touch gesture recognition and perception of a red bean (0.065 g) in the hand are demonstrated to illustrate the potential applications in wearables, robotics and bionic prostheses.

Multi-touch gesture recognition of Braille input based on Petri Net and RBF Net

Multi-touch gesture recognition of Braille input based on Petri Net and RBF Net

AugIR Meets GestureCards: A Digital Sketching Environment for Gesture-Based Applications

Abstract To gain a common understanding of an application’s layouts, dialogs and interaction flows, development teams often sketch user interface (UI). Nowadays, they must also define multi-touch gestures, but tools for sketching UIs often lack support for custom gestures and typically just integrate a basic predefined gesture set, which might not suffice to specifically tailor the interaction to the desired use cases. Furthermore, sketching can be enhanced with digital means, but it remains unclear whether digital sketching is actually beneficial when designing gesture-based applications. We extended the AugIR, a digital sketching environment, with GestureCards, a hybrid gesture notation, to allow software engineers to define custom gestures when sketching UIs. We evaluated our approach in a user study contrasting digital and analog sketching of gesture-based UIs.

Examining the usability of touchscreen gestures for adults with DS

This document is part of a global investigation that aims to establish best practices in usability testing of mobile applications, that fit the specific needs of persons with cognitive disabilities. The motivating factor is to improve the quality of life of people with special needs. As a first step, we want to discover what are the skills of people with DS (DS) when using a mobile device. Thanks to its direct manipulation interaction style, multi-touch technology is the ideal mechanism for learning activities with a view to the social inclusion of people with DS. This paper investigates the most common touchscreen gestures on commercial existing software. A commercial analysis was carried out to discover the touch screen gestures used by 103 free software Apps running on a mobile touch screen tablet of 11 where the applications run”. The commercial analysis showed that most applications support tap and drag operations on multi-touch technology. Additionally, the research sought to discover the adults with DS (19–54 years of age) ability to perform other gestures on multi-touch surfaces. A DS user skill study was performed to assess the ability of this user segment to interact with multi-touch surfaces. The analysis involved 53 participants, aged between 19 and 54 years, from two vocational training centers attended by people with DS in Madrid. Authors used the Gesture Games App for experimenting with multi-touch gestures such as tap, double tap, long press, drag, scale up, scale down, and one-finger rotation. Tap, double tap, and drag were the three gestures that the most participant could use. In contrast, participants had difficulty performing one-finger rotation and long press gesture. Our statistical analysis showed that the ability to perform each gesture was independent of gender, age group, and previous experience with touchscreen of a participant.

WITHDRAWN: Implementation of TTS with tactile mobile based assistive device for visually impaired people

Abstract The creation of interactive bit devices poses a brand-new set of text-entry challenges. Once considering blind folks, these are additional evident, as a result of bit screens lack the tactile feedback, they're accustomed once human action with phones. The solutions offered to permit non-visual text input resort to a broad vary of goals, complicated interaction strategies or unfamiliar layouts. We tend to propose Braille sort, a Braille language-based, text-entry system. In favor of a simplified single-finger interface, Braille sort avoids multi-touch gestures, that includes few and enormous targets. The intention of this project is to bridge a spot among the blind and even the scientific development in the world of telecommunications. We’ve got deter-mined to style a system wherever data is distributed via GSM that is any sent to Braille pad. Once the message is received, a buzzer is ready to alert that the visually handicapped person will reach the Braille pad to induce the message. As per the received message, individual switch is pushed victimization servo motor and therefore the message received may be understood by sensing the push buttons. Associate degree automaton app is meant, wherever they received text is reborn to voice message so it becomes additional convenient for the visually handicapped person to listen to and perceive the received message by victimization Wi-Fi.

Multitouch Interaction with Parallel Coordinates on Large Vertical Displays

This paper presents a multitouch vocabulary for interacting with parallel coordinates plots on wall-sized displays. The gesture set relies on principles such as two-finger range definition, a functional distinction of background and foreground for applying the Hold-and-Move concept to wall-sized displays as well as fling-based interaction for triggering and controlling long-range movements. Our implementation demonstrates that out-of-reach problems and limitations regarding multitouch technology and display size can be tackled by the coherent integration of our multitouch gestures. Expert reviews indicate that our gesture vocabulary helps to solve typical analysis tasks that require interaction beyond arms' reach, and it also shows how often certain gestures were used.

Operational Feasibility Analysis of the Multimodal Controller Working Position “TriControl”

Current Air Traffic Controller working positions (CWPs) are reaching their capacity owing to increasing levels of air traffic. The multimodal CWP prototype TriControl combines automatic speech recognition, multitouch gestures, and eye-tracking, aiming for more natural and improved human interaction with air traffic control systems. However, the prototype has not yet undergone systematic evaluation with respect to feasibility. This paper evaluates the operational feasibility, focusing on the system usability of the approach CWP TriControl and its fulfillment of operational requirements. Fourteen controllers took part in a simulation study to evaluate the TriControl concept. The active approach controllers among the group of participants served as the main core target subgroup. The ratings of all controllers in the TriControl assessment were, on average, generally in slight agreement, with just a few showing statistical significance. However, the active approach controllers performed better and rated the system much more positively. The active approach controllers were strongly positive regarding the system usability and acceptance of this early-stage prototype. Particularly, ease of use, user-friendliness, and learnability were perceived very positively. Overall, they were also satisfied with the command input procedure, and would use it for their daily work. Thus, the participating controllers encourage further enhancements to be made to TriControl.

What they can and cannot: A meta-analysis of research on touch and multi-touch gestures by two to seven-year-olds

Abstract There is growing interest in understanding to what extent small children can use multi-touch technologies as there is much scope for using the same to support their educational activities. Touch gesture performance is one of the main desired elements in touch-based e-learning systems designed for children aged two to seven. Accumulated research on touch gesture performance by children has identified several common and frequently used gestures that can be performed by children in the pre-operational stage (from age two to seven). However, none of the existing review articles have covered touch gesture performance of pre-operational children in a single study. The contribution of this paper will be two-fold. First, we present a review of literature that aims to provide more information on this subject. Second, we offer insights into the need to enhance the touch gesture performance of children. Finally, the study suggests some techniques to improve the touch gesture performance of children of any particular age group.

The Comparison between Fine Motor Skills Traditional Activities and Multi-Touch Gestures Screen Design: Expert Reviews

The Comparison between Fine Motor Skills Traditional Activities and Multi-Touch Gestures Screen Design: Expert Reviews

RETRACTED: Multi-touch, gesture-based simulations: Impacts on learning optical imaging and mental model development

Abstract This study examines the effects of different simulation interface designs on students' conceptual understanding of optical imaging and mental model development, and clarifies these design implications on students' behavioral performance and average residence time. We developed simulations of two different interfaces—gesture- and mouse-based simulations—to collect and compare data from two respective student groups. We utilized a quasi-experimental design, and selected and assigned 64 sixth-grade students to the gesture-and mouse-based simulation groups. The results showed a significant difference between the mental model tests (MMT) of the two groups. Specifically, among the three aspects of a mental model, answer (phenomenon) and explanations (which explain the phenomenon) showed more significant differences with the use of gesture-based than mouse-based simulations. The multivariate analysis of variance results indicated that students using gesture-based simulations demonstrated a longer average residence time than those using mouse-based simulations. Further, the study results suggest that compared to a traditional mouse-based interface, the application of multi-touch gestures in optical imaging concept learning enabled not only the development of a more complete mental model but also the improvement of students’ science explanation skills.

Using Convergent Parallel Design Mixed Method to Assess the Usage of Multi-Touch Hand Gestures Towards Fine Motor Skills Among Pre-School Children

The Convergent Parallel Design mixed method is used as the research methodology to assess the usage of multi-touch hand gestures towards fine motor skills among preschool children. This study involved 30 pre-school children aged between 5 to 6 years old from Pusat Asuhan Tunas Islam. The samples are divided into multi-touch gestures group and single touch gestures group. The quantitative data were the test scores of a dot-to-dot activity and the qualitative data were the pre-school children’s emergent behaviors while engaging with mobile apps and controlling writing instruments. The quantitative data were analyzed by using independent sample T-Test and paired sample T-Test. The findings showed that there is a significant difference between the two groups as the independent sample T-Test indicate the value of sig, p = 0.029. The paired sample T-Test also indicate that the pre-test score and post-test score are significantly positive correlated (r = 0.503) and the value of sig, p = 0.005 showed that there is significance difference between pre-test score and post-test score. The statistical data proved that there is an effect of the use multi-touch hand gesture in mobile application towards pre-school children’s fine motor skills which is also supported by qualitative findings. As for conclusion, the use of multi-touch hand gestures can help the pre-school children to implement small movements of their fingers which indirectly develop their fine motor skills thus helping them holding and using pencils and some other instruments appropriately.

GestureCards

Describing gestures in detail has various advantages for project teams: communication is simplified, interaction concepts are documented, and technical decisions are supported. Common gesture notations focus on textual or graphical elements only, but we argue that hybrid approaches have various advantages, especially because some gesture traits are easier to describe with text and others with arrows or icons. We present GestureCards, a hybrid gesture notation mixing graphical and textual elements we developed to describe multi-touch gestures. To evaluate our approach, we compared how users perceive and are affected by different notations. First, we compared GestureCards with a textual notation and observed advantages in terms of speed, correctness, and confidence. Second, we asked participants to compare GestureCards, a textual, and a graphical notation and rate them. The results indicate that the participants' perception depends on the gesture, but GestureCards received consistently good ratings. Third, we monitored several participants working with GestureCards solving practical development tasks for gesture-based applications and they felt well supported by GestureCards.

Projected visible light for 3D finger tracking and device augmentation on everyday objects

Recent advances on the Internet of Things (IoT) lead to an explosion of physical objects being connected to the Internet. These objects sense, compute, interpret what is occurring within themselves and the world, and preferably interact with users. In this work, we present a visible light-enabled finger tracking technique allowing users to perform freestyle multi-touch gestures on everyday object's surface. By projecting encoded patterns onto an object's surface (e.g. paper, display, or table) through a projector, and localizing the user's fingers with light sensors, the proposed system offers users a richer interactive space than the device's existing interfaces. More importantly, results from our experiments indicate that this system can localize ten fingers simultaneously with an accuracy of 1.7 mm and an refresh rate of 84 Hz with only 31 ms delay on WiFi or 23 ms delay on serial communication, easily supporting multi-finger gesture interaction on everyday objects. We also develop two example applications to demonstrate possible scenarios. Finally, we conduct a preliminary exploration of 3D depth inference using the same setup and achieve 2.43 cm depth estimation accuracy.

THE ASSESSMENT OF MULTI-TOUCH HAND GESTURES TOWARDS FINE MOTOR SKILLS AMONG PRE-SCHOOL CHILDREN

This study aims to evaluate the usage of multi-touch hand gestures towards fine motor skills among preschool children. This study implements a mixed methods involving 30 pre-school children aged between 5 to 6 years old from Pusat Asuhan Tunas Islam as respondents for quantitative method and supported by interviewing teachers as qualitative method. The samples consist of multi-touch and single touch gestures groups, leading to a pre-test and a post-test of a Dot-to-dot activity after implementing the“ Mari Belajar Bentuk” mobile application. Findings indicate that the post-test mean and multi-touch gestures mean is higher than the pre-test mean and single-touch gesture mean respectively. This shows that the multi-touch hand gestures effects the fine motor skills development which is also supported by interview results. As for conclusion, the multi-touch hand gestures are an essential factor when designing mobile applications for pre-school children

FlyCam: Multitouch Gesture Controlled Drone Gimbal Photography

We introduce FlyCam—a novel framework—for gimbal drone camera photography. Our approach abstracts the camera and the drone into a single flying camera object so that the user does not need to think about the drone movement and camera control as two separate actions. The camera is controlled from a single mobile device with six simple touch gestures such as rotate, move forward, yaw, and pitch. The gestures are implemented as seamless commands that combine the gimbal motion with the drone movement. Moreover, we add a sigmoidal motion response that compensates for abrupt drone swinging when moving horizontally. The smooth and simple camera movement has been evaluated by user study, where we asked 20 human subjects to mimic a photograph taken from a certain location. The users used both the default two joystick control and our new touch commands. Our results show that the new interaction performed better in both intuitiveness and easiness of navigation. The users spent less time on task, and the System Usability Scale index of our FlyCam method was 75.13, which is higher than the traditional dual joystick method that scored at 67.38. Moreover, the NASA task load index also showed that our method had lower workload than the traditional method.

Evaluating Children’s Interaction with Touchscreens From 0 to 8 Years Old

Children start using mobile devices at a very young age. In this paper, we review the research done to evaluate children’s capabilities and limitations when interacting with multi-touch screens. The different factors affecting children’s touch gestures performance are evaluated. It was found that children are able to perform several complex touch gestures that are not currently used in applications targeting their age. The current research showed that children’s usage of different multi-touch gestures is affected by their age and screen size. Design guidelines are recommended to improve young children’s usability of multi-touch devices.

Speech Input as an Alternative Mode to Perform Multi-touch Gestures

Speech is fundamental and the most dominant form of communication. Speech input may facilitate natural interaction between humans and computers. For children, this input mode supports their interaction with application systems. This study addresses speech input as an alternative mode in order to improve multi-touch gesture interactions. Previous studies show that children difficult to perform multi-touch gestures successfully. In fact, multi-touch is parts of basic core gestures that have been adapted to most of the learning applications. This study uses the Wizard-of-Oz method and posts interview, involving nine preschool children between ages of four to six years old. Results of the study show children’s interactional ability using speech input towards systems and positive feedback received from them regarding the prototype of the systems. The findings from this study highlight the opportunities and challenges in using speech input to increase the success of children’s interaction with multi-touch gestures.