Stroke Gestures Research Articles

Gestures à Go Go

Training a high-quality gesture recognizer requires providing a large number of examples to enable good performance on unseen, future data. However, recruiting participants, data collection, and labeling, etc., necessary for achieving this goal are usually time consuming and expensive. Thus, it is important to investigate how to empower developers to quickly collect gesture samples for improving UI usage and user experience. In response to this need, we introduce Gestures à Go Go ( g 3), a web service plus an accompanying web application for bootstrapping stroke gesture samples based on the kinematic theory of rapid human movements. The user only has to provide a gesture example once, and g 3 will create a model of that gesture. Then, by introducing local and global perturbations to the model parameters, g 3 generates from tens to thousands of synthetic human-like samples. Through a comprehensive evaluation, we show that synthesized gestures perform equally similar to gestures generated by human users. Ultimately, this work informs our understanding of designing better user interfaces that are driven by gestures.

Differences and Similarities between Finger and Pen Stroke Gestures on Stationary and Mobile devices

This study investigated differences and similarities between finger and pen gestures on stationary devices (sitting posture) and mobile devices (sitting and walking postures). The recorded gestures were analyzed according to multiple gesture features. We found (1) pen and index finger gestures were different in features like size ratio but similar in features like angle difference ; (2) implement (pen vs. index finger vs. thumb) interacted with gesture complexity and size in features like articulation time ; (3) features like time and shape distance , were different between the pen and index finger on mobile devices (walking) but similar on stationary devices; (4) one-handed thumb gestures had worse performances than index finger gestures by time and accuracy in sitting but similar performances in walking; and (5) for the three implements, gesture drawing time and accuracy on mobile devices reduced from sitting to walking condition. We discuss these findings with implications for future gesture design and research.

Solution strokes

Downward stroke gestures which are produced in conjunction with the solution of speaking trouble, such as a word search, during English language discussions among Japanese university students, are described and labeled solution strokes. Similar gestures that involve striking something are labeled solution strikes. These gestures can be understood as the gestural component of the utterance which solves the trouble. They index the subjective experience of the trouble and its solution and possibly a positive affective stance toward the solution. Solution strokes are argued to be the gestural component of a solution and it is shown that understanding the meaning of a solution stroke requires attending to how it is situated in the local context.

Humanoid Tactile Gesture Production using a Hierarchical SOM-based Encoding

The existence of cortical hierarchies has long since been established and the advantages of hierarchical encoding of sensor-motor data for control, have long been recognized. Less well understood are the developmental processes whereby such hierarchies are constructed and subsequently used. This paper presents a new algorithm for encoding sequential sensor and actuator data in a dynamic, hierarchical neural network that can grow to accommodate the length of the observed interactions. The algorithm uses a developmental robotics methodology as it extends the Constructivist Learning Architecture, a computational theory of infant cognitive development. This paper presents experimental data demonstrating how the extended algorithm goes beyond the original theory by supporting goal oriented control. The domain studied is the encoding and reproduction of tactile gestures in humanoid robots. In particular, we present results from using a Programming by Demonstration approach to encode a stroke gesture. Our results demonstrate how the novel encoding enables a Nao humanoid robot with a touch sensitive fingertip to successfully encode and reproduce a stroke gesture in the presence of perturbations from internal and external forces.

Behavioral Biometrics Authentication Using Gesture-Based

The need to secure private or sensitive information in our mobile devices, desktop and laptop computers is one of the main problems in information security. Touch gesture behavioral biometrics provides a more convenient method and has so many advantages for authentication purposes. This study provides a comprehensive and critical study of features employed in stroke gesture. In order to reduce the weakness of physiological and behavioral biometrics, extracting the finger pressure, finger size and time for touch gesture features and will addressee and solve the problem. Future work can be done by implementing and testing the touch gesture features and identifies the most suitable classification method.

Design and Evaluation of Cell Phone Pointing Interface for Robot Control

In this work a pointing interface based on human gestures using a mobile phone accelerometer for interaction with robots is proposed. Through this interface the user can sketch stroke gestures on a computer screen using the cell phone accelerometer to make a selection and instruct a robot to perform a task. Selection, cancelation and movement, as well as some additional commands such as stop, pause and resume are supported. All the projected gestures are processed using known image analysis algorithms providing instantaneous position and path information. The proposed interface is experimentally evaluated and the results show that it is efficient, natural, robust and easy to use. It also provides asynchronous robot control with minimum operator effort and engagement.

An advanced nonlinear model of a low-g MEMS accelerometer for a computer pen

This paper presents an improvement in the gesture reconstruction for a computer pen using the inertial principle. The computer pen is equipped with a complete, miniaturized, six-degrees-of-freedom, inertial measurement unit in order to reduce the error of the pen’s orientation during the gesture stroke and to reduce the error of the calculated pen tip’s position on the piece of paper. The performance of the computer pen is improved by calibrating the MEMS accelerometer. The presented research focuses on the advanced modeling of the nonlinear voltage output of a triple-axis low-g MEMS accelerometer. The model considers the interacting influence of the low-g accelerations between the sensing axes in an arbitrarily oriented pen without any required knowledge about the underlying performance of the MEMS accelerometer. In combination with the attitude-compensation and zero-velocity-compensation techniques the positional accuracy of the pen’s tip on the paper is greatly improved.

Making touch-based kiosks accessible to blind users through simple gestures

Touch-based interaction is becoming increasingly popular and is commonly used as the main interaction paradigm for self-service kiosks in public spaces. Touch-based interaction is known to be visually intensive, and current non-haptic touch-display technologies are often criticized as excluding blind users. This study set out to demonstrate that touch-based kiosks can be designed to include blind users without compromising the user experience for non-blind users. Most touch-based kiosks are based on absolute positioned virtual buttons which are difficult to locate without any tactile, audible or visual cues. However, simple stroke gestures rely on relative movements and the user does not need to hit a target at a specific location on the display. In this study, a touch-based train ticket sales kiosk based on simple stroke gestures was developed and tested on a panel of blind and visually impaired users, a panel of blindfolded non-visually impaired users and a control group of non-visually impaired users. The tests demonstrate that all the participants managed to discover, learn and use the touch-based self-service terminal and complete a ticket purchasing task. The majority of the participants completed the task in less than 4 min on the first attempt.

Visually-Cued Touch Gestures for Accurate Mobile Interaction

Portable devices such as tablet personal computers can allow a user to perform tasks while walking. These devices provide new interaction opportunities and challenges. LocalSwipes is a new interaction technique that utilizes localized, multi-touch, radial stroke gestures to reduce the difficulty of interacting with large data sets on mobile devices. LocalSwipes adapts traditional point-to-click widgets to visually support gestural input. A user evaluation demonstrated that LocalSwipes results in fewer interaction errors and is preferred when compared to a touch-based interface incorporating traditional graphical user interface widgets.

High contextual sensitivity of metaphorical expressions and gesture blending: A video event-related potential design

Human communication in a natural context implies the dynamic coordination of contextual clues, paralinguistic information and literal as well as figurative language use. In the present study we constructed a paradigm with four types of video clips: literal and metaphorical expressions accompanied by congruent and incongruent gesture actions. Participants were instructed to classify the gesture accompanying the expression as congruent or incongruent by pressing two different keys while electrophysiological activity was being recorded. We compared behavioral measures and event related potential (ERP) differences triggered by the gesture stroke onset. Accuracy data showed that incongruent metaphorical expressions were more difficult to classify. Reaction times were modulated by incongruent gestures, by metaphorical expressions and by a gesture–expression interaction. No behavioral differences were found between the literal and metaphorical expressions when the gesture was congruent. N400-like and LPC-like (late positive complex) components from metaphorical expressions produced greater negativity. The N400-like modulation of metaphorical expressions showed a greater difference between congruent and incongruent categories over the left anterior region, compared with the literal expressions. More importantly, the literal congruent as well as the metaphorical congruent categories did not show any difference. Accuracy, reaction times and ERPs provide convergent support for a greater contextual sensitivity of the metaphorical expressions.

Gesture and metaphor comprehension: Electrophysiological evidence of cross-modal coordination by audiovisual stimulation

In recent years, studies have suggested that gestures influence comprehension of linguistic expressions, for example, eliciting an N400 component in response to a speech/gesture mismatch. In this paper, we investigate the role of gestural information in the understanding of metaphors. Event related potentials (ERPs) were recorded while participants viewed video clips of an actor uttering metaphorical expressions and producing bodily gestures that were congruent or incongruent with the metaphorical meaning of such expressions. This modality of stimuli presentation allows a more ecological approach to meaning integration. When ERPs were calculated using gesture stroke as time-lock event, gesture incongruity with metaphorical expression modulated the amplitude of the N400 and of the late positive complex (LPC). This suggests that gestural and speech information are combined online to make sense of the interlocutor’s linguistic production in an early stage of metaphor comprehension. Our data favor the idea that meaning construction is globally integrative and highly context-sensitive.

“Writing with music”

We investigate the use of auditory feedback in pen-gesture interfaces in a series of informal and formal experiments. Initial iterative exploration showed that gaining performance advantage with auditory feedback was possible using absolute cues and state feedback after the gesture was produced and recognized. However, gaining learning or performance advantage from auditory feedback tightly coupled with the pen-gesture articulation and recognition process was more difficult. To establish a systematic baseline, Experiment 1 formally evaluated gesture production accuracy as a function of auditory and visual feedback. Size of gestures and the aperture of the closed gestures were influenced by the visual or auditory feedback, while other measures such as shape distance and directional difference were not, supporting the theory that feedback is too slow to strongly influence the production of pen stroke gestures. Experiment 2 focused on the subjective aspects of auditory feedback in pen-gesture interfaces. Participants' rating on the dimensions of being wonderful and stimulating was significantly higher with musical auditory feedback. Several lessons regarding pen gestures and auditory feedback are drawn from our exploration: a few simple functions such as indicating the pen-gesture recognition results can be achieved, gaining performance and learning advantage through tightly coupled process-based auditory feedback is difficult, pen-gesture sets and their recognizers can be designed to minimize visual dependence, and people's subjective experience of gesture interaction can be influenced using musical auditory feedback. These lessons may serve as references and stepping stones toward future research and development in pen-gesture interfaces with auditory feedback.