Candidate Gestures Research Articles

Motion Gesture Delimiters for Smartwatch Interaction

Smartwatches are increasingly popular in our daily lives. Motion gestures are a common way of interacting with smartwatches, e.g., users can make a movement in the air with their arm wearing the watch to trigger a specific command of the smartwatch. Motion gesture interaction can compensate for the small screen size of the smartwatch to some extent and enrich smartwatch-based interactions. An important aspect of motion gesture interaction lies in how to determine the start and end of a motion gesture. This paper is aimed at selecting gestures as suitable delimiters for motion gesture interaction with the smartwatch. We designed six gestures (“shaking wrist left and right,” “shaking wrist up and down,” “holding fist and opening,” “turning wrist clockwise,” “turning wrist anticlockwise,” and “shaking wrist up”) and conducted two experiments to compare the performance of these six gestures. Firstly, we used dynamic time warping (DTW) and feature extraction with KNN ( K -nearest neighbors) to recognize these six gestures. The average recognition rate of the latter algorithm for the six gestures was higher than that of the former. And with the latter algorithm, the recognition rate for the first three of the six gestures was greater than 98%. According to experiment one, gesture 1 (shaking wrist left and right), gesture 2 (shaking wrist up and down), and gesture 3 (holding fist and opening) were selected as the candidate delimiters. In addition, we conducted a questionnaire data analysis and obtained the same conclusion. Then, we conducted the second experiment to investigate the performance of these three candidate gestures in daily scenes to obtain their misoperation rates. The misoperation rates of two candidate gestures (“shaking wrist left and right” and “shaking wrist up and down”) were approximately 0, which were significantly lower than that of the third candidate gesture. Based on the above experimental results, gestures “shaking wrist left and right” and “shaking wrist up and down” are suitable as motion gesture delimiters for smartwatch interaction.

L-Sign: Large-Vocabulary Sign Gestures Recognition System

Understanding sign gestures is an essential step to helping individuals with hearing impaired. The existing works can only identify a small set of gestures accurately and the accuracy rate drops sharply with an increasing number of gestures. Because there are two challenges—a large number of similar gestures in sign language and the various signing speed of different people. Based on commercial smart bracelets, this article proposes a large-vocabulary sign language recognition system (which we call L-sign). First, we propose an entropy-based forward and backward matching algorithm to segment each gesture signal. Second, we design a gesture recognizer including a candidate gesture generator and semantic-based voter. The candidate gesture generator is aimed at providing candidate gesture designs based on a 3-branch convolutional neural network. The purpose of a semantic-based voter is to select the target gesture from candidate gestures by scoring, where the semantic distances between the last gesture in the current sentence and any candidate gestures is calculated, and a multilayer k-means algorithm is proposed to obtain a multilayer sign word structure to complete the scores of candidate gestures. Lastly, we deployed L-sign on the MYO bracelet. For 200 commonly used Chinese sign gestures, the experimental results show that the average accuracy rate was greater than 90%.

User-Defined Gestures for Mid-Air Interaction: A Comparison of Upper Limb Muscle Activity, Wrist Kinematics, and Subjective Preference

ABSTRACT Traditional gesture elicitation studies generally adopted the frequency ratio to select popular gestures among users. However, the chosen gestures were not always optimal and might pose a potential risk of musculoskeletal disorder under long-term use. The purpose of our research was to apply a novel assessment system combining the indices of electromyography (EMG), electrogoniometry, and subjective preference to the end-user elicitation experiment. In this study, we conducted a two-stage experiment to compare and analyze the results of the physiological and psychological measures for 33 candidate gestures of 16 given commands. Our results indicated that this assessment system could effectively determine the candidate gestures with lower physiological loads for all given commands except Maximize, and most of the selected gestures were in line with users’ mental models. Finally, we developed an optimal user-defined gesture set for the commands accommodated to human-computer interaction (HCI).

Політико-психологічні маніпуляції у передвиборчій кампанії президента України 2019 року

Стаття присвячена темі, що завжди актуальна для демократичних сус­пільств, — маніпуляцій у передвиборчих кампаніях. Оскільки медіаграмотність як освітня дисципліна не впроваджена в широкий загал в українському суспільстві — це призводить до того, що громадськістю маніпулюють, спотворюючи реальність. Саме використання різних технік, «брудних» технологій, тактик та стратегій маніпуляцій утруднює існування демократії.На основі медіадослідження (використання електронних джерел, соціальних мереж та відеохостингів) автор аналізує те, які маніпуляційні стратегії (за типологією Н. Хомського) були використані кандидатами у передвиборчій кампанії Президента України 2019 року, а також вплив технік і технологій, що були використані в рамках маніпулятивних стратегій.Було досліджено здійснення вербального і невербального впливу на електорат: лінгвістично, психологічно, нейролінгвістично; були проаналізовані міміка, жести кандидатів; зауважено як посил до емоційно-підсвідомої складової суспільства може стати вирішальним у передвиборчій гонці, яким чином «відволікання від важливого» може сприяти кандидату сформувати свій імідж. Також розглянуто архетипи, які використовувалися в передвиборчій кампанії, їх суть і значення для суспільства, «навішування ярликів»; поступовість рішень, що зробили кандидата популярним серед громадськості; тренди, котрі були створені іміджмейкерами кандидатів задля підвищення рейтингу та підтримки. З’ясовано як можна використати «провину» на свою користь; відстежено маніпуляції соціологічними дослідженнями; розмову з народом, як з дитиною і гру в «простонародність». Автором досліджено як гіперболізація фактів без порівняння може викликати серйозне обурення суспільства та зруйнувати імідж опонента. Був проведений ґрунтовний аналіз впливу на маси, який застосовувався за допомогою маніпулятивних технологій і стратегій.Об’єктом дослідження було обрано передвиборчі кампанії трьох кандидатів: В. Зе­ленського, П. Порошенка, Ю. Тимошенко.

FLGR: Fixed Length Gists Representation Learning for RNN-HMM Hybrid-Based Neuromorphic Continuous Gesture Recognition.

A neuromorphic vision sensors is a novel passive sensing modality and frameless sensors with several advantages over conventional cameras. Frame-based cameras have an average frame-rate of 30 fps, causing motion blur when capturing fast motion, e.g., hand gesture. Rather than wastefully sending entire images at a fixed frame rate, neuromorphic vision sensors only transmit the local pixel-level changes induced by the movement in a scene when they occur. This leads to advantageous characteristics, including low energy consumption, high dynamic range, a sparse event stream and low response latency. In this study, a novel representation learning method was proposed: Fixed Length Gists Representation (FLGR) learning for event-based gesture recognition. Previous methods accumulate events into video frames in a time duration (e.g., 30 ms) to make the accumulated image-level representation. However, the accumulated-frame-based representation waives the friendly event-driven paradigm of neuromorphic vision sensor. New representation are urgently needed to fill the gap in non-accumulated-frame-based representation and exploit the further capabilities of neuromorphic vision. The proposed FLGR is a sequence learned from mixture density autoencoder and preserves the nature of event-based data better. FLGR has a data format of fixed length, and it is easy to feed to sequence classifier. Moreover, an RNN-HMM hybrid was proposed to address the continuous gesture recognition problem. Recurrent neural network (RNN) was applied for FLGR sequence classification while hidden Markov model (HMM) is employed for localizing the candidate gesture and improving the result in a continuous sequence. A neuromorphic continuous hand gestures dataset (Neuro ConGD Dataset) was developed with 17 hand gestures classes for the community of the neuromorphic research. Hopefully, FLGR can inspire the study on the event-based highly efficient, high-speed, and high-dynamic-range sequence classification tasks.

User-Defined Gestures for Gestural Interaction: Extending from Hands to Other Body Parts

ABSTRACTMost gestural interaction studies on gesture elicitation have focused on hand gestures, and few have considered the involvement of other body parts. Moreover, most of the relevant studies used the frequency of the proposed gesture as the main index, and the participants were not familiar with the design space. In this study, we developed a gesture set that includes hand and non-hand gestures by combining the indices of gesture frequency, subjective ratings, and physiological risk ratings. We first collected candidate gestures in Experiment 1 through a user-defined method by requiring participants to perform gestures of their choice for 15 most commonly used commands, without any body part limitations. In Experiment 2, a new group of participants evaluated the representative gestures obtained in Experiment 1. We finally obtained a gesture set that included gestures made with the hands and other body parts. Three user characteristics were exhibited in this set: a preference for one-handed movements, a preference for gestures with social meaning, and a preference for dynamic gestures over static gestures.

Streamlined and accurate gesture recognition with Penny Pincher

Penny Pincher is a recently introduced template matching $-family gesture recognizer that exhibits competitive accuracy with even just one template. However, our recognizer is also able to rapidly compare a candidate gesture against numerous templates in a short amount of time, as compared to other recognizers, in order to achieve higher accuracy within a given time budget. Penny Pincher achieves this goal by reducing the template matching process to merely addition and multiplication; by avoiding translation, scaling, and rotation; and by avoiding calls to expensive geometric functions. In an evaluation compared against four other $-family recognizers, in three of our six datasets, Penny Pincher achieves the highest accuracy of all recognizers reaching 97.5%, 99.8%, and 99.9% user independent recognition accuracy, while remaining competitive with the three remaining datasets. Further, when a time constraint is imposed, our recognizer always exhibits the highest accuracy, realizing a reduction in recognition error of between 83% and 99% in most cases as Penny Pincher is able to process five times as many templates in the same amount of time as its closest competitor. Further, in this extended work, we also evaluate the effectiveness of Penny Pincher in a stressful setting using a video game prototype that makes heavy use of gestures, so that rushed and malformed gesture articulation is more likely. Our evaluation was conducted with a 24 participant between-subject user study of Protractor and Penny Pincher. Training data and in-game data collected during the user study was further used to evaluate several $-family recognizers. Again we find that our recognizer is on par with or better than the others, reducing the recognition error by as much as 5.8% to 10.4% with just a small number of templates per gesture.