Gesture Commands Research Articles

A comparative study of user-defined handheld vs. freehand gestures for home entertainment environments

Home entertainment systems have been continuously providing new functionalities to their users in a process in which they evolved from standalone electronic appliances to complex digital milieus superimposed on the home environment. Such complex entertainment environments are in need of appropriate interfaces with gestures standing as a viable option. However, designing gesture commands is a challenging task and today's designs do not always reflect users' natural gestures. For this reason, this work presents the results of a participatory study in which users were involved in the design and evaluation of gesture commands for 22 typical home entertainment tasks. This is the first study to deliver a comparison between user-defined gestures for two different capture technologies: using handheld devices to sense motion and employing freehand gestures and postures. Specifically, we found that: 1 consensus exists among participants in both scenarios; 2 when available, buttons are preferred to motion commands; 3 familiar point & click and drag & drop techniques are naturally reused for new environments; 4 one-hand gestures are preferred to bimanual gestures; 5 hand posture is important yet culture specific; and 6 body-referenced and parametrized gestures appeal to users. We report gesture sets for the handheld and freehand gesture scenarios in order to inform, assist, and inspire practitioners designing the gestural interfaces of our future home environments.

임의 두 지점의 웹 카메라와 퍼지 가비지 모델을 이용한 사용자의 의미 있는 동작 검출

Many research of hand gesture recognition based on vision system have been conducted which enable user operate various electronic devices more easily. 3D position calculation and meaningful gesture classification from similar gestures should be executed to recognize hand gesture accurately. A simple and cost effective method of 3D position calculation and gesture spotting (a task to recognize meaningful gesture from other similar meaningless gestures) is described in this paper. 3D position is achieved by calculation of two cameras relative position through pan/tilt module and a marker regardless with the placed position. Fuzzy garbage model is proposed to provide a variable reference value to decide whether the user gesture is the command gesture or not. The reference is achieved from fuzzy command gesture model and fuzzy garbage model which returns the score that shows the degree of belonging to command gesture and garbage gesture respectively. Two-stage user adaptation is proposed that off-line (batch) adaptation for inter-personal difference and on-line (incremental) adaptation for intra-difference to enhance the performance. Experiment is conducted for 5 different users. The recognition rate of command (discriminate command gesture) is more than 95% when only one command like meaningless gesture exists and more than 85% when the command is mixed with many other similar gestures.

Target-shooting exergame with a hand gesture control

Exertion games (exergames) pose interesting challenges in terms of user interaction techniques. Players are commonly unable to use traditional input devices such as mouse and keyboard, given the body movement requirements of this type of videogames. In this work we propose a hand gesture interface to direct actions in a target-shooting exertion game that is played while exercising on an ergo-bike. A vision-based hand gesture interface for interacting with objects in a 3D videogame is designed and implemented. The system is capable to issue game commands to any computer game that normally responds to mouse and keyboard without modifying the underlying source code of the game. The vision system combines Bag-of-features and Support Vector Machine (SVM) to achieve user-independent and real-time hand gesture recognition. In particular, a Finite State Machine (FSM) is used to build the grammar that generates gesture commands for the game. We carried out a user study to gather feedback from participants, and our preliminary results show the high level of interest from users use this multimedia system that implements a natural way of interaction. Albeit some concerns in terms of comfort, users had a positive experience using our exertion game and they expressed their positive intention to use a system like this in their daily lives.

Can User-Derived Gesture be Considered as the Best Gesture for a Command?: Focusing on the Commands for Smart Home System

With advanced technologies of gesture recognition, various researchers have started to focus on deriving intuitive gestures for commands from users. Most of the studies suggested only the gestures with the highest frequency (a top gesture) for the commands. In this study, a total of two experiments were conducted to identify whether the top gestures in the first experiment are maintained in the second experiment. A total of thirty participants took part in the first experiment and twenty eight in the second experiment. As a result of the experiments, 65% of the top gestures from the first experiment were changed in the second experiment, although the same participants conducted the two experiments. Also, the agreement score was significantly higher in the second experiment.

Three Dimensional Hand Gesture Taxonomy for Commands

Objective: The aim of this study is to suggest three-dimensional(3D) hand gesture taxonomy to organize the intention of his/her decisions on deriving a certain gesture systematically. Background: With advanced technologies of gesture recognition, various researchers have studied to focus on deriving intuitive gestures for commands from users. In most of the previous studies, the reasons for deriving a certain gesture for a command were only used as a reference to group various gestures. Method: A total of eleven studies which categorized gestures accompanied by speech were investigated. Also a case study with thirty participants was conducted to understand gesture-features which derived from the users specifically. Results: Through the literature review, a total of nine gesture-features were extracted. After conducting the case study, the nine gesture-features were narrowed down a total of seven gesture-features. Conclusion: Three-dimensional hand gesture taxonomy including a total of seven gesture-features was developed. Application: Three-dimensional hand gesture taxonomy might be used as a check list to understand the reasons.

Nomadic gestures: A technique for reusing gesture commands for frequent ambient interactions

The age of ambient intelligence has already incorporated gestures into practical applications with the goal of delivering adaptive and personalized interactions. However, practitioners are faced wi...

Nomadic gestures: A technique for reusing gesture commands for frequent ambient interactions

The age of ambient intelligence has already incorporated gestures into practical applications with the goal of delivering adaptive and personalized interactions. However, practitioners are faced with many problems when implementing gesture-based interfaces for such interactive ambient systems. Although gestures offer great opportunity for natural and intuitive interactions, there are currently little to no rules for creating the set of gestures for a given application. Therefore, designers associate gestures and functions by relying only on their own expertize and experience which leads to different systems exposing different standards. This approach does not only create the premises for confusion among users in a future world with such hundreds daily interactions but it also contradicts the goals of ambient intelligence in which interaction should be personalized and adapted to each user. This work introduces a novel concept (nomadic gestures) for reusing a set of user-defined gesture commands in the context of interacting with ambient systems. Nomadic gestures live on each user's personal mobile device and are uploaded to the ambient system prior to interaction. The concept relies on an important shift of perspective strictly adhering to the goals of ambient intelligence: it is not the users that adapt to the interface learning its commands but instead the interface employs the users' own gesture sets with their own preferred function associations.

Real-Time Hand Gesture Detection and Recognition Using Bag-of-Features and Support Vector Machine Techniques

This paper presents a novel and real-time system for interaction with an application or video game via hand gestures. Our system includes detecting and tracking bare hand in cluttered background using skin detection and hand posture contour comparison algorithm after face subtraction, recognizing hand gestures via bag-of-features and multiclass support vector machine (SVM) and building a grammar that generates gesture commands to control an application. In the training stage, after extracting the keypoints for every training image using the scale invariance feature transform (SIFT), a vector quantization technique will map keypoints from every training image into a unified dimensional histogram vector (bag-of-words) after K-means clustering. This histogram is treated as an input vector for a multiclass SVM to build the training classifier. In the testing stage, for every frame captured from a webcam, the hand is detected using our algorithm, then, the keypoints are extracted for every small image that contains the detected hand gesture only and fed into the cluster model to map them into a bag-of-words vector, which is finally fed into the multiclass SVM training classifier to recognize the hand gesture.

Laser spot detection-based computer interface system using autoassociative multilayer perceptron with input-to-output mapping-sensitive error back propagation learning algorithm

This paper presents a new computer interface system based on laser spot detection and moving pattern analysis of the detected laser spots in real-time processing. We propose a systematic method that uses either the frame difference of successive input images or an autoassociative multilayer perceptron (AAMLP) to detect laser spots. The AAMLP is applied only to areas of the input images where the frame difference of the successive images is not effective for detecting laser spots. In order to enhance the detection performance, the AAMLP is trained by a new training algorithm that increases the sensitivity of the input-to-output mapping of the AAMLP allowing a small variation in the input feature of the laser spot image to be successfully indicated. The proposed interface system is also able to keep track of the laser spot and recognize gesture commands. The moving pattern of the laser spot is recognized by using a multilayer perception. It is experimentally shown that the proposed computer interface system is fast enough for real-time operation with reliable accuracy.

Gesture Spotting Using Fuzzy Garbage Model and User Adaptation

Thanks to the rapid advancement of human-computer interaction technologies it is becoming easier for the elderly and/or people with disabilities to operate various electrical systems. Operation of home appliances by using a set of predefined hand gestures is an example. However, hand gesture recognition may fail when the predefined command gestures are similar to some ordinary but meaningless behaviors of the user. This paper uses a gesture spotting method to recognize a designated gesture from other similar gestures. A fuzzy garbage model is proposed to provide a variable reference value to determine whether the user’s gesture is the command gesture or not. Further, the authors propose two-stage user adaptation to enhance recognition performance: that is, off-line (batch) adaptation for inter-person variation and on-line (incremental) adaptation for intra-person variation. For implementation of the two-stage adaptation method, a genetic algorithm (GA) and the steepest descent method are adopted for each stage. Experimental results were obtained for 5 different users with left and up command gestures.

Visual recognition of 3D emblematic gestures in an HMM framework

In human-machine interaction, gestures play an important role as input modality for natural and intuitive interfaces. While the usage of sign languages or crafted command gestures typically requires special user training, the class of gestural actions called “emblems” represents more intuitive yet expressive signs that seem well suited for the task. Following this, an approach for the visual recognition of 3D emblematic arm gestures in a realistic smart room scenario is presented. Hand and head positions are extracted in multiple unsynchronized monocular camera streams, combined to spatiotemporal 3D gesture trajectories and classified in an Hidden Markov Model (HMM) classification and detection framework. The contributions within this article are threefold: Firstly, a solution for the 3D combination of trajectories obtained from unsynchronized cameras and with varying frame rates is proposed. Secondly, the suitability of different alternative feature representations derived from a hand trajectory is assessed, and it is shown that intuitive gestures can be represented by projection on their principal plane of motion. Thirdly, it is demonstrated that a rejection model for gesture spotting and segmentation can be constructed using out-of-domain data. The approach is evaluated on a challenging realistic data set.

A ROBUST AND FRIENDLY HUMAN–ROBOT INTERFACE SYSTEM BASED ON NATURAL HUMAN GESTURES

In this paper, we design a robust and friendly human–robot interface (HRI) system for our intelligent mobile robot based only on natural human gestures. It consists of a triple-face detection method and a fuzzy logic controller (FLC)-Kalman filter tracking system to check the users and predict their current position in a dynamic and cluttered working environment. In addition, through the combined classifier of the principal component analysis (PCA) and back-propagation artificial neural network (BPANN), single and successive commands defined by facial positions and hand gestures are identified for real-time command recognition after dynamic programming (DP). Therefore, the users can instruct this HRI system to make member recognition or expression recognition corresponding to their gesture commands, respectively based on the linear discriminant analysis (LDA) and BPANN. The experimental results prove that the proposed HRI system perform accurately in real-time face detection and tracking, and robustly react to the corresponding gesture commands at eight frames per second (fps).

GeeAir: a universal multimodal remote control device for home appliances

In this paper, we present a handheld device called GeeAir for remotely controlling home appliances via a mixed modality of speech, gesture, joystick, button, and light. This solution is superior to the existing universal remote controllers in that it can be used by the users with physical and vision impairments in a natural manner. By combining diverse interaction techniques in a single device, the GeeAir enables different user groups to control home appliances effectively, satisfying even the unmet needs of physically and vision-impaired users while maintaining high usability and reliability. The experiments demonstrate that the GeeAir prototype achieves prominent performance through standardizing a small set of verbal and gesture commands and introducing the feedback mechanisms.

Actual remote control: a universal remote control using hand motions on a virtual menu

Consumer electronic devices are becoming more complex and diverse. In order to provide users with an easy and convenient way of controlling various consumer electronics, it is necessary to use a universal way to control various devices. This requires using a user-friendly and intuitive interface. We introduce a wristwatch-type of remote that offers a unified way to control various devices. Total of seven gestures that are based on hand motions are defined. These seven gestures are suitable to the wearable wristwatch-type device. For a variety of systems, these gestures can be used in the same way. Gestures are not designed and analyzed in a discrete manner; instead, they are designed in continuous hand motions and analyzed in gesture commands. Also, a method of modeling a virtual menu from the menu on a consumer electronic device is introduced. With the virtual menu, the user is able to control various devices by using gestures. The virtual menu has to reflect hand motion characteristics and represents functions of electronic appliances. The virtual menu is implemented in XML to represent the basic menu construction and its properties. In order to use virtual menu, we introduce how a user's hand motions can be used in a fast and effective way. Finally we compare our wearable remote control to the conventional remote control through user tests in terms of user convenience and efficiency.

Improving computer interaction for older adults

A point and click interface can present difficulties for older adults (particularly those with physical impairments) who may not easily be able to operate a standard input device such as a mouse or trackball. The use of gestural commands, via a multitouch touchscreen device, is an alternative and direct method of interacting with an application. This research investigates whether gestural commands on a multitouch touchscreen device can make interfaces easier to use, learn and remember for older adults.

Sketching and Composing Widgets for 3D Manipulation

AbstractWe present an interface for 3D object manipulation in which standard transformation tools are replaced with transient 3D widgets invoked by sketching context‐dependent strokes. The widgets are automatically aligned to axes and planes determined by the user's stroke. Sketched pivot‐points further expand the interaction vocabulary. Using gestural commands, these basic elements can be assembled into dynamic, user‐constructed 3D transformation systems. We supplement precise widget interaction with techniques for coarse object positioning and snapping. Our approach, which is implemented within a broader sketch‐based modeling system, also integrates an underlying “widget history” to enable the fluid transfer of widgets between objects. An evaluation indicates that users familiar with 3D manipulation concepts can be taught how to efficiently use our system in under an hour.

Papiercraft

Paper persists as an integral component of active reading and other knowledge-worker tasks because it provides ease of use unmatched by digital alternatives. Paper documents are light to carry, easy to annotate, rapid to navigate, flexible to manipulate, and robust to use in varied environments. Interactions with paper documents create rich webs of annotation, cross reference, and spatial organization. Unfortunately, the resulting webs are confined to the physical world of paper and, as they accumulate, become increasingly difficult to store, search, and access. XLibris [Schilit et al. 1998] and similar systems address these difficulties by simulating paper with tablet PCs. While this approach is promising, it suffers not only from limitations of current tablet computers (e.g., limited screen space) but also from loss of invaluable paper affordances. In this article, we describe PapierCraft, a gesture-based command system that allows users to manipulate digital documents using paper printouts as proxies. Using an Anoto [Anoto 2002] digital pen, users can draw command gestures on paper to tag a paragraph, e-mail a selected area, copy selections to a notepad, or create links to related documents. Upon pen synchronization, PapierCraft executes the commands and presents the results in a digital document viewer. Users can then search the tagged information and navigate the web of annotated digital documents resulting from interactions with the paper proxies. PapierCraft also supports real time interactions across mix-media, for example, letting users copy information from paper to a Tablet PC screen. This article presents the design and implementation of the PapierCraft system and describes user feedback from initial use.

A FULL-BODY GESTURE DATABASE FOR HUMAN GESTURE ANALYSIS

This paper presents a full-body gesture database which contains 2D video data and 3D motion data of 14 normal gestures, 10 abnormal gestures and 30 command gestures for 20 subjects. We call this database the Korea University Gesture (KUG) database. Using 3D motion cameras and 3 sets of stereo cameras, we captured 3D motion data and 3 pairs of stereo-video data in 3 different directions for normal and abnormal gestures. In case of command gestures, 2 pairs of stereo-video data were obtained by 2 sets of stereo cameras with different focal lengths in order to capture views of whole body and upper body, simultaneously. The 2D silhouette data was synthesized by separating a subject and background in 2D stereo-video data. In this paper, we describe the gesture capture system, the organization of database, the potential usages of the database and the contact point for the KUG database. We expect that this database would be very useful for the study of 2D/3D human gesture and its application.

Gesture Spotting and Recognition for Human–Robot Interaction

Visual interpretation of gestures can be useful in accomplishing natural human-robot interaction (HRI). Previous HRI research focused on issues such as hand gestures, sign language, and command gesture recognition. Automatic recognition of whole-body gestures is required in order for HRI to operate naturally. This presents a challenging problem, because describing and modeling meaningful gesture patterns from whole-body gestures is a complex task. This paper presents a new method for recognition of whole-body key gestures in HRI. A human subject is first described by a set of features, encoding the angular relationship between a dozen body parts in 3-D. A feature vector is then mapped to a codeword of hidden Markov models. In order to spot key gestures accurately, a sophisticated method of designing a transition gesture model is proposed. To reduce the states of the transition gesture model, model reduction which merges similar states based on data-dependent statistics and relative entropy is used. The experimental results demonstrate that the proposed method can be efficient and effective in HRI, for automatic recognition of whole-body key gestures from motion sequences

Online Handwritten Shape Recognition Using Segmental Hidden Markov Models

We investigate a new approach for online handwritten shape recognition. Interesting features of this approach include learning without manual tuning, learning from very few training samples, incremental learning of characters, and adaptation to the user-specific needs. The proposed system can deal with two-dimensional graphical shapes such as Latin and Asian characters, command gestures, symbols, small drawings, and geometric shapes. It can be used as a building block for a series of recognition tasks with many applications.