Gesture Interface Research Articles

A Framework for Designing Closed-loop Hand Gesture Interface Incorporating Compatibility between Human and Monocular Device

Objective: This paper targets a framework of a hand gesture based interface design. Background: While a modeling of contact-based interfaces has focused on users" ergonomic interface designs and real-time technologies, an implementation of a contactless interface needs error-free classifications as an essential prior condition. These trends made many research studies concentrate on the designs of feature vectors, learning models and their tests. Even though there have been remarkable advances in this field, the ignorance of ergonomics and users" cognitions result in several problems including a user"s uneasy behaviors. Method: In order to incorporate compatibilities considering users" comfortable behaviors and device"s classification abilities simultaneously, classification-oriented gestures are extracted using the suggested human-hand model and closed-loop classification procedures. Out of the extracted gestures, the compatibility-oriented gestures are acquired though human"s ergonomic and cognitive experiments. Then, the obtained hand gestures are converted into a series of hand behaviors - Handycon - which is mapped into several functions in a mobile device. Results: This Handycon model guarantees users" easy behavior and helps fast understandings as well as the high classification rate. Conclusion and Application: The suggested framework contributes to develop a hand gesture-based contactless interface model considering compatibilities between human and device. The suggested procedures can be applied effectively into other contactless interface designs.

A Study on Developmental Direction of Interface Design for Gesture Recognition Technology

Objective: Research on the transformation of interaction between mobile machines and users through analysis on current gesture interface technology development trend. Background: For smooth interaction between machines and users, interface technology has evolved from

Design and Reliability Analysis of Image Type Head Gesture Interface in Distributed Control System

In this study, a novel method for head-controlled human-machine interface system is presented. The proposed system is composed of the image type head gesture interface and the distributed control system (DCS). In the head gesture interface, the frame grabber and CCD camera track the lighting pointer of double-source eyeglass frame. The computer programs with new algorithms locate the center-point of the head, transferring it to the screen coordinates, and then the user can control the cursor by head motions. The DCS with controlling commands and could be used for driving the relative hardware to assist the system more stably. The failure rate was obtained by testing in 60 days. The present system can provide an economical and effective method for reliability. It is also useful in the field of interface and technology.

Design and reliability analysis of image type head gesture interface in distributed control system

Design and reliability analysis of image type head gesture interface in distributed control system

퀴즈게임의 체감형 제스처 인터페이스 프로토타입 개발

This paper introduce a quiz game contents based on gesture interface. We analyzed the off-line quiz games, extracted its presiding components, and digitalized them so that the proposed game contents is able to substitute for the off-line quiz games. We used the Kinect camera to obtain the depth images and performed the preprocessing including vertical human segmentation, head detection and tracking and hand detection, and gesture recognition for hand-up, hand vertical movement, fist shape, pass and fist-and-attraction. Especially, we defined the interface gestures designed as a metaphor for natural gestures in real world so that users are able to feel abstract concept of movement, selection and confirmation tangibly. Compared to our previous work, we added the card compensation process for completeness, improved the vertical hand movement and the fist shape recognition methods for the example selection and presented an organized test to measure the recognition performance. The implemented quiz application program was tested in real time and showed very satisfactory gesture recognition results.

Touchless gesture user interface for interactive image visualization in urological surgery

In the operating room (OR) a touchless interface is an ideal solution since it does not demand any physical contact and still can provide the necessary control features in a cleansed and sterilized environment. Using open-source software libraries and image processing techniques, we implemented a hand tracking and gesture recognition system based on the Kinect device that enables surgeon to successfully touchlessly navigate through the image in the intraoperative setting through a personal computer. We used the InVesalius software, which provides high-quality 3D reconstruction of medical images. Computed tomography data were intraoperatively fruitfully accessed through a simple and cheap solution in 4 tumor enucleations in 3 male patients in whom elective nephron-sparing surgeries were performed for small non-exophitic tumors. Mean tumor length was 2.7 cm (2.1, 2.7, 2.9 and 3.1 cm), and real-time ultrasound was not necessary for intraoperative identification in 3 of 4 endorenal tumors. All pathological reports revealed renal cell carcinoma, Fuhrman grade I, and negative inked surgical margins. No intra- or postoperative complication was reported. For the first time in the literature, a touchless user interface solution applying the Kinect device showed to be very efficient and enabled a low-cost and accurate control of the software InVesalius intraoperative, just using hand gestures. It can be used with any mouse-controlled software, opening an avenue for potential applications in many other areas, such as data visualization, augmented reality, accessibility, and robotics. The further validation and advancement of this technology are underway.

Magic Ring: A Finger-Worn Device for Multiple Appliances Control Using Static Finger Gestures

An ultimate goal for Ubiquitous Computing is to enable people to interact with the surrounding electrical devices using their habitual body gestures as they communicate with each other. The feasibility of such an idea is demonstrated through a wearable gestural device named Magic Ring (MR), which is an original compact wireless sensing mote in a ring shape that can recognize various finger gestures. A scenario of wireless multiple appliances control is selected as a case study to evaluate the usability of such a gestural interface. Experiments comparing the MR and a Remote Controller (RC) were performed to evaluate the usability. From the results, only with 10 minutes practice, the proposed paradigm of gestural-based control can achieve a performance of completing about six tasks per minute, which is in the same level of the RC-based method.

1515 TOUCHLESS GESTURE USER INTERFACE FOR INTERACTIVE IMAGE VISUALIZATION IN UROLOGICAL SURGERY

You have accessJournal of UrologyTechnology & Instruments: Surgical Education & Skills Assessment/Ureteroscopy II1 Apr 20121515 TOUCHLESS GESTURE USER INTERFACE FOR INTERACTIVE IMAGE VISUALIZATION IN UROLOGICAL SURGERY Guilherme Ruppert, Leonardo Reis, Paulo Amorim, Tiago Moraes, and Jorge Silva Guilherme RuppertGuilherme Ruppert Campinas, Brazil More articles by this author , Leonardo ReisLeonardo Reis Campinas, Brazil More articles by this author , Paulo AmorimPaulo Amorim Campinas, Brazil More articles by this author , Tiago MoraesTiago Moraes Campinas, Brazil More articles by this author , and Jorge SilvaJorge Silva Campinas, Brazil More articles by this author View All Author Informationhttps://doi.org/10.1016/j.juro.2012.02.1282AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookTwitterLinked InEmail INTRODUCTION AND OBJECTIVES In the operating room a touchless interface is an ideal solution since it does not demand any physical contact and still can provide the necessary control features in a cleansed and sterilized environment. METHODS Using open-source software libraries and image processing techniques, we implemented a hand tracking and gesture recognition system based on the Kinect device that enables surgeon to successfully navigate through the image in the intraoperative setting. We used the InVesalius software, which provides high-quality 3D reconstruction of medical images. The hand of the user is tracked and the corresponding position is used to move the mouse pointer and button clicks events are also virtually generated. RESULTS Nephron-sparing surgery can be facilitated by using intra-operative hand gestures to provide touchless control of 3D medical imaging software. Computed tomography data from patients with renal masses were fruitfully accessed during surgery through a simple and cheap solution. The model was easily transferred to a personal computer, allowing the surgeon to touchlessly navigate through the 3D images in the intraoperative setting. We have utilized the proposed solution in 3 tumor enucleations in 2 male patients in whom elective nephron sparing surgery was performed for small non-exophitic tumors. Mean tumor length was 2.7 cm (2.1, 2.9 and 3.1 cm) and real-time ultrasound was not necessary in 2 tumors. All surgeries were guided by image review through gesture interface during the procedure, allowing anatomical support based on the 3D imaging previously obtained by CT scan. All pathological reports revealed renal cell carcinoma, Furhman grade I, and negative inked surgical margins. CONCLUSIONS The solution was successfully used intraoperative and can be used with any mouse-controlled software. This is an important step in bringing technology to the operating room to assist the surgeon directly, opening an avenue for potential applications in many other areas, such as: data visualization, augmented reality, accessibility and robotics. © 2012 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 187Issue 4SApril 2012Page: e613-e614 Advertisement Copyright & Permissions© 2012 by American Urological Association Education and Research, Inc.MetricsAuthor Information Guilherme Ruppert Campinas, Brazil More articles by this author Leonardo Reis Campinas, Brazil More articles by this author Paulo Amorim Campinas, Brazil More articles by this author Tiago Moraes Campinas, Brazil More articles by this author Jorge Silva Campinas, Brazil More articles by this author Expand All Advertisement Advertisement PDF downloadLoading ...

Playing into the wild: A gesture-based interface for gaming in public spaces

Gestural-based interfaces have become one of the fundamental technologies that can determine the success of new computer games. In fact, computer games today offer interaction paradigms that go well beyond the use of remote controls, letting players directly perform exchanges with the objects and characters that compose the virtual worlds that are displayed in front of them. To perform such exchanges, new algorithms and technologies have been devised which include advanced visual recognition schemes, new video cameras and accelerometer sensors. At the same time, other important trends are also quietly emerging in the same domain: game designers, in fact, are slowly shifting their attention out of the walls of gaming fanatics homes, broadening their interests to computer games that can be played in public spaces, as exhibitions and museums. However, to the best of our knowledge, only a very limited amount of research experiences have taken into account the problem of producing computer games, based on gesture-based interfaces that well suit such settings. Hence, in this paper we address the problem of differentiating the design of a gesture-based interface for a console from the problem of designing it for a public space setting. Moreover, we will show that within a public space, it is possible to narrow down the vision algorithms that can well support the recognition of complex actions, whereas solely relying on a simple webcam. In particular, we will describe the design and implementation of an interface that well suits public immersive scenarios, since it is based on a simple and efficient set of algorithms which, combined with the intelligence given by the knowledge of the context of where a game is played, leads to a fast and robust interpretation of hand gestures. To witness this last aspect, we will report on the results obtained from the deployment of a computer game we specifically developed for public spaces, termed Tortellino X-Perience, which has been enjoyed by hundreds of visitors at the 2010 Shanghai World Expo.

Evaluation of Gesture Based Interfaces for Medical Volume Visualization Tasks

Interactive systems are increasingly used in medical applications with the widespread availability of various imaging modalities. Gesture-based interfaces can be beneficial to interact with these kinds of systems in a variety of settings, as they can be easier to learn and can eliminate several shortcomings of traditional tactile systems, especially for surgical applications. We conducted two user studies that explore different gesture-based interfaces for interaction with volume visualizations. The first experiment focused on rotation tasks, where the performance of the gesture-based interface (using Microsoft Kinect) was compared to using the mouse. The second experiment studied localization of internal structures, comparing slice-based visualizations via gestures and the mouse, in addition to a 3D Magic Lens visualization. The results of the user studies showed that the gesture-based interface outperform the traditional mouse both in time and accuracy in the orientation matching task. The traditional mouse was the superior interface for the second experiment in terms of accuracy. However, the gesture-based Magic Lens interface was found to have the fastest target localization time. We discuss these findings and their further implications in the use of gesture-based interfaces in medical volume visualization, and discuss the possible underlying psychological mechanisms why these methods can outperform traditional interaction methods

Adaptive mixture-of-experts models for data glove interface with multiple users

Hand gestures have great potential to act as a computer interface in the entertainment environment. However, there are two major problems when implementing the hand gesture-based interface for multiple users, the complexity problem and the personalization problem. In order to solve these problems and implement multi-user data glove interface successfully, we propose an adaptive mixture-of-experts model for data-glove based hand gesture recognition models which can solve both the problems. The proposed model consists of the mixture-of-experts used to recognize the gestures of an individual user, and a teacher network trained with the gesture data from multiple users. The mixture-of-experts model is trained with an expectation–maximization (EM) algorithm and an on-line learning rule. The model parameters are adjusted based on the feedback received from the real-time recognition of the teacher network. The model is applied to a musical performance game with the data glove (5DT Inc.) as a practical example. Comparison experiments using several representative classifiers showed both outstanding performance and adaptability of the proposed method. Usability assessment completed by the users while playing the musical performance game revealed the usefulness of the data glove interface system with the proposed method.

미국 방위산업체 상황실의 인물검색 활동을 돕는 가상현실 공간 인터페이스 환경에 관한 연구

본 연구는 빠른 속도로 발전하는 멀티미디어 정보 환경 중에서도 미국 방위산업체 정보기관의 감시 카메라 속 특정 인물을 색출하는 작업 상황을 가정으로, 사용자가 방대한 영상 데이터 기록물들 가운데 원하는 정보를 한 눈에 분석하고 검색해 낼 수 있는 효율적이고 만족스러운 멀티미디어 검색 환경에 대해 연구하였다. 이를 위해 첫 번째 실행된 실험에서 CAVE형 가상현실 공간 인터페이스를 검색환경으로 제안하고 데스크톱과 CAVE형 공간 인터페이스를 비교 평가하였다. 이 실험에 사용한 콘텐츠는 각 과제에 최적화되어 개발된 소프트웨어들과 데이터베이스를 사용하였다. 두 번째 실험은 CAVE형 가상현실 공간 인터페이스에서 검색의 효율을 높여주는 입력시스템에 대한 연구를 실행하였다. 특히, 인간의 움직임에 보다 가깝게 디자인되어 자연스럽게 조작할 수 있도록 설계된 제스처 입력시스템을 중심으로 CAVE형 가상현실 인터페이스에 적합한 3가지 입력시스템을 사용하여 검색 후 각 과제들에 대한 선호도와 만족도를 측정하였다. 또한 각 과제별 검색 수행 시간을 측정하여 효율성 높은 입력시스템을 분석하였다.

디지털 미디어 콘텐츠 개발을 위한 교육용 플랫폼의 활용성

기술의 발전과 함께 매해 쏟아져 나오는 새로운 플랫폼들은 콘텐츠 생산자들에게 창의적이고 혁신적인 콘텐츠를 개발할 수 있는 새로운 기회를 제공하고 있다. 이러한 현상은 디지털 미디어 콘텐츠 분야를 전공하는 학생들에게도 중요한 영향을 미쳐 콘텐츠 개발 시 새로운 기술 기반의 플랫폼을 활용해 봄으로써 그들에게 보다 새롭고 유익한 배움의 기회를 제공한다. 본 논문에서는 최근 디지털 미디어분야 학생들에게 주목받고 있는 주요기술로서 가상현실 디스플레이, 동작인식부, 물리엔진 그리고 체감형 인터페이스 기술을 제시하고, 이 4가지 기술을 사용하는 통합 플랫폼을 개발하여 콘텐츠 개발 시 보다 쉽고 편리하게 구현할 수 있도록 설계하였다. 또한 디지털 콘텐츠 개발 교육을 위한 목적으로서 플랫폼의 실용성을 검토하고자 제안한 플랫폼을 기반으로 4개의 프로토타입 콘텐츠를 개발하고 플랫폼의 활용성을 평가하였다. 사용자 평가결과 시스템의 기능성, 사용성, 유용성, 만족도 및 선호도 모두 전반적으로 높게 나타났으며 이 같은 플랫폼의 활용도는 학생들에게 쉽고 빠르게 프로토타입 콘텐츠(Rapid prototype)를 제작 할 수 있는 환경을 조성해 주어 창의적인 미디어 콘텐츠 연구에 있어 긍정적인 영향을 줄 것으로 사료된다. The advent of new platforms each year along with the advancement of technology provides a new opportunity for digital media designers to develop creative and innovative contents. This phenomenon affect the same way the students that major in the digital media, and the use of the platforms that is based on the new technology in the development of contents gives a newer and useful opportunity for learning to the students who recently study the digital media area. As the main technology of the recent digital media that attract many students' attention, we are presenting virtual reality display, movement cognition, physical engine and the gesture interface, and developed the consolidated platform based on these four technologies, and designed them in a way that can be more easily implemented in a simpler way. In order to study the efficiency of the platform with the objective of the development of digital media contents, we have developed four different prototype contents, and have measured based on the user's preference, efficiency and satisfaction. In the results of usability evaluation, functionality, effectiveness, efficiency, satisfaction were rated as 'high'. This results shows that the suggested 3D platform environment provides students to develop a rapid prototype fast and easy, and this may have a positive influence on students major in the digital media to conduct creative development research.

I'm home: Defining and evaluating a gesture set for smart-home control

Mobile phones seem to present the perfect user interface for interacting with smart environments, e.g. smart-home systems, as they are nowadays ubiquitous and equipped with an increasing amount of sensors and interface components, such as multi-touch screens. After giving an overview on related work this paper presents the adapted design methodology proposed by Wobbrock et al. (2009) for the development of a gesture-based user interface to a smart-home system. The findings for the new domain, device and gesture space are presented and compared to findings by Wobbrock et al. (2009). Three additional steps are described: A small pre-test survey, a mapping and a memory test and a performance test of the implemented system. This paper shows the adaptability of the approach described by Wobbrock et al. (2009) for three-dimensional gestures in the smart-home domain. Elicited gestures are described and a first implementation of a user interface based on these gestures is presented.

Expanding kinds of gestures for hierarchical menu selection by unicursal gesture interface

We propose Unicursal Gesture Interface (UGI) which is a thumb gesture to expand the number of gestures possible from a TV remote with a touchpad. UGI pairs a shape with orientation to create a command. The shapes are unicursal figures whose paths start and end at the same position and that are easy for the user's thumb to remember. UGI allow user to access different layers of hierarchical menu and to select network content comfortably. As an example of Electronic Program Guide access, we evaluate the performance four kinds of UGI. The mean recognition rate of the UGI is 96.1%.

A Development of Gesture Interfaces using Spatial Context Information

Gestures have been employed for human computer interaction to build more natural interface in new computational environments. In this paper, we describe our approach to develop a gesture interface using spatial context information. The proposed gesture interface recognizes a system action (e.g. commands) by integrating gesture information with spatial context information within a probabilistic framework. Two ontologies of spatial contexts are introduced based on the spatial information of gestures: gesture volume and gesture target. Prototype applications are developed using a smart environment scenario that a user can interact with digital information embedded to physical objects using gestures.

2A1-B10 3-DOFパラレルメカニズムを有する力覚提示装置の開発(人間機械協調)

The purpose of this study is to develop and evaluate the haptic device, "Parallel Glove" which displays reaction force caused by contact of robots with work environment to a human operator's hand. The Parallel Glove is installed around a human wrist and employed for gesture interface systems. The proposed haptic device is designed to implement a 3-DOF parallel mechanism. The perceptive force of a human operator is estimated from force sensor signals of the Parallel Glove and the surface EMG signals. To evaluate the proposed control system, experiments of force display for human operators were carried out. Experimental results show that the motion of the Parallel Glove is controlled to keep the magnitude of perceptive force of a human operator with that of displayed force of the device.

A Vision based Hand Gesture Interface for Controlling VLC Media Player

Human Computer Interaction can acquire several advantages with the introduction of different natural forms of device free communication. Gestures are a natural form of actions which we often use in our daily life for interaction, therefore to use it as a communication medium with computers generates a new paradigm of interaction with computers. This paper implements computer vision and gesture recognition techniques and develops a vision based low cost input device for controlling the VLC player through gestures. VLC application consists of a central computational module which uses the Principal Component Analysis for gesture images and finds the feature vectors of the gesture and save it into a XML file. The Recognition of the gesture is done by K Nearest Neighbour algorithm. The theoretical analysis of the approach shows how to do recognition in static background. The Training Images are made by cropping the hand gesture from static background by detecting the hand motion using Lucas Kanade Pyramidical Optical Flow algorithm. This hand gesture recognition technique will not only replace the use of mouse to control the VLC player but also provide different gesture vocabulary which will be useful in controlling the application.

Development of Deskwork Support System Using Pointing Gesture Interface

The authors have proposed a deskwork support system called an “Attentive Workbench” (AWB), which uses a camera, a projector, and automatically moving trays to support the user both physically and informationally. This research intends to build an interface between the user and the AWB system based on pointing gestures. Considering the specific purposes of the AWB, a simple, reliable, and highly responsive interface is implemented. We demonstrate a deskwork support system with actual automatically moving trays that uses the proposed interface.

Research on On-Line Two-Way Adaptive Pen Gesture Interface Framework

This paper presents a framework for on-line two-way adaptive pen gestures based user interface.In order to solve the problem that the traditional stationary pen gestures recognition engine can't support users' personified gestures input habits and the problem that users maybe can't remember some gestures when using them,the authors propose an on-line two-way adaptive strategy,on the one hand,system can adapt to different users according to their different input habits(system supports on-line personified pen gestures input habits through mediate interface),on the other hand,users can learn the interpretation capability of the system(users can learn some system gestures through mediate interface or on-line pen gestures query and help system).The framework consists of five parts:(1) on-line two-way adaptive pen gestures input interpretation model;(2) on-line two-way adaptive pen gestures input interpretation flowchart;(3) contextual priority definitions;(4) context-based mediate interface;(5) on-line context-based pen gestures query and help system.The authors implement a prototype under the guidance of the framework;in such prototype users can input gestures to manipulate photos.A controlled comparative quantitative experiment based on the prototype is carried out.The result of the experiment validates the advantages of the framework in usability.