Gesture Interface Research Articles

Evaluation of spatial interaction techniques for virtual heritage applications: A case study of an interactive holographic projection

The increasing use of information and communication technologies (ICT) in museums is providing curators with new opportunities for the display of cultural heritage content, making it possible to merge real and digital works of art in a coherent exhibition space. However, humans learn and perceive by following an interactive process, a fact that is particularly true in relation to the understanding, analysis and interpretation of the cultural heritage. In order to allow visitors to fully exploit the potential of this new hybrid cultural communication, interactivity is essential. This paper analyzes interaction design focusing on a holographic projection system equipped with a gesture-based interface and discussing the results of both quantitative and qualitative user studies aimed at empirically investigating users’ preferences in relation to interaction techniques when used in a museum context. The experimental findings suggest the adoption of task-specific patterns in the design of touchless user interfaces for the exploration of digital heritage content.

Gesture-Based Control of Scanning Electron Microscopes Using Leap Sensors

This work forms part of our ongoing research into enhancing and improving SEM technologies, and was one of two projects initiated to explore the potential of gesture-based interface control. This project focused on gesture-based manipulation of the SEM.

Gesture-Based Control of ImageJ Using Leap Sensors

This work forms part of our ongoing research into enhancing and improving microscopy and image processing technologies, and was one of two projects initiated to explore the potential of gesture-based interface control. This project focused on gesture-based manipulation of the ImageJ package, whose functionality can be extended through macros and plugins. In this case, additional plugins were developed to demonstrate the feasibility of interfacing ImageJ with a Leap Motion device.

A Study on Gesture Interface through User Experience

A Study on Gesture Interface through User Experience

A Study on Gesture Interface through User Experience

A Study on Gesture Interface through User Experience

Hand-Based Gesture Recognition for Vehicular Applications Using IR-UWB Radar.

Modern cars continue to offer more and more functionalities due to which they need a growing number of commands. As the driver tries to monitor the road and the graphic user interface simultaneously, his/her overall efficiency is reduced. In order to reduce the visual attention necessary for monitoring, a gesture-based user interface is very important. In this paper, gesture recognition for a vehicle through impulse radio ultra-wideband (IR-UWB) radar is discussed. The gestures can be used to control different electronic devices inside a vehicle. The gestures are based on human hand and finger motion. We have implemented a real-time version using only one radar sensor. Studies on gesture recognition using IR-UWB radar have rarely been carried out, and some studies are merely simple methods using the magnitude of the reflected signal or those whose performance deteriorates largely due to changes in distance or direction. In this study, we propose a new hand-based gesture recognition algorithm that works robustly against changes in distance or direction while responding only to defined gestures by ignoring meaningless motions. We used three independent features, i.e., variance of the probability density function (pdf) of the magnitude histogram, time of arrival (TOA) variation and the frequency of the reflected signal, to classify the gestures. A data fitting method is included to differentiate between gesture signals and unintended hand or body motions. We have used the clustering technique for the classification of the gestures. Moreover, the distance information is used as an additional input parameter to the clustering algorithm, such that the recognition technique will not be vulnerable to distance change. The hand-based gesture recognition proposed in this paper would be a key technology of future automobile user interfaces.

Aarya – A Kinesthetic companion for children with Autism Spectrum Disorder

Autism Spectrum Disorder (ASD) is defined as a condition or disorder that begins in childhood and that causes problems in establishing relationships and communicating with other people. Aarya works as a personal well-being companion to children with Autism Spectrum Disorder while they interact with a virtual environment that is gesture based. By making an ASD affected child face real world situations, we try to improve his/her confidence in facing the world and being open to learning various skills. Social interaction and communication are the major challenges faced by children with ASD. In Aarya, we use gesture-based interface that is the Microsoft Kinect so that the child can find it easier to interact in the real world environment. Through the interactions made with the children and the results obtained, we understand that this tool can be a companion while giving chance for growth and improving their interacting ability. With further refinement and expert inputs, this tool can be built better.

On the Efficiency of a VR Hand Gesture-Based Interface for 3D Object Manipulations in Conceptual Design

ABSTRACTIn the early stages of 3D design, sketches are used to quickly conceptualize ideas and gain insight into problems and possible solutions. Computer-aided design tools are widely used for 3D modeling and design, but their required precision and 2D mouse and screen-based interface inhibit the flow of ideas. A study was conducted to explore the efficiency of hand tracking and virtual reality (VR) for 3D object manipulations in conceptual design. Based on existing research on conceptual design and hand gestures, an intuitive hand-based interaction model is proposed. An experiment on basic 3D manipulation shows that participants using a simple VR and hand-tracking interface prototype have similar performance to those using a traditional mouse and screen interface. For the improvement of gestural conceptual design interfaces, the relevant issues are identified.

Smart-Pockets: Body-deictic gestures for fast access to personal data during ambient interactions

This work introduces Smart-Pockets, a new set of whole-body gesture recognition techniques that enables users to access their personal digital content efficiently for visualization on ambient displays. Smart-Pockets works by recognizing users' body-deictic gestures entailing access to their pockets, for which associations between specific pockets and personal digital content anchored to those pockets has been managed a priori. The pocket metaphor that we explore in this work enables links to digital content using physical personal containers (i.e., pockets) placed at convenient locations on the user's body, containers that have been specifically devised over decades of fashion design to store and carry people's personal belongings comfortably and conveniently. Consequently, Smart-Pockets gestures are fast, require absolutely no precision to perform effectively, and are robustly recognized in user-independent scenarios with absolutely no training required from the user of the ambient display. Also, the Smart-Pockets technique is flexible and easily extensible to other physical containers, such as bags and hand-held objects, which we demonstrate in the form of Smart-Containers. We evaluate the accuracy of several techniques for recognizing Smart-Pockets access gestures, for which we report +99% accuracy for user-independent classification and explicit segmentation. We discuss users' kinematic performance with Smart-Pockets and Smart-Containers and show that the average pocket access time of 2.2s is comparable to the average production time of touch gestures on smart mobile devices and is much smaller than the time required to produce other whole-body gestures. Beyond their practical implications for advancing knowledge in gesture-based interface design for ambient interactions, we believe that the contributions introduced by the Smart-Pockets concept will also foster new developments by pointing the community attention toward (i) more examination of the potential of a new class of whole-body gestures, i.e., body-deictics, (ii) more attention toward how users access their personal digital content on public displays, an important preliminary step before actual interaction, and (iii) inspiring work in the community to examine new and creative associations between users' physical personal objects and their digital content visualized on ambient displays. HighlightsSmart-Pockets is a new technique to access personal digital content on ambient displays.The pocket metaphor creates the link between physical objects and personal digital content.Smart-Pockets employ a new type of hybrid deictic and body-deictic gestures.Smart-Pockets actions are fast to perform (2.2s) and robustly recognized (+99%).The Smart-Pockets concept is flexible and extendable to Smart-Containers.

Comparison of gesture and conventional interaction techniques for interventional neuroradiology.

Interaction with radiological image data and volume renderings within a sterile environment is a challenging task. Clinically established methods such as joystick control and task delegation can be time-consuming and error-prone and interrupt the workflow. New touchless input modalities may have the potential to overcome these limitations, but their value compared to established methods is unclear. We present a comparative evaluation to analyze the value of two gesture input modalities (Myo Gesture Control Armband and Leap Motion Controller) versus two clinically established methods (task delegation and joystick control). A user study was conducted with ten experienced radiologists by simulating a diagnostic neuroradiological vascular treatment with two frequently used interaction tasks in an experimental operating room. The input modalities were assessed using task completion time, perceived task difficulty, and subjective workload. Overall, the clinically established method of task delegation performed best under the study conditions. In general, gesture control failed to exceed the clinical input approach. However, the Myo Gesture Control Armband showed a potential for simple image selection task. Novel input modalities have the potential to take over single tasks more efficiently than clinically established methods. The results of our user study show the relevance of task characteristics such as task complexity on performance with specific input modalities. Accordingly, future work should consider task characteristics to provide a useful gesture interface for a specific use case instead of an all-in-one solution.

A Comparative Study of Map Exploration Interfaces for Multi-Touch Tabletops

ABSTRACTWith the advent of multi-touch tabletops, software developers are creating innovative interfaces for applications such as interactive map systems. In this article, we summarized three representative interfaces from prevalent electronic map applications and literatures—gesture based, widget based, and hybrid, which supported both gestures and widgets—and then compared the effectiveness and the efficiency of these three interfaces by experiments. The results show that the hybrid interface had a better performance, requiring less task-completion time and operations, and causing less arm fatigue. The widget-based interface had the least distance traversed by fingers, but it required many tapping operations. The gesture-based interface was easier to learn compared to the widget-based interface, but the distance traversed during finger movements was the longest. Based on the findings from our experiments, we discuss the implications for the design of map exploration interfaces on multi-touch tabletops.

ジェスチャーインターフェースのための手の位置計測の研究

As a method of inputting digital electronic equipment, there is a gesture interface. It is important that the gesture interface can be measured without being influenced by changes in the usage environment. As one of hand measurement methods, there is The Distance Image Sensor. Therefore, we evaluate the measurement accuracy of the hand by The Distance Image Sensor. For that purpose, the measured value of Motion Capture is taken as a true value, the error of The Distance Image Sensor is calculated, and the error is evaluated. As a result, we confirmed the effectiveness of the hand measurement accuracy by The Distance Image Sensor. We conclude that The Distance Image Sensor can be used for gesture interface.

Variability Analysis on Gestures for People With Quadriplegia.

Gesture-based interfaces have become an effective control modality within the human computer interaction realm to assist individuals with mobility impairments in accessing technologies for daily living to entertainment. Recent studies have shown that gesture-based interfaces in tandem with gaming consoles are being used to complement physical therapies at rehabilitation hospitals and in their homes. Because the motor movements of individuals with physical impairments are different from persons without disabilities, the gesture sets required to operate those interfaces must be customized. This limits significantly the number and quality of available software environments for users with motor impairments. Previous work presented an analytic approach to convert an existing gesture-based interface designed for individuals without disabilities to be usable by people with motor disabilities. The objective of this paper is to include gesture variability analysis into the existing framework using robotics as an additional validation framework. Based on this, a physical metric (referred as work) was empirically obtained to compare the physical effort of each gesture. An integration method was presented to determine the accessible gesture set based on stability and empirical robot execution. For all the gesture types, the accessible gestures were found to lie within 34% of the optimality of stability and work. Lastly, the gesture set determined by the proposed methodology was practically evaluated by target users in experiments while solving a spatial navigational problem.

Advanced information processing of MEMS motion sensors for gesture interaction

Abstract. Sensor-based gesture interaction technology has been widely adopted in consumer electronics. Nevertheless, bias, drift, and noise existing in sensor signals are difficult to eliminate, and accurate movement trajectory information is still needed to achieve flexible interaction application. This paper presents micro-electro-mechanical system (MEMS) motion sensor information processing algorithms designed on a gesture interaction system which integrates multiple low-cost MEMS motion sensors with ZigBee wireless technology to support embodied communication while acting together with machines. Sensor signal processing systems mainly solve noise removal, signal smoothing, gravity influence separation, coordinate system conversion, and position information retrieval. The attitude information which is an important movement parameter and required by position estimation is calculated with a quaternion-based extended Kalman filter (EKF). The effectiveness of the movement information retrieval of this gesture interface is verified by experiments and test analysis, both in static and moving cases. In the end, related applications of the described sensor information processing are discussed.

Head-mounted gesture controlled interface for human-computer interaction

This paper proposes a novel human-computer interaction system exploiting gesture recognition. It is based on the combined usage of an head-mounted display and a multi-modal sensor setup including also a depth camera. The depth information is used both to seamlessly include augmented reality elements into the real world and as input for a novel gesture-based interface. Reliable gesture recognition is obtained through a real-time algorithm exploiting novel feature descriptors arranged in a multi-dimensional structure fed to an SVM classifier. The system has been tested with various augmented reality applications including an innovative human-computer interaction scheme where virtual windows can be arranged into the real world observed by the user.

Electro-Tactile Feedback System to Enhance Virtual Reality Experience

Virtual Reality (VR) has been proven to be capable of enabling its users to experience immersive virtual environments and interact with the computer. Most VR applications utilize stereo visual perception, however, stereo visual perception alone is not enough to fully immerse users in VR environments. Haptic feedback technology when applied to VR applications can offer improved immersion and interactivity and can extend the range of sensations experienced by the user. For example, tactile feedback can enable the user to feel an object they touch with their virtual hand or body. To provide tactile feedback in VR applications most researchers construct cumbersome mechanical actuators and linkages for applying feedback forces or vibrations to the user's real hand or body. Having to wear cumbersome feedback hardware can have the negative effect of making the user more aware of their physical body which can reduce the sense of being immersed in the VR world. To overcome this drawback we propose an alternative feedback system which is comprised of a hand gesture interface, a stereo visual head-set and an electro-tactile feedback system. Our electro-tactile feedback system is compact, simple, inexpensive and capable of delivering a variety of feedback sensations to the user. Experimental results are provided which demonstrate how the electro-tactile feedback system can enhance both the sense of immersion and interactivity for VR users.

Breakthroughs and issues in the implementation of Six Sense Technology: A review

During the attention at a new object our senses try to sense and analyze that particular object and make us interact with them. But to know the complete information about a particular object we should go through it in detail by surfing on net or asking the relevant person for information. Instead, sixth sense technology can be used to immediately trace out the whole information about an object as soon as we see it. Sixth Sense technology is a revolutionary way to interface the physical world with digital information. Sixth sense is a wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information. This paper focuses on and makes us aware with the sixth sense technology which provides an integration of the digital world with the physical world; it helps us understand how the sixth sense device had overpowered the five natural senses

Project anywhere: An interface for virtual architecture

Virtual and augmented realities open a new world of great potential for spatial research and experimentation by allowing new forms of unbuilt sensible architectural space. This article starts with a sketch of the current context in virtual reality and continues by outlining the development and structure of the research ‘project Anywhere’. The project is an easily deployable, wireless, multi-user, augmented reality app system that offers full body immersion through body, head and hands tracking. It can host multiple concurrent users, able to move freely in the virtual space, by moving in the real and also perform actions through a gesture interface to affect their shared environment. In conclusion, we describe the inherent properties of such a space, which we propose as a novel spatio-temporal medium for architecture that suggests an enriched notion of space for exploration and experimentation, through an example of a potential application.

Infants, Interfaces, and Intermediation: Digital Parenting and the Production of “iPad Baby” Videos on YouTube

We investigate the ways young children’s use of mobile touchscreen interfaces is both understood and shaped by parents through the production of YouTube videos and discussions in associated comment threads. This analysis expands on, and departs from, theories of parental mediation, which have traditionally been framed through a media effects approach in analyzing how parents regulate their children’s use of broadcast media, such as television, within family life. We move beyond the limitations of an effects framing through more culturally and materially oriented theoretical lenses of mediation, considering the role mobile interfaces now play in the lives of infants through analysis of the ways parents intermediate between domestic spaces and networked publics.We propose the concept of intermediation, which builds on insights from critical interface studies as well as cultural industries literature to help account for these expanded aspects of digital parenting. Here, parents are not simply moderating children’s media use within the home, but instead operating as an intermediary in contributing to online representations and discourses of children’s digital culture. This intermediary role of parents engages with ideological tensions in locating notions of “naturalness:” the iPad’s gestural interface or the child’s digital dexterity.

User intent perception by gesture and eye tracking

AbstractUser intent is a highly cohesive human activity which is difficult to encode and measure. It is critical to seize the target user’s psychological expectations, feelings, needs, and other elements of user’s true intentions in the area of product design. This paper proposes a task-based method to perceive user intent by capturing their “Hand movement” and “Eye movement” when target users are performing user tasks. In terms of “Hand movement”, user intents are obtained through gesture recognition based on computer vision, in particular, task-based gesture recognition solves the classic difficult problem “Midas Touch” in gesture interface of computer vision; in terms of “Eye movement”, user interests and concerns on product are captured by eye-tracking equipment. As the “Hand movement” and “Eye movement” represent the vast majority of user’s nonverbal expression of intents, this method is proved to have great practicality and pertinence by both theory and experiment.