Mid-air Interaction Research Articles

Effects of Switchable DOF for Mid-Air Manipulation in Immersive Virtual Environments

ABSTRACTIn the context of mid-air manipulation, this paper presents the effects of allowing the user to dynamically switch between 1DOF, 2DOF, and 3DOF operations. Such “manipulation with switchable DOF” has been widely used in commercial graphics packages and is increasingly being adopted for virtual reality editing, where the 3D scene is constructed through mid-air interaction in immersive virtual environments. However, its effectiveness and advantages/disadvantages have not been investigated. This paper compares “manipulation with switchable DOF” with “manipulations with DOF separation,” which allows only 1DOF operations, and “manipulation without DOF separation,” which provides 3DOF operations. Using translation, rotation, and scaling, three methods were evaluated in terms of completion time and precision. The experiment results showed that “manipulation with switchable DOF” outperformed “manipulation with DOF separation” in terms of completion time whereas three methods were comparable in terms of precision. “Manipulation with switchable DOF” was further analyzed, and the results showed that the more 3DOF operations led to the shorter completion time and the more 1DOF operations led to the higher precision.

Mid-air interaction with optical tracking for 3D modeling

Compared to common 2D interaction done with mouse and other 2D tracking devices, 3D hand tracking with low-cost optical cameras can provide more degrees of freedom, as well as natural gestures, when shape modeling is done in virtual spaces. However, though quite precise, the optical tracking devices cannot avoid problems intrinsic to hand interaction, such as hand tremor and jump release, and they also introduce an additional problem of hand occlusion. We investigate how to minimize the negative impact of these problems, and eventually propose to use hands in a way similar to how it is done when playing the Theremin – an electronic musical instrument controlled without physical contact by hands of the performer. We suggest that the dominant hand controls manipulation and deformation of objects while the non-dominant hand controls grasping, releasing and precision of interaction. Based on this method, we describe a generic set of reliable and precise interaction gestures for various manipulation and deformation tasks. We then prove with the user study that for the tasks involving 3D manipulations and deformations, hand interaction is faster than common 2D interaction done with mouse.

Investigating large curved interaction devices

Large interactive surfaces enable novel forms of interaction for their users, particularly in terms of collaborative interaction. During longer interactions, the ergonomic factors of interaction systems have to be taken into consideration. Using the full interaction space may require considerable motion of the arms and upper body over a prolonged period of time, potentially causing fatigue. In this work, we present Curved, a large-surface interaction device, whose shape is designed based on the natural movement of an outstretched arm. It is able to track one or two hands above or on its surface by using 32 capacitive proximity sensors. Supporting both touch and mid-air interaction can enable more versatile modes of use. We use image processing methods for tracking the user’s hands and classify gestures based on their motion. Virtual reality is a potential use case for such interaction systems and was chosen for our demonstration application. We conducted a study with ten users to test the gesture tracking performance, as well as user experience and user preference for the adjustable system parameters.

Target Selection Technique using Mid-air Interaction for Cardboard VR Environment

Target Selection Technique using Mid-air Interaction for Cardboard VR Environment

Exploring a user-defined gesture vocabulary for descriptive mid-air interactions

Gesturing provides an alternative interaction input for design that is more natural and intuitive. However, standard input devices do not completely reflect natural hand motions in design. A key challenge lies in how gesturing can contribute to human–computer interaction, as well as understanding the patterns in gestures. This paper aims to analyze human gestures to define a gesture vocabulary for descriptive mid-air interactions in a virtual reality environment. We conducted experiments with twenty participants describing two chairs (simple and abstract) with different levels of complexity. This paper presents a detailed analysis of gesture distribution and hand preferences for each description task. Comparisons are drawn between the proposed approach to the definition of a vocabulary using combined gestures (GestAlt) and previously suggested methods. The findings state that GestAlt is successful in describing the employed gestures in both tasks (60% of all gestures for simple chair and 69% for abstract chair). The findings can be applied to the development of an intuitive mid-air interface using gesture recognition.

Stable Haptic Feedback Generation during Mid air Interactions using Hidden Markov Model based Motion Synthesis

Generation of stable and realistic haptic feedback during mid-air gesture interactions has garnered significant research interests recently. But limitations of sensing technologies such as unstable tracking, non uniform sampling duration , occlusions during interactions etc distort motion based haptic feedback. In this paper, we propose an improved motion synthesis model which tracks human gestures during interaction and recreates smooth and synchronized motion data from detected Hidden Markov Model (HMM) states. The proposed model uses ideal motion data of humans and duration of HMM states recognised during gestures to modulate the realtime motion synthesis to synchronize with actual human motion speed. The synthesized and raw motion patterns were compared with ideal motion curve obtained from a benchmark motion capture system to verify the effectiveness of the proposed method. Subjective evaluation of virtual reality (VR) system based on synthesized motion data showed significant improvements in user satisfaction over raw motion based VR systems.

Prototyping Capacitive Sensing Applications with OpenCapSense

OpenCapSense is a prototyping platform to develop innovative applications that rely on perceiving humans with electric fields. Despite today's use of capacitive sensing mostly as a method to detect touch, it offers many interesting facets that range from mid-air interaction to contactless indoor localization and identification. The platform provides active sensors to detect human interactions at distances of more than 40 cm, by generating electric fields. Passive sensors allow for measuring changes in electric fields that occur naturally in the environment, enabling detection distances up to 2 m.

Extended LazyNav: Virtual 3D Ground Navigation for Large Displays and Head-Mounted Displays.

This paper presents the extended work on LazyNav, a head-free, eyes-free and hands-free mid-air ground navigation control model presented at the IEEE 3D User Interfaces (3DUI) 2015, in particular with a new application to the head-mounted display (HMD). Our mid-air interaction metaphor makes use of only a single pair of the remaining tracked body elements to tailor the navigation. Therefore, the user can navigate in the scene while still being able to perform other interactions with her hands and head, e.g., carrying a bag, grasping a cup of coffee, or observing the content by moving her eyes and locally rotating her head. We design several body motions for navigation by considering the use of non-critical body parts and develop assumptions about ground navigation techniques. Through the user studies, we investigate the motions that are easy to discover, easy to control, socially acceptable, accurate and not tiring. Finally, we evaluate the desired ground navigation features with a prototype application in both a large display (LD) and a HMD navigation scenarios. We highlight several recommendations for designing a particular mid-air ground navigation technique for a LD and a HMD.

CUDA-based real-time hand gesture interaction and visualization for CT volume dataset using leap motion

Touchless interaction has received considerable attention in recent years with benefit of removing barriers of physical contact. Several approaches are available to achieve mid-air interactions. However, most of these techniques cause discomfort when the interaction method is not direct manipulation. In this paper, gestures based on unimanual and bimanual interactions with different tools for exploring CT volume dataset are designed to perform the similar tasks in realistic applications. Focus + context approach based on GPU volume ray casting by trapezoid-shaped transfer function is used for visualization and the level-of-detail technique is adopted for accelerating interactive rendering. Comparing the effectiveness and intuitiveness of interaction approach with others by experiments, ours has a better performance and superiority with less completion time. Moreover, the bimanual interaction with more advantages is timesaving when performing continuous exploration task.

A gesture-free geometric approach for mid-air expression of design intent in 3D virtual pottery

The advent of depth cameras has enabled mid-air interactions for shape modeling with bare hands. Typically, these interactions employ a finite set of pre-defined hand gestures to allow users to specify modeling operations in virtual space. However, human interactions in real world shaping processes (such as pottery or sculpting) are complex, iterative, and continuous. In this paper, we show that the expression of user intent in shaping processes can be derived from the geometry of contact between the hand and the manipulated object. Specifically, we describe the design and evaluation of a geometric interaction technique for bare-hand mid-air virtual pottery. We model the shaping of a pot as a gradual and progressive convergence of the pot’s profile to the shape of the user’s hand represented as a point-cloud (PCL). Thus, a user does not need to learn, know, or remember any gestures to interact with our system. Our choice of pottery simplifies the geometric representation, allowing us to systematically study how users use their hands and fingers to express the intent of deformation during a shaping process. Our evaluations demonstrate that it is possible to enable users to express their intent for shape deformation without the need for a fixed set of gestures for clutching and deforming a shape.

Design Choices and Their Implications for 3D Mid-Air Manipulation Techniques

Manipulation is one of the most important tasks required in virtual environments and thus it has been thoroughly studied for widespread input devices such as mice or multi-touch screens. Nowadays, the Kinect sensor has turned mid-air interaction into another affordable and popular way of interacting. Mid-air interaction enables the possibility of interacting remotely without any physical contact and in a more natural manner. Nonetheless, although some scattered manipulation techniques have been proposed for mid-air interaction, there is a lack of evaluations and comparisons that hinders the selection and development of these techniques. To solve this issue, we gathered four design choices that can be used to classify mid-air manipulation techniques. Namely, choices are based on the required number of hands, separation of translation–rotation, decomposition of rotation, and interaction metaphors. Furthermore, we developed, adapted, and compared three manipulation techniques selected for studying the implications of the design choices. These implications are useful to select among already existing techniques as well as to inform technique developers.