Natural Interaction Techniques Research Articles

Exploring Data Input Problems in Mixed Reality Environments: Proposal and Evaluation of Natural Interaction Techniques

Data input within mixed reality environments poses significant interaction challenges, notably in immersive visual analytics applications. This study assesses five numerical input techniques: three benchmark methods (Touch-Slider, Keyboard, Pinch-Slider) and two innovative multimodal techniques (Bimanual Scaling, Gesture and Voice). An experimental design was employed to compare these techniques’ input efficiency, accuracy, and user experience across varying precision and distance conditions. The findings reveal that multimodal techniques surpass slider methods in input efficiency yet are comparable to keyboards; the voice method excels in reducing cognitive load but falls short in accuracy; and the scaling method marginally leads in user satisfaction but imposes a higher physical load. Furthermore, this study outlines these techniques’ pros and cons and offers design guidelines and future research directions.

Human–System Interaction Based on Eye Tracking for a Virtual Workshop

With the constant exploration and development of intelligent manufacturing, the concept of digital twins has been proposed and applied. In view of the complexity and intellectualization of virtual workshop systems, real workshops can link with virtual workshosp based on AR under the structure of digital twins, which allows users to interact with virtual information and perceive the virtual information superimposed on the real world with great immersion. However, the three-dimensionality of virtual workshops and interaction with complex workshop information can be challenging for users. Due to a shortage of input bandwidth and the nontraditional mode of interaction, a more natural interaction technique for virtual workshops is required. To solve such problems, this paper presents a technical framework for 3D eye movement interaction applied to a virtual workshop. An eye movement interaction technique, oriented to implicit interaction and explicit interaction, is developed by establishing behavior recognition and interaction intention understanding. An eye-movement experiment verifies the former’s accuracy is above 90% and had better recognition performance. A better feature vector group of the latter is selected to establish a model and verify its feasibility and effectiveness. Finally, the feasibility of the framework is verified through the development of an application example.

Exploration and Assessment of Interaction in an Immersive Analytics Module: A Software-Based Comparison

The focus of computer systems in the field of visual analytics is to make the results clear and understandable. However, enhancing human-computer interaction (HCI) in the field is less investigated. Data visualization and visual analytics (VA) are usually performed using traditional desktop settings and mouse interaction. These methods are based on the window, icon, menu, and pointer (WIMP) interface, which often results in information clutter and is difficult to analyze and understand, especially by novice users. Researchers believe that introducing adequate, natural interaction techniques to the field is necessary for building effective and enjoyable visual analytics systems. This work introduces a novel virtual reality (VR) module to perform basic visual analytics tasks and aims to explore new interaction techniques in the field. A pilot study was conducted to measure the time it takes students to perform basic tasks for analytics using the developed VR module and compares it to the time it takes them to perform the same tasks using a traditional desktop to assess the effectiveness of the VR module in enhancing student’s performance. The results show that novice users (Participants with less programming experience) took about 50% less time to complete tasks using the developed VR module as a comrade to a programming language, notably R. Experts (Participants with advanced programming experience) took about the same time to complete tasks under both conditions (R and VR).

Naturally Together: A Systematic Approach for Multi-User Interaction With Natural Interfaces

New technology is moving towards intuitive and natural interaction techniques that are increasingly embedded in human space (e.g., home and office environment) and aims to support multiple users, yet their interfaces do not cover it to the full. Imagine that you have a multi-user device, should it act differently in different situations, people, and group settings? Current Multi-User Interfaces address each of the users as an individual that works independently from others, and there is a lack of understanding of the mechanisms that impact shared usage of these products. Thus we have linked environmental (external) and user-centered (internal) factors to the way users interact with multi-user devices. We analyzed 124 papers that involve multi-user interfaces and created a classification model out of 8 factors. Both the model and factors were validated by a large-scale online study. Our model defines the factors affecting multi-user usage with a single device and leads to a decision on the most important ones in different situations. This paper is the first to identify these factors and to create a set of practical guidelines for designing Multi-User Interfaces.

Controlling Teleportation-Based Locomotion in Virtual Reality with Hand Gestures: A Comparative Evaluation of Two-Handed and One-Handed Techniques

Virtual Reality (VR) technology offers users the possibility to immerse and freely navigate through virtual worlds. An important component for achieving a high degree of immersion in VR is locomotion. Often discussed in the literature, a natural and effective way of controlling locomotion is still a general problem which needs to be solved. Recently, VR headset manufacturers have been integrating more sensors, allowing hand or eye tracking without any additional required equipment. This enables a wide range of application scenarios with natural freehand interaction techniques where no additional hardware is required. This paper focuses on techniques to control teleportation-based locomotion with hand gestures, where users are able to move around in VR using their hands only. With the help of a comprehensive study involving 21 participants, four different techniques are evaluated. The effectiveness and efficiency as well as user preferences of the presented techniques are determined. Two two-handed and two one-handed techniques are evaluated, revealing that it is possible to move comfortable and effectively through virtual worlds with a single hand only.

GG Interaction: a gaze\u2013grasp pose interaction for 3D virtual object selection

During the last two decades, development of 3D object selection techniques has been widely studied because it is critical for providing an interactive virtual environment to users. Previous techniques encounter difficulties with selecting small or distant objects, as well as naturalness and physical fatigue. Although eye-hand based interaction techniques have been promoted as the ideal solution to these problems, research on eye-hand based spatial interaction techniques in 3D virtual spaces has progressed very slowly. We propose a natural and efficient spatial interaction technique for object selection, which is motivated by understanding the human grasp. The proposed technique, gaze–grasp pose interaction (GG Interaction), has many advantages, such as quick and easy selection of small or distant objects, less physical fatigue, and elimination of eye-hand visibility mismatch. Additionally, even if an object is partially overlapped by other objects, GG Interaction enables a user to select the target object easily. We compare GG Interaction with a standard ray-casting technique through a formal user study (participants = 20) across two scenarios. The results of the study confirm that GG Interaction provides natural, quick and easy selection for users.

Handwriting Recognition in Free Space Using WIMU-Based Hand Motion Analysis

We present a wireless-inertial-measurement-unit- (WIMU-) based hand motion analysis technique for handwriting recognition in three-dimensional (3D) space. The proposed handwriting recognition system is not bounded by any limitations or constraints; users have the freedom and flexibility to write characters in free space. It uses hand motion analysis to segment hand motion data from a WIMU device that incorporates magnetic, angular rate, and gravity sensors (MARG) and a sensor fusion algorithm to automatically distinguish segments that represent handwriting from nonhandwriting data in continuous hand motion data. Dynamic time warping (DTW) recognition algorithm is used to recognize handwriting in real-time. We demonstrate that a user can freely write in air using an intuitive WIMU as an input and hand motion analysis device to recognize the handwriting in 3D space. The experimental results for recognizing handwriting in free space show that the proposed method is effective and efficient for other natural interaction techniques, such as in computer games and real-time hand gesture recognition applications.

Natural interaction for online documentation in industrial maintenance

This paper presents a novel application of natural interaction techniques for technical documentation navigation in augmented reality maintenance. The objective is to provide technical information where and when needed in order to reduce the time spent in retrieving procedural data from manuals. The system integrates a video and depth camera for user gesture acquisition, a webcam for marker-based tracking and a see through head mounted display for digital information overlay. We designed a novel hand free gesture-based interface to access technical documentation while performing maintenance tasks. We implemented a specific flexible system to prototype natural interfaces using UML configurable state machines. It supports 2D and 3D content navigation, CAD assembly explosion, real time 3D sectioning and operation browsing. We differentiate the interaction metaphors for 2D documents and 3D CAD models in order to be intuitive and easy to learn. A case study and a user test demonstrated the feasibility and the good acceptance of the presented gesture interface.

Work Analysis Methods in Practice

Human work interaction design is an emerging discipline that aims to encourage empirical studies and conceptualizations of the interaction among humans, their variegated social contexts and the technology they use both within and across these contexts. In this paper the authors present a virtual reality system for visualization, navigation and reviewing of 3D CAD models within the oil industry domain. This system combines a large-screen interaction environment with remote mobile devices, thus allowing engineers in the field and teams in a control center to work in collaboration. To navigate through models the system uses the mobile device’s camera and inertial sensors and takes advantage of recent natural interaction techniques on large-screen environments. The authors describe and elaborate around the usage of different work analysis methods in this complex, real world work domain. The analysis is based on (i) input from experts in the oil platform engineering field, (ii) previous and related work and (iii) application of different methods considering the recent advances in technology. The authors conclude that hierarchical task analysis was not effective in obtaining a clear, common vision about the work domain. Storyboarding was the most useful technique as it promoted the discovery of novelty factors that differentiate the solution, while simultaneously supporting the human work at offshore engineering design and review sessions.

Development of an Interface to a Spacecraft Simulator empowered by Virtual Reality

Due to its large territory, Brazil relies on space systems to perform a myriad of supporting activities. Space systems design requires strong modeling and simulations techniques for achieving high performance. This article describes the development of a goal-driven user interface (UI) for spacecraft distributed simulations using a service oriented paradigm and supporting different space missions. From usability problems with traditional simulators UIs, described by satellite subsystem design engineers, this work proposes three dimensional visualization, natural interaction techniques, virtual and augmented reality as well as interaction with touch-screen and gesture recognitions. This research uses an open source C/C++ toolkit, designed to provide interactivity, networking and scripting capabilities for simulator developers, this facilitates in providing specific input or output driver engines. Finally, some further developments are described for future work.

VOXAR Labs - augmenting experiences for a better life

The VOXAR Labs is a research group with the mission of developing people by augmenting experiences. It develops and transfers technology related to visualization, tracking and natural interaction techniques focusing augmented reality in multi-disciplinary application domains. The laboratory has several ongoing projects including an international cooperation, projects with the industry, involving energy and military ones, as well as research and academic projects. VOXAR Labs is part of the Informatics Center of the Federal University of Pernambuco, located in Recife – Pernambuco, Brazil.

3D Interaction Techniques Based on Semantics in Virtual Environments

Natural and efficient 3D interaction techniques play one of the key roles in the development of virtual reality systems. Most of these techniques, which are used currently, mainly aim at supporting the implementation of interaction tasks in geometric level. As a result, they usually lack of the sufficient ability for executing those interaction tasks that orient to high-level application. Based on the cognitive principles in the real world, interaction objects in virtual environments not only include some geometric attributes in the visual view, but also own some specific semantic attributes such as interaction rules, constraints and affordances, which are related with interaction process tightly. In this paper these objects are called semantic objects, in the sense that they know how the user can interact with them, giving clues to aid the interaction. Through parsing and interpreting interaction semantics, these semantic objects can help to realize high-level interaction metaphors above direct manipulation. Because they hide low-level details about the implementation of interaction techniques, this kind of metaphors evidently improve the efficiency and usability of interaction techniques, and enable user to concentrate more on the high-level interaction control directly related with the special applications.