3D Interaction Techniques Research Articles

How Interaction Techniques Affect Workload in a Virtual Environment During Multitasking

Abstract Virtual environments (VEs) can be modulated and adapted to the needs of each user, but in the case of patients, performance can be affected by several factors that are difficult to expertise. The objective of the study is to explore the relationship between workload and the interaction techniques used during selection in a virtual apartment. Fifty-six participants performed tasks in a VE with 2D or, more immersive, 3D interaction techniques. The VE used was the Virtual Multitasking Test (Banville et al., 2018) where participants realized several everyday tasks in a virtual apartment. Workload and variables describing how individuals felt in the VE were measured using questionnaires, and performance in VE has been assessed. Results showed that 2D selection techniques have a better usability than 3D ones. The performance (success in task realization) on the virtual tasks was not impacted by the interaction techniques. Our results suggest that the easier it is to use an interaction technique, the less workload is associated with it. Sense of presence and cybersickness were affected by 3D interaction techniques. Thus, future VEs for cognitive assessment and rehabilitation, based on instrumental activities of daily living, will have to adapt their interaction techniques to different users.

Designing a 3D gestural interface to support user interaction with time-oriented data as immersive 3D radar charts

The design of intuitive three-dimensional user interfaces is vital for interaction in virtual reality, allowing to effectively close the loop between a human user and the virtual environment. The utilization of 3D gestural input allows for useful hand interaction with virtual content by directly grasping visible objects, or through invisible gestural commands that are associated with corresponding features in the immersive 3D space. The design of such interfaces remains complex and challenging. In this article, we present a design approach for a three-dimensional user interface using 3D gestural input with the aim to facilitate user interaction within the context of Immersive Analytics. Based on a scenario of exploring time-oriented data in immersive virtual reality using 3D Radar Charts, we implemented a rich set of features that is closely aligned with relevant 3D interaction techniques, data analysis tasks, and aspects of hand posture comfort. We conducted an empirical evaluation (n=12)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$(n=12)$$\\end{document}, featuring a series of representative tasks to evaluate the developed user interface design prototype. The results, based on questionnaires, observations, and interviews, indicate good usability and an engaging user experience. We are able to reflect on the implemented hand-based grasping and gestural command techniques, identifying aspects for improvement in regard to hand detection and precision as well as emphasizing a prototype’s ability to infer user intent for better prevention of unintentional gestures.

Zoom‐fwd: Efficient technique for 3D gestual interaction with distant and occluded objects in virtual reality

AbstractIn the last decade, there has been an extraordinary acceleration in immersive technologies, including virtual reality (VR) and augmented reality (AR). The VR interfaces have widely explored various 3D interaction‐based approaches. 3D interaction is one of the main features of any VR system. However, the problems while selecting and manipulating distant and occluded 3D virtual objects in VR are still unresolved and suffer from a lack of precision and accuracy. To interact with VR/AR systems, various devices such as VR headsets or gloves are used. These devices are not very reliable yet; in fact they are inconvenient and invasive to a person's comfort. Currently, the natural user interfaces (NUIs) are increasingly introduced in human computer interaction (HCI) systems as gestures recognition, which prove to be very useful in the improvement of the user engagement and presence sensing. More particularly, they provide more user‐friendly and nonintrusive 3D interaction methods and techniques. In this article, the Zoom‐fwd, which is an efficient 3D interaction technique is presented. The proposed technique uses gesture recognition for different 3D interaction tasks like selection and manipulation. This new approach allows an efficient interaction with distant and occluded objects, while providing a precise selection, even when the environment is crowded. The Zoom‐fwd technique is a software solution for a number of hardware and software problems. A user study is conducted to determine whether the proposed technique is more suitable when performing interaction tasks. The results show that the Zoom‐fwd technique provides effective interaction with distant and occluded objects, by improving the user task completion performance. In addition, this indicates that selection precision has been enhanced significantly with the Zoom‐fwd technique.

The State of the Art of Spatial Interfaces for 3D Visualization

AbstractWe survey the state of the art of spatial interfaces for 3D visualization. Interaction techniques are crucial to data visualization processes and the visualization research community has been calling for more research on interaction for years. Yet, research papers focusing on interaction techniques, in particular for 3D visualization purposes, are not always published in visualization venues, sometimes making it challenging to synthesize the latest interaction and visualization results. We therefore introduce a taxonomy of interaction technique for 3D visualization. The taxonomy is organized along two axes: the primary source of input on the one hand and the visualization task they support on the other hand. Surveying the state of the art allows us to highlight specific challenges and missed opportunities for research in 3D visualization. In particular, we call for additional research in: (1) controlling 3D visualization widgets to help scientists better understand their data, (2) 3D interaction techniques for dissemination, which are under‐explored yet show great promise for helping museum and science centers in their mission to share recent knowledge, and (3) developing new measures that move beyond traditional time and errors metrics for evaluating visualizations that include spatial interaction.

3D interaction techniques for musical expression

As Virtual Reality headsets become accessible, more and more artistic applications are developed, including immersive musical instruments. 3D interaction techniques designed in the 3D User Interfaces research community, such as navigation, selection and manipulation techniques, open numerous opportunities for musical control. For example, navigation techniques such as teleportation, free walking/flying and path-planning enable different ways of accessing musical scores, scenes of spatialised sound sources or even parameter spaces. Manipulation techniques provide novel gestures and metaphors, e.g. for drawing or sculpting sound entities. Finally, 3D selection techniques facilitate the interaction with complex visual structures which can represent hierarchical temporal structures, audio graphs, scores or parameter spaces. However, existing devices and techniques were developed mainly with a focus on efficiency, i.e. minimising error rate and task completion times. They were therefore not designed with the specifics of musical interaction in mind. In this paper, we review existing 3D interaction techniques and examine how they can be used for musical control, including the possibilities they open for instrument designers. We then propose a number of research directions to adapt and extend 3DUIs for musical expression

A multi-GPU accelerated virtual-reality interaction simulation framework.

In this paper, we put forward a real-time multiple GPUs (multi-GPU) accelerated virtual-reality interaction simulation framework where the reconstructed objects from camera images interact with virtual deformable objects. Firstly, based on an extended voxel-based visual hull (VbVH) algorithm, we design an image-based 3D reconstruction platform for real objects. Then, an improved hybrid deformation model, which couples the geometry constrained fast lattice shape matching method (FLSM) and total Lagrangian explicit dynamics (TLED) algorithm, is proposed to achieve efficient and stable simulation of the virtual objects’ elastic deformations. Finally, one-way virtual-reality interactions including soft tissues’ virtual cutting with bleeding effects are successfully simulated. Moreover, with the purpose of significantly improving the computational efficiency of each time step, we propose an entire multi-GPU implementation method of the framework using compute unified device architecture (CUDA). The experiment results demonstrate that our multi-GPU accelerated virtual-reality interaction framework achieves real-time performance under the moderate calculation scale, which is a new effective 3D interaction technique for virtual reality applications.

3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review

Handheld mobile devices referring to mobile phones and tablets become the major output medium for augmented reality (AR), which have seen significant growth in popularity and usage among the public due to the growing release of consumer-oriented communication products nowadays, especially touchscreen smartphones. Unlike traditional desktop or tabletop AR (large display-based) and head-mounted display-based AR systems, small display-based AR (handheld mobile display) requires different interaction techniques that mostly utilize single-hand interaction as well as the limitation of small screen display and limited activity time due to the battery operation hour of handheld mobile devices. However, in handheld mobile AR, research is still lacking, especially research that focuses on 3D interaction in handheld mobile AR for virtual object manipulation. Thus, this paper provides an overview of 3D interaction techniques in handheld mobile AR with critical analysis. First, we describe three main interaction technique categories that applicable in handheld mobile AR, which is touch-based interaction, mid-air gestures-based interaction, and device-based interaction techniques, of their basic concepts on 3D object manipulation. Then, we classify and systematize the highlighted techniques and discuss the advantages and drawbacks of each. Previous studies for widely used techniques have been studied comprehensively. We then draw up a comparison among the different techniques based on the important elements considered in handheld mobile AR. The aim of this paper is to provide researchers with background information on AR and those who are interested in the field of 3D interaction, realizing each technique category.

ShapeGuide: Shape-Based 3D Interaction for Parameter Modification of Native CAD Data.

While Virtual Reality (VR) technologies are commonly used in industrial companies, loading and modifying CAD parts of commercial CAD systems in virtual environments are still challenging. A few VR applications for Computer Aided Design (CAD) enable users to modify native CAD data in an immersive environment. Even if such VR-CAD applications use 3D interaction space, interaction with parameter values of CAD parts could be enhanced. This paper presents ShapeGuide, a technique allowing users to modify native CAD parts using a shape-based 3D interaction technique. With ShapeGuide, users can achieve modification of parameter values by directly pushing or pulling the surface of a CAD object. In addition, force feedback can be integrated into the technique to enhance the precision of the users' hand motions in the 3D space. In a controlled experiment, we compared ShapeGuide to a standard one-dimensional scroll technique to measure its added value for parametric CAD data modification on a simple industrial product example with an adjusted modification capability. We also evaluated the effect of force feedback assistance on both techniques. Results of this experiment demonstrate that ShapeGuide is significantly faster and more efficient than the scroll technique. Furthermore, they show that the force feedback assistance enhances the precision of both techniques, especially of ShapeGuide.

Move the Object or Move Myself? Walking vs. Manipulation for the Examination of 3D Scientific Data

Physical walking is consistently considered a natural and intuitive way to acquire viewpoints in a virtual environment. However, research findings also show that walking requires cognitive resources. To understand how this tradeoff affects the interaction design for virtual environments; we evaluated the performance of 32 participants, ranging from 18 to 44 years old, in a demanding visual and spatial task. Participants wearing a virtual reality (VR) headset counted features in a complex 3D structure while walking or while using a 3D interaction technique for manipulation. Our results indicate that the relative performance of the interfaces depends on the spatial ability and game experience of the participants. Participants with previous game experience but low spatial ability performed better using the manipulation technique. However, walking enabled higher performance for participants with high spatial ability or without significant game experience. These findings suggest that the optimal design choices for demanding visual tasks in VR should consider both controller experience and the spatial ability of the target users.

Enabling Symmetric Collaboration in Public Spaces through 3D Mobile Interaction

Collaboration has been common in workplaces in various engineering settings and in our daily activities. However, how to effectively engage collaborators with collaborative tasks has long been an issue due to various situational and technical constraints. The research in this paper addresses the issue in a specific scenario, which is how to enable users to interact with public information from their own perspective. We describe a 3D mobile interaction technique that allows users to collaborate with other people by creating a symmetric and collaborative ambience. This in turn can increase their engagement with public displays. In order to better understand the benefits and limitations of this technique, we conducted a usability study with a total of 40 participants. The results indicate that the 3D mobile interaction technique promotes collaboration between users and also improves their engagement with the public displays.

Virtual Valcamonica: Collaborative Exploration of Prehistoric Petroglyphs and Their Surrounding Environment in Multi-User Virtual Reality

Abstract In this article, we present a novel, multi-user, virtual reality environment for the interactive, collaborative 3D analysis of large 3D scans and the technical advancements that were necessary to build it: a multi-view rendering system for large 3D point clouds, a suitable display infrastructure, and a suite of collaborative 3D interaction techniques. The cultural heritage site of Valcamonica in Italy with its large collection of prehistoric rock-art served as an exemplary use case for evaluation. The results show that our output-sensitive level-of-detail rendering system is capable of visualizing a 3D dataset with an aggregate size of more than 14 billion points at interactive frame rates. The system design in this exemplar application results from close exchange with a small group of potential users: archaeologists with expertise in rockart. The system allows them to explore the prehistoric art and its spatial context with highly realistic appearance. A set of dedicated interaction techniques was developed to facilitate collaborative visual analysis. A multi-display workspace supports the immediate comparison of geographically distributed artifacts. An expert review of the final demonstrator confirmed the potential for added value in rock-art research and the usability of our collaborative interaction techniques.

TEKNIK INTERAKSI UNIMANUAL PUSH AND PULL MENGGUNAKAN DATA GLOVE

In a 3D user interface, interaction plays an important role in helping users to manipulate 3D objects in virtual environments. 3D devices, such as data glove and motion tracking, can potentially give users the opportunity to manipulate 3D objects in virtual reality environments such as checking, zooming, translating, rotating, merging and splitting 3D objects in a more natural and easy manner through the use of hand gestures. Hand gestures are often applied in 3D interaction techniques for converting the manipulation mode. This paper will discuss the interaction technique in a virtual environment using a combination of the Push and Pull navigation and the rotation technique. The unimanual use of these 3D interaction techniques can improve the effectiveness of users in their interaction with and manipulation of 3D objects. This study has enhanced the capability of the unimanual 3D interaction technique in terms of 3D interaction feedback in virtual environments.

Bare-Hand Volume Cracker for Raw Volume Data Analysis

Analysis of raw volume data generated from different scanning technologies faces a variety of challenges, related to search, pattern recognition, spatial understanding, quantitative estimation, and shape description. In a previous study, we found that the Volume Cracker (VC) 3D interaction (3DI) technique mitigated some of these problems, but this result was from a tethered glove-based system with users analyzing simulated data. Here, we redesigned the VC by using untethered bare-hand interaction with real volume datasets, with a broader aim of adoption of this technique in research labs. We developed symmetric and asymmetric interfaces for the Bare-Hand Volume Cracker (BHVC) through design iterations with a biomechanics scientist. We evaluated our asymmetric BHVC technique against standard 2D and widely used 3D interaction techniques with experts analyzing scanned beetle datasets. We found that our BHVC design significantly outperformed the other two techniques. This study contributes a practical 3DI design for scientists, documents lessons learned while redesigning for bare-hand trackers, and provides evidence suggesting that 3D interaction could improve volume data analysis for a variety of visual analysis tasks. Our contribution is in the realm of 3D user interfaces tightly integrated with visualization, for improving the effectiveness of visual analysis of volume datasets. Based on our experience, we also provide some insights into hardware-agnostic principles for design of effective interaction techniques.

Leveraging Passive Haptic Feedback in Virtual Environments with the Elastic-Arm Approach

Haptic feedback is known to improve 3D interaction in virtual environments but current haptic interfaces remain complex and tailored to desktop interaction. In this paper, we describe an alternative approach called “Elastic-Arm” for incorporating haptic feedback in immersive virtual environments in a simple and cost-effective way. The Elastic-Arm is based on a body-mounted elastic armature that links the user's hand to the body and generates a progressive egocentric force when extending the arm. A variety of designs can be proposed with multiple links attached to various locations on the body in order to simulate different haptic properties and sensations such as different levels of stiffness, weight lifting, and bimanual interaction. Our passive haptic approach can be combined with various 3D interaction techniques and we illustrate the possibilities offered by the Elastic-Arm through several use cases based on well-known techniques such as the Bubble technique, redirected touching, and pseudo-haptics. A user study was conducted which showed the effectiveness of our pseudo-haptic technique as well as the general appreciation of the Elastic-Arm. We believe that the Elastic-Arm could be used in various VR applications which call for mobile haptic feedback or human-scale haptic sensations.

An Exploratory Study of 3D Interaction Techniques in Augmented Reality Environments

Augmented Reality (AR) interfaces typically involve the overlay of virtual imagery onto the real world. 3D interaction with the virtual scene is a key feature to explore the full potential of AR. Despite the large number of interaction techniques that have been proposed to enable for intuitive 3D interaction in AR environments, little effort has been done to compare these techniques. In addition, many techniques have been arbitrarily used without formative evaluation or without considering the requirements of different AR applications. This paper discusses three interaction techniques with 3D content in AR environments and reports on an experiment that we conducted to compare them. These techniques are: 1) manipulation of hand-held fiducial markers, 2) free hand interaction, and 3) a keypad controller. 18 participants were recruited and asked to perform a series of tasks which involved the manipulation of a 3D object. The study sheds the light on the strengths and limitations of the studied techniques and when it is appropriate to use each of them. We believe that the reported results will help inform the design and customization of 3D interaction techniques for AR applications.

Dual-Finger 3D Interaction Techniques for mobile devices

Three-dimensional capabilities on mobile devices are increasing, and the interactivity is becoming a key feature of these tools. It is expected that users will actively engage with the 3D content, instead of being passive consumers. Because touch-screens provide a direct means of interaction with 3D content by directly touching and manipulating 3D graphical elements, touch-based interaction is a natural and appealing style of input for 3D applications. However, developing 3D interaction techniques for handheld devices using touch-screens is not a straightforward task. One issue is that when interacting with 3D objects, users occlude the object with their fingers. Furthermore, because the user's finger covers a large area of the screen, the smallest size of the object users can touch is limited. In this paper, we first inspect existing 3D interaction techniques based on their performance with handheld devices. Then, we present a set of precise Dual-Finger 3D Interaction Techniques for a small display. Finally, we present the results of an experimental study, where we evaluate the usability, performance, and error rate of the proposed and existing 3D interaction techniques.

Context-driven interaction in immersive virtual environments

There are many interaction tasks a user may wish to accomplish in an immersive virtual environment. A careful examination of these tasks reveals that they are often performed under different contexts. For each task and context, specialized interaction techniques can be developed. We present the context-driven interaction model: a design pattern that represents contextual information as a first-class, quantifiable component within a user interface and supports the development of context-sensitive applications by decoupling context recognition, context representation, and interaction technique development. As a primary contribution, this model provides an enumeration of important representations of contextual information gathered from across the literature and describes how these representations can effect the selection of an appropriate interaction technique. We also identify how several popular 3D interaction techniques adhere to this design pattern and describe how the pattern itself can lead to a more focused development of effective interfaces. We have constructed a formalized programming toolkit and runtime system that serves as a reference implementation of the context-driven model and a discussion is provided explaining how the toolkit can be used to implement a collection of representative 3D interaction interfaces.

3DUI-EF: Towards a Framework for Easy Empirical Evaluation of 3D User Interfaces and Interaction Techniques

Designing usable and effective 3D User Interfaces and 3D Interaction Techniques is very challenging for Virtual Reality system developers and human factors specialists. Indeed, time consuming empirical evaluation is necessary to have an idea about the goodness of the 3D User Interface (3DUI) and the 3D Interaction Technique (3DIT) at the end of their development lifecycle. This may induce a huge loss of time if the result appears not to be satisfying in the end. Moreover, 3DUI evaluation is much more complex than 2D User Interfaces evaluation which is due to heterogeneous Virtual Reality (VR) devices and 3DIT. The aim of this work is to provide a framework allowing developers and experimenters to quickly evaluate 3DUIs and 3DITs during the design and the development lifecycle. The proposed framework is divided into two tools. The first one enables to create an evaluation protocol based on a knowledge database using two data mining algorithms, the "C4.5" to avoid from impossible combinations between devices and indicators and the "Spv Assoc Tree" to build a decision tree between indicators and factors. The second tool of the framework is an Evaluation Virtual Environment (EVE) to perform the evaluation according the protocol created with the first tool.

Building on realism and magic for designing 3D interaction techniques

Building on realism and magic for designing 3D interaction techniques

Building on realism and magic for designing 3D interaction techniques.

Imagination-based interaction can complement reality-based interaction in the design of 3D user interfaces. This hybrid approach could lead to interface design guidelines that promote higher-level consistency, and thus usability, for a large range of diverse interfaces.