Hand Interaction Toolset for Augmented Reality Environments

Abstract

As Augmented and Virtual Reality (AR/VR) technologies become nowadays very popular, there is the need for better interaction in these environments. In this paper, a toolset that facilitates the process of creating interaction in AR and VR environments is presented. The toolset is mainly targeted on mobile devices as the majority of AR and VR applications are exported. The toolset is divided into two main parts; the first part presents the components that are utilized inside the Unity 3D game engine, and the second part involves a webservice to handle the hand tracking task. The separation of hand tracking into a different part allows changes of the algorithms without the need to update the entire package. To evaluate the proposed toolset, a scenario of usage was fabricated in which users -with a computer science background- were shared a package and were asked to complete three tasks and evaluate their experience of using the toolset to create applications. Furthermore, the resulting applications were tested in real case scenarios to examine the overall performance and experience.