Gamified AR/VR Character Rendering and Animation-Enabling Technologies

Abstract

In recent years, the popularity of mixed reality (MR) environments has increased as they provide attractive and immersive experiences for educational, entertainment, and training purposes. The increasing advances of Augmented Reality (AR) and Virtual Reality (VR) hardware and software technologies also constitute an interesting area of research. In this work, we present the main pipeline followed for creating virtual character-based AR experiences, specifically in cultural heritage environments. Our main goal in this chapter is to compare different software methodologies for creating VR environments and present a complete novel methodology for authoring life-sized AR virtual characters and life-sized AR crowd simulation using only modern mobile devices. One important aspect of these environments that we focus on is creating realistic and interactive virtual characters via procedurally generated body and facial animations which are illuminated with real environment light. Virtual characters' transformations are handled efficiently using a single mathematical framework, the 3D Euclidean geometric algebra (GA), and the conformal geometric algebra (CGA) which is able to handle translations, rotations, and dilations. Using such a single algebraic framework, we avoid conversions between different mathematical representations; as a result, we achieve more efficient performance. We also compare the efficiency of different GA code generators—(a) the Gaigen library, (b) libvsr, and (c) Gaalop—so that a future user of GA can choose the most appropriate, currently available s/w library that will provide the most optimal and efficient results. Our main research involves the following questions: (a) Are novel, low-cost, modern HMDs suitable as VR platforms? (b) Which are the most appropriate s/w platforms needed to realize such VR digital heritage gamified experiences? (c) Can we achieve more efficient AR scene authoring? (d) Can we achieve more efficient AR animation interpolation and skinning using a single mathematical framework employing GA?