TooltY: An Approach for the Combination of Motion Capture and 3D Reconstruction to Present Tool Usage in 3D Environments

Abstract

Visualization techniques for the usage of tools, handicrafts, and assembly operations are employed for demonstrating processes (e.g., assembly instructions). Until today, most commonly used techniques include written information, sketches in manuals, video instructions, etc. The evolution of technology nowadays has generated mature methods for transforming movement to digital information that can be processed and replicated. Motion capture together with 3D reconstruction techniques can provide new ways of digitizing handicrafts. At the same time, Virtual Humans can be used to present craft processes, as well as to demonstrate the usage of tools. For this, the tools utilized in these processes need to be transferred to the digital world. In this paper, we present TooltY, a 3D authoring platform for tool usage presentation in 3D environments, to demonstrate simple operations (e.g., usage of a hammer, scissors, screwdriver), where the tools are the product of 3D reconstruction. The movement of the Virtual Humans derives from motion capture, while for the movement of the tools, a novel approach is used, for inducing the tool motion from the human motion capture. The products of TooltY are Virtual Environments that can be experienced in 3D or through immersion in Virtual Reality.