Enabling and Analyzing Natural Interaction with Functional Virtual Prototypes

Abstract

Functional validations of virtual prototypes are a promising application area of immersive Virtual Reality (Moehring & Froehlich, 2011). Through the interactive simulation of operating procedures such analyses aim at providing insight about virtual prototypes, e. g. concerning visibility aspects, reachability of control elements, and ergonomics. A key requirement is the enabling of natural defined by Zachmann as interaction which imitates that same in the real world as close as possible (Zachmann & Rettig, 2001). In immersive VR, natural can be realized through data gloves and motion tracking devices. However, a challenging task remains in the modeling and simulation of interactions with dynamic control elements such as sliders, switches etc. (Moehring & Frohlich, 2010). We have devised and implemented a framework that not only aims at simplifying the specification of such control actuators for interactive virtual environments but also facilitates the recording, automated classification, and analysis of natural interactions with such control actuators. The presented approach builds on the European Standard EN 894-3 (DIN894-3, 2006) and covers all types of control actuators defined therein. This standard defines systematic guidelines for the choice and configuration of control actuators to ergonomically design machinery. A characterizing feature of control actuators in the real world is given by their function: Controlling the behavior of machines and other appliances. In order to model the triggering of changes of environment objects in the simulation, our framework allows to associate control actuators with events. Interaction events may be fired continuously, e. g. during movement of sliders, or when discrete states are reached, e. g. switches. Besides making interactive simulations more realistic, events can also be recorded to allow later playback and analysis of the interaction. For this, detailed timing information is stored in the events. For example, events may be used for the animation of virtual humans that imitate the user interactions when operating virtual prototypes. Contributions of our research include an XML-based modeling language for control actuators that can be seen as both an abstraction layer and an extension to lower-level rigid body physics modeling capabilities provided e. g. by Collada (Khronos Group, 2008) and X3D (International Organization for Standardization, 2008). Further, we produce a reference implementation of control actuators with direct object manipulation, an database for analyzing events including recognizing basic interactions and grasp classification, and a powerful graphical analysis tool. Through this, the process of adding interactive elements to virtual prototypes can be drastically simplified. 11