Virtually Documented Environments: A New Interface Paradigm for Task‐oriented Access to Information

Abstract

We present a suitable virtually documented environment system providing the user with high level interaction possibilities. The system is dedicated to applications where the operator needs to have his hands free in order to access information, carry out measurements and/or operate on a device (e.g. maintenance, instruction). The system merges video images acquired through a head‐mounted video camera with synthetic data (multimedia documents including CAD models and text) and presents these merged images to the operator. Registration techniques allow the operator to visualise information properly correlated to the real world: this is an essential aspect in order to achieve a feeling of presence in a real environment.We increase the sense of immersion through high level Human‐Computer Interaction (HCI) allowing hands‐free access to information through vocal commands as well as multimodal interaction associating speech and gesture. In this way, the user can access information and manipulate it in a very natural manner. We discuss the construction of the documentation system and the requested functionalities which led to the system architecture.