Integration of Physical Reality Objects with Their 3D Models Visualized in Virtual Environment Systems

Abstract

The paper describes methods and approaches for integrating, from the user's point of view, three-dimensional virtual models visible to them in a virtual reality headset, and their physical prototypes. This allows for the addition of some important elements of physical reality to the virtual environment, such as tactile and muscle-motor sensations. Developed solutions are based on the use of wireless HTC Vive trackers, which determine their position and orientation in space and have the ability to be attached to various objects in the real world. An original approach to building a tracking system from several controllers of such type with one anchor element selection is proposed, as well as new methods for real-time integration of physical reality objects with their virtual models by means of functional control schemes with blocks for the Vive trackers. Software modules were created based on developed methods and approaches. Approbation of them was carried out in virtual environment system VirSim and showed adequacy and effectiveness of proposed solutions when using in virtual environment systems and training complexes.