A multi-GPU accelerated virtual-reality interaction simulation framework.

Xuqiang Shao,Fengquan Zhang,Lina Lin,Weifeng Xu,Huijia Li

doi:10.1371/journal.pone.0214852

Abstract

In this paper, we put forward a real-time multiple GPUs (multi-GPU) accelerated virtual-reality interaction simulation framework where the reconstructed objects from camera images interact with virtual deformable objects. Firstly, based on an extended voxel-based visual hull (VbVH) algorithm, we design an image-based 3D reconstruction platform for real objects. Then, an improved hybrid deformation model, which couples the geometry constrained fast lattice shape matching method (FLSM) and total Lagrangian explicit dynamics (TLED) algorithm, is proposed to achieve efficient and stable simulation of the virtual objects’ elastic deformations. Finally, one-way virtual-reality interactions including soft tissues’ virtual cutting with bleeding effects are successfully simulated. Moreover, with the purpose of significantly improving the computational efficiency of each time step, we propose an entire multi-GPU implementation method of the framework using compute unified device architecture (CUDA). The experiment results demonstrate that our multi-GPU accelerated virtual-reality interaction framework achieves real-time performance under the moderate calculation scale, which is a new effective 3D interaction technique for virtual reality applications.

Highlights

Natural and real-time 3D interaction with computer-generated virtual environment is an important research topic of virtual reality and augmented reality technologies, as it increases the participant’s sense of immersion
There have been many practical application examples of 3D interaction framework based on the haptic feedback devices, especially in the more and more commonly used virtual surgery area
In order to solve the above problems to improve the interaction experiences, based on our previous works on hybrid deformation model for virtual cutting [12], fluid-solid coupling [13] and visual hull reconstruction [14], we develop a type of image-based virtual-reality interaction framework implemented on multiple GPUs using compute unified device architecture (CUDA), which supports more types of interactions including virtual cutting and achieves higher computational efficiency compared to other similar methods

Summary

Introduction

Natural and real-time 3D interaction with computer-generated virtual environment is an important research topic of virtual reality and augmented reality technologies, as it increases the participant’s sense of immersion. There have been many practical application examples of 3D interaction framework based on the haptic feedback devices, especially in the more and more commonly used virtual surgery area. Bro-Nielsen et al [1] developed a virtual surgical simulator for training the removal of a shattered kidney by open surgery, which allows the users to probe and cut the virtual soft tissues by using the SenSable Technologies Phantom haptic feedback device. By supporting haptic feedbacks through Phantom Omni devices, Webster et al [2] proposed a realistic, easy to operate, and economical prototype of haptic suturing simulator to train basic suturing operations for simple wound closure. To support complex virtual cutting interactions, Bielser et al [3] designed a realistic.

Methods

Results

Conclusion