Collaborative diagnosis in mixed-reality using deep-learning networks and RE-WAPICP algorithm

Abstract

This investigation explores the use of mixed-reality in collaborative diagnosis by sharing medical data in real-time between multiple physicians using Head-Mounted Display (HMD) devices. Object detection and alignment of the digitized data with the object are the backbone in any mixed-reality application. In this paper, deep-learning networks are used in detecting the patient’s face in the physical world and the medical data is aligned to the patient via the Region-Enhanced-Weight-and-Perturb Iterative-Closest-Point (RE-WAPICP) algorithm. Experiments were performed by sharing a 3D digital model of intracerebral vascular with multi-viewers in a mix-reality environment and the results show that this approach is feasible.