Medical augment reality using a markerless registration framework

Abstract

This study proposes a cranial augmented-reality (AR) system which characterizes on performing image-to-patient registration using only natural facial features. Its hardware includes only three calibrated CCD cameras but any commercial spatial digitizing devices or artificial skin markers. Two of the cameras are mounted together to form a stereo-vision system, while the third camera moves freely and captures real-time images for AR-fusion display. The facial surface of a patient is first reconstructed by stereo vision. Meanwhile, another facial surface is reconstructed from preoperative computed tomography (CT) images of the patient. A iterative closest point (ICP)-based algorithm is then used to register the two facial data sets, which transfers the anatomical information from the CT images to the physical space. Since the natural of stereo-vision reconstruction usually accompanies with noisy data, the conventional ICP also suffers from its inherent local-minimum problem. Therefore, we propose an improved ICP which embeds a weighting and perturbing strategy to increase robustness and the ability of noise resistance. As a result, with a seamless integration among the preoperative CT images, the patient, the stereo-vision cameras, and the movable camera, an immersive medical AR environment based on a markerless image-to-patient registration is thus accomplished.