A Novel Neurosurgery Registration Pipeline Based on Heat Maps and Anatomic Facial Feature Points

Abstract

Current neurosurgery registration methods can be divided into two categories: marker-based and markerless. Maker-based methods need fiducials on patients' heads, which may cause additional harm to them. Traditional markerless registration methods always require interaction, expensive equipment and specific posture of patients. In this paper, we explore how deep neutral network can be used in neurosurgery registration problem. And the method of this paper mainly includes three parts: 3D reconstruction and face mesh model feature point extraction, binocular face feature point cloud generation, point cloud registration. To this end, we make the following contributions: (a) we propose an automatic registration pipeline based on heat maps generated from Convolutional Neural Network (CNN), which is totally dependent on the anatomic facial feature points of patients and free of extra fiducials. (b) we use comparatively cheap binocular cameras instead of surface scanner to collect patients' facial information. (c) we don't use 3D CNNs to extract facial feature points of the medical images like CT, instead we use 2D CNNs on the projection images and get 3D facial feature points from their corresponding 2D facial feature points. As a result, it can relieve the lack of 3D training data and save the run-time memory. We show that the registration precision is dominant to the accuracy of facial key point extraction, and the network we use exhibits powerful performance.