3D object reconstruction in image-guided interventions using multi-view X-ray

Abstract

In the last two decades, minimally-invasive interventions have replaced traditional surgery in many clinical scenarios. In these interventions, the doctor manipulates small devices inside the patient through a small incision, while guided by live imaging. In many cases, this guidance is provided by low-dose X-ray imaging. At this moment, live image guidance conveys only two-dimensional (2D) information, whereas information on the 3D location and orientation of structures of interest would resolve 3D positioning ambiguities and significantly aid the accuracy and safety of these procedures. The work described in this thesis aims at providing accurate and reliable 3D information for current interventional systems, by employing multiple X-ray views, acquired with a limited motion of the X-ray imaging apparatus. The application of 2D image analysis techniques in combination with 3D modeling enables 3D reconstructions of objects in the image. The work in this thesis is organized into three layers of increasing complexity of the 3D reconstructed objects. Prior to addressing the reconstruction problem, the thesis begins in Chapter 2 with a consideration of important system aspects pertaining to X-ray imaging, focusing on image quality, which plays a crucial role in the success of image analysis algorithms. The reported work contributes an image quality assessment method, based on an information-theoretic approach, which encapsulates the major image quality aspects (namely contrast, sharpness and noise) and formulates them in the domain of information. Chapters 3 and 4 present the first layer of reconstruction, targeting single feature points. At this layer, our work has provided a thorough analysis of multi-view relations, as formulated for C-arm based X-ray. A distinction is made between the 2D image transformation, feature point detection and tracking step (Chapter 3), and the subsequent 3D camera modeling and reconstruction steps (Chapter 4). In this part of the thesis, we have contributed: (1) a method for evaluation of feature point detection techniques for non-planar scenes, (2) a tracking algorithm based on geometric constraints, which allows fast tracking of feature points, and (3) the first –to the best of our knowledge– analysis of the 3D point reconstruction accuracy and related requirements of multi-view X-ray. Simulation results show that 3D point reconstruction using 5-10 views spanning a rotation angle of 8.5± i17± is accurate to within 1 mm, while results on phantom sequences have shown that the tracked feature points can be reconstructed with an accuracy of about 1-4 mm. Chapters 5 and 6 discuss the second reconstruction layer of rigid objects. We have chosen to reconstruct curvilinear objects, as these may be used to model many surgical instruments such as e.g. catheters, needles, etc. A 2D modeling step, described in Chapter 5, precedes the 3D reconstruction. The 2D modeling aims at detecting and tracking curves in the multi-view sequence, which are subsequently used in Chapter 6, to obtain 3D curves representing the objects of interest. The main contributions here are: (1) a novel algorithm, called SPD-RANSAC, for the detection of multiple (curvilinear) models in noisy images, (2) a curve tracking algorithm, based on geometric constraints and a cost function, and (3) a curve reconstruction technique, which can be potentially refined by adding a non-linear optimization step. Here we have demonstrated reconstructions with an accuracy of 1-2 mm for phantom datasets, and ¼ 5 mm for clinical datasets. This enables the simultaneous 2D detection, tracking and 3D reconstruction of several curvilinear instruments using only a few X-ray views. Chapter 7 treats the third layer of non-rigid object reconstruction, dealing with the challenging problem of 3D reconstruction when motion occurs during the image acquisition. In our application scenario, such motion stems from patient breathing, heartbeat, instrument manipulation by the doctor, etc. In computer vision, observing a moving object with a moving camera is an inherently underconstrained problem, termed Non-Rigid Structure-from-Motion. We analyze this complex problem for the case of steerable catheters used in cardiac ablation and contribute a solution for deformable, time-varying catheter reconstruction. A model from the field of Robotics is employed to parameterize deforming 3D+T shape. Simulations have shown that a non-linear optimization scheme succeeds in correctly recovering 3D+T catheter shape with an accuracy of a few millimeters, while phantom experiments recover catheter shape with a repeatability of 5 mm. The results demonstrated for each of the reconstruction layers have shown that multi-view X-ray can provide 3D reconstructions of relevant objects, with a sufficiently high accuracy for a number of interventions; the setup employed requires no additional equipment apart from the existing interventional X-ray system. We therefore conclude that multi-view X-ray, along with the techniques proposed in this thesis, can be employed in the near future for unambiguous 3D guidance in a real clinical scenario.