2D-3D Image Registration Research Articles

Intraoperative evaluation of device placement in spine surgery using known-component 3D–2D image registration

Intraoperative x-ray radiography/fluoroscopy is commonly used to assess the placement of surgical devices in the operating room (e.g. spine pedicle screws), but qualitative interpretation can fail to reliably detect suboptimal delivery and/or breach of adjacent critical structures. We present a 3D–2D image registration method wherein intraoperative radiographs are leveraged in combination with prior knowledge of the patient and surgical components for quantitative assessment of device placement and more rigorous quality assurance (QA) of the surgical product. The algorithm is based on known-component registration (KC-Reg) in which patient-specific preoperative CT and parametric component models are used. The registration performs optimization of gradient similarity, removes the need for offline geometric calibration of the C-arm, and simultaneously solves for multiple component bodies, thereby allowing QA in a single step (e.g. spinal construct with 4–20 screws). Performance was tested in a spine phantom, and first clinical results are reported for QA of transpedicle screws delivered in a patient undergoing thoracolumbar spine surgery. Simultaneous registration of ten pedicle screws (five contralateral pairs) demonstrated mean target registration error (TRE) of 1.1 ± 0.1 mm at the screw tip and 0.7 ± 0.4° in angulation when a prior geometric calibration was used. The calibration-free formulation, with the aid of component collision constraints, achieved TRE of 1.4 ± 0.6 mm. In all cases, a statistically significant improvement (p < 0.05) was observed for the simultaneous solutions in comparison to previously reported sequential solution of individual components. Initial application in clinical data in spine surgery demonstrated TRE of 2.7 ± 2.6 mm and 1.5 ± 0.8°. The KC-Reg algorithm offers an independent check and quantitative QA of the surgical product using radiographic/fluoroscopic views acquired within standard OR workflow. Such intraoperative assessment could improve quality and safety, provide the opportunity to revise suboptimal constructs in the OR, and reduce the frequency of revision surgery.

Parallel generation of digitally reconstructed radiographs on heterogeneous multi-GPU workstations.

The growing importance of three-dimensional radiotherapy treatment has been associated with the active presence of advanced computational workflows that can simulate conventional x-ray films from computed tomography (CT) volumetric data to create digitally reconstructed radiographs (DRR). These simulated x-ray images are used to continuously verify the patient alignment in image-guided therapies with 2D-3D image registration. The present DRR rendering pipelines are quite limited to handle huge imaging stacks generated by recent state-of-the-art CT imaging modalities. We present a high performance x-ray rendering pipeline that is capable of generating high quality DRRs from large scale CT volumes. The pipeline is designed to harness the immense computing power of all the heterogeneous computing platforms that are connected to the system relying on OpenCL. Load-balancing optimization is also addressed to equalize the rendering load across the entire system. The performance benchmarks demonstrate the capability of our pipeline to generate high quality DRRs from relatively large CT volumes at interactive frame rates using cost-effective multi-GPU workstations. A 5122 DRR frame can be rendered from 1024 × 2048 × 2048 CT volumes at 85 frames per second.

Efficient rendering of digitally reconstructed radiographs on heterogeneous computing architectures using central slice theorem.

Digitally reconstructed radiographs (DRRs) play a significant role in modern clinical radiation therapy. They are used to verify patient alignments during image guided therapies with 2D-3D image registration. The generation of DRRs can be implemented intuitively in O(N3) relying on direct volume rendering (DVR) methods, such as ray marching. This complexity imposes certain limitations on the rendering performance if high quality DRR images are needed. Those DRRs can be alternatively generated in the k-space using the central slice theorem in O(N2logN). Several rendering pipelines have been designed to create the DRRs in the k-space, but they were either limited to specific vendor or entail particular software requirements. We present a high performance implementation of a k-space-based DRR generation pipeline that is executable on various heterogeneous computing architectures using OpenCL. Our implementation generates a DRR for a 5123 CT volume in 6, 2.7 and 0.68 milli-seconds on a commodity CPU, mid-range and high-end GPUs respectively.

3D–2D image registration for target localization in spine surgery: investigation of similarity metrics providing robustness to content mismatch

In image-guided spine surgery, robust three-dimensional to two-dimensional (3D–2D) registration of preoperative computed tomography (CT) and intraoperative radiographs can be challenged by the image content mismatch associated with the presence of surgical instrumentation and implants as well as soft-tissue resection or deformation. This work investigates image similarity metrics in 3D–2D registration offering improved robustness against mismatch, thereby improving performance and reducing or eliminating the need for manual masking.The performance of four gradient-based image similarity metrics (gradient information (GI), gradient correlation (GC), gradient information with linear scaling (GS), and gradient orientation (GO)) with a multi-start optimization strategy was evaluated in an institutional review board-approved retrospective clinical study using 51 preoperative CT images and 115 intraoperative mobile radiographs. Registrations were tested with and without polygonal masks as a function of the number of multistarts employed during optimization. Registration accuracy was evaluated in terms of the projection distance error (PDE) and assessment of failure modes (PDE > 30 mm) that could impede reliable vertebral level localization.With manual polygonal masking and 200 multistarts, the GC and GO metrics exhibited robust performance with 0% gross failures and median PDE < 6.4 mm (±4.4 mm interquartile range (IQR)) and a median runtime of 84 s (plus upwards of 1–2 min for manual masking). Excluding manual polygonal masks and decreasing the number of multistarts to 50 caused the GC-based registration to fail at a rate of >14%; however, GO maintained robustness with a 0% gross failure rate. Overall, the GI, GC, and GS metrics were susceptible to registration errors associated with content mismatch, but GO provided robust registration (median PDE = 5.5 mm, 2.6 mm IQR) without manual masking and with an improved runtime (29.3 s).The GO metric improved the registration accuracy and robustness in the presence of strong image content mismatch. This capability could offer valuable assistance and decision support in spine level localization in a manner consistent with clinical workflow.

Automatic Masking for Robust 3D-2D Image Registration in Image-Guided Spine Surgery.

During spinal neurosurgery, patient-specific information, planning, and annotation such as vertebral labels can be mapped from preoperative 3D CT to intraoperative 2D radiographs via image-based 3D-2D registration. Such registration has been shown to provide a potentially valuable means of decision support in target localization as well as quality assurance of the surgical product. However, robust registration can be challenged by mismatch in image content between the preoperative CT and intraoperative radiographs, arising, for example, from anatomical deformation or the presence of surgical tools within the radiograph. In this work, we develop and evaluate methods for automatically mitigating the effect of content mismatch by leveraging the surgical planning data to assign greater weight to anatomical regions known to be reliable for registration and vital to the surgical task while removing problematic regions that are highly deformable or often occluded by surgical tools. We investigated two approaches to assigning variable weight (i.e., "masking") to image content and/or the similarity metric: (1) masking the preoperative 3D CT ("volumetric masking"); and (2) masking within the 2D similarity metric calculation ("projection masking"). The accuracy of registration was evaluated in terms of projection distance error (PDE) in 61 cases selected from an IRB-approved clinical study. The best performing of the masking techniques was found to reduce the rate of gross failure (PDE > 20 mm) from 11.48% to 5.57% in this challenging retrospective data set. These approaches provided robustness to content mismatch and eliminated distinct failure modes of registration. Such improvement was gained without additional workflow and has motivated incorporation of the masking methods within a system under development for prospective clinical studies.

Self-calibration of cone-beam CT geometry using 3D–2D image registration

Robotic C-arms are capable of complex orbits that can increase field of view, reduce artifacts, improve image quality, and/or reduce dose; however, it can be challenging to obtain accurate, reproducible geometric calibration required for image reconstruction for such complex orbits. This work presents a method for geometric calibration for an arbitrary source-detector orbit by registering 2D projection data to a previously acquired 3D image. It also yields a method by which calibration of simple circular orbits can be improved. The registration uses a normalized gradient information similarity metric and the covariance matrix adaptation-evolution strategy optimizer for robustness against local minima and changes in image content. The resulting transformation provides a ‘self-calibration’ of system geometry. The algorithm was tested in phantom studies using both a cone-beam CT (CBCT) test-bench and a robotic C-arm (Artis Zeego, Siemens Healthcare) for circular and non-circular orbits. Self-calibration performance was evaluated in terms of the full-width at half-maximum (FWHM) of the point spread function in CBCT reconstructions, the reprojection error (RPE) of steel ball bearings placed on each phantom, and the overall quality and presence of artifacts in CBCT images. In all cases, self-calibration improved the FWHM—e.g. on the CBCT bench, FWHM = 0.86 mm for conventional calibration compared to 0.65 mm for self-calibration (p < 0.001). Similar improvements were measured in RPE—e.g. on the robotic C-arm, RPE = 0.73 mm for conventional calibration compared to 0.55 mm for self-calibration (p < 0.001). Visible improvement was evident in CBCT reconstructions using self-calibration, particularly about high-contrast, high-frequency objects (e.g. temporal bone air cells and a surgical needle). The results indicate that self-calibration can improve even upon systems with presumably accurate geometric calibration and is applicable to situations where conventional calibration is not feasible, such as complex non-circular CBCT orbits and systems with irreproducible source-detector trajectory.

Deformable 3D-2D Registration of Known Components for Image Guidance in Spine Surgery.

A 3D-2D image registration method is reported for guiding the placement of surgical devices (e.g., K-wires). The solution registers preoperative CT (and planning data therein) to intraoperative radiographs and computes the pose, shape, and deformation parameters of devices (termed "components") known to be in the radiographic scene. The deformable known-component registration (dKC-Reg) method was applied in experiments emulating spine surgery to register devices (K-wires and spinal fixation rods) undergoing realistic deformation. A two-stage registration process (i) resolves patient pose from individual radiographs and (ii) registers components represented as polygonal meshes based on a B-spline model. The registration result can be visualized as overlay of the component in CT analogous to surgical navigation but without conventional trackers or fiducials. Target registration error in the tip and orientation of deformable K-wires was and , respectively. For spinal fixation rods, the registered components achieved Hausdorff distance of 3.4 mm. Future work includes testing in cadaver and clinical data and extension to more generalized deformation and component models.

Simultaneous 3D-2D image registration and C-arm calibration: Application to endovascular image-guided interventions.

Three-dimensional to two-dimensional (3D-2D) image registration is a key to fusion and simultaneous visualization of valuable information contained in 3D pre-interventional and 2D intra-interventional images with the final goal of image guidance of a procedure. In this paper, the authors focus on 3D-2D image registration within the context of intracranial endovascular image-guided interventions (EIGIs), where the 3D and 2D images are generally acquired with the same C-arm system. The accuracy and robustness of any 3D-2D registration method, to be used in a clinical setting, is influenced by (1) the method itself, (2) uncertainty of initial pose of the 3D image from which registration starts, (3) uncertainty of C-arm's geometry and pose, and (4) the number of 2D intra-interventional images used for registration, which is generally one and at most two. The study of these influences requires rigorous and objective validation of any 3D-2D registration method against a highly accurate reference or "gold standard" registration, performed on clinical image datasets acquired in the context of the intervention. The registration process is split into two sequential, i.e., initial and final, registration stages. The initial stage is either machine-based or template matching. The latter aims to reduce possibly large in-plane translation errors by matching a projection of the 3D vessel model and 2D image. In the final registration stage, four state-of-the-art intrinsic image-based 3D-2D registration methods, which involve simultaneous refinement of rigid-body and C-arm parameters, are evaluated. For objective validation, the authors acquired an image database of 15 patients undergoing cerebral EIGI, for which accurate gold standard registrations were established by fiducial marker coregistration. Based on target registration error, the obtained success rates of 3D to a single 2D image registration after initial machine-based and template matching and final registration involving C-arm calibration were 36%, 73%, and 93%, respectively, while registration accuracy of 0.59 mm was the best after final registration. By compensating in-plane translation errors by initial template matching, the success rates achieved after the final stage improved consistently for all methods, especially if C-arm calibration was performed simultaneously with the 3D-2D image registration. Because the tested methods perform simultaneous C-arm calibration and 3D-2D registration based solely on anatomical information, they have a high potential for automation and thus for an immediate integration into current interventional workflow. One of the authors' main contributions is also comprehensive and representative validation performed under realistic conditions as encountered during cerebral EIGI.

2D Ultrasound and 3D MR Image Registration of the Prostate for Brachytherapy Surgical Navigation.

Two-dimensional (2D) ultrasound (US) images are widely used in minimally invasive prostate procedure for its noninvasive nature and convenience. However, the poor quality of US image makes it difficult to be used as guiding utility. To improve the limitation, we propose a multimodality image guided navigation module that registers 2D US images with magnetic resonance imaging (MRI) based on high quality preoperative models. A 2-step spatial registration method is used to complete the procedure which combines manual alignment and rapid mutual information (MI) optimize algorithm. In addition, a 3-dimensional (3D) reconstruction model of prostate with surrounding organs is employed to combine with the registered images to conduct the navigation. Registration accuracy is measured by calculating the target registration error (TRE). The results show that the error between the US and preoperative MR images of a polyvinyl alcohol hydrogel model phantom is 1.37 ± 0.14 mm, with a similar performance being observed in patient experiments.

Known-component 3D–2D registration for quality assurance of spine surgery pedicle screw placement

A 3D–2D image registration method is presented that exploits knowledge of interventional devices (e.g. K-wires or spine screws—referred to as ‘known components’) to extend the functionality of intraoperative radiography/fluoroscopy by providing quantitative measurement and quality assurance (QA) of the surgical product.The known-component registration (KC-Reg) algorithm uses robust 3D–2D registration combined with 3D component models of surgical devices known to be present in intraoperative 2D radiographs. Component models were investigated that vary in fidelity from simple parametric models (e.g. approximation of a screw as a simple cylinder, referred to as ‘parametrically-known’ component [pKC] registration) to precise models based on device-specific CAD drawings (referred to as ‘exactly-known’ component [eKC] registration). 3D–2D registration from three intraoperative radiographs was solved using the covariance matrix adaptation evolution strategy (CMA-ES) to maximize image-gradient similarity, relating device placement relative to 3D preoperative CT of the patient. Spine phantom and cadaver studies were conducted to evaluate registration accuracy and demonstrate QA of the surgical product by verification of the type of devices delivered and conformance within the ‘acceptance window’ of the spinal pedicle.Pedicle screws were successfully registered to radiographs acquired from a mobile C-arm, providing TRE 1–4 mm and <5° using simple parametric (pKC) models, further improved to <1 mm and <1° using eKC registration. Using advanced pKC models, screws that did not match the device models specified in the surgical plan were detected with an accuracy of >99%. Visualization of registered devices relative to surgical planning and the pedicle acceptance window provided potentially valuable QA of the surgical product and reliable detection of pedicle screw breach.3D–2D registration combined with 3D models of known surgical devices offers a novel method for intraoperative QA. The method provides a near-real-time independent check against pedicle breach, facilitating revision within the same procedure if necessary and providing more rigorous verification of the surgical product.

Development of fast patient position verification software using 2D-3D image registration and its clinical experience.

To improve treatment workflow, we developed a graphic processing unit (GPU)-based patient positional verification software application and integrated it into carbon-ion scanning beam treatment. Here, we evaluated the basic performance of the software. The algorithm provides 2D/3D registration matching using CT and orthogonal X-ray flat panel detector (FPD) images. The participants were 53 patients with tumors of the head and neck, prostate or lung receiving carbon-ion beam treatment. 2D/3D-ITchi-Gime (ITG) calculation accuracy was evaluated in terms of computation time and registration accuracy. Registration calculation was determined using the similarity measurement metrics gradient difference (GD), normalized mutual information (NMI), zero-mean normalized cross-correlation (ZNCC), and their combination. Registration accuracy was dependent on the particular metric used. Representative examples were determined to have target registration error (TRE) = 0.45 ± 0.23 mm and angular error (AE) = 0.35 ± 0.18° with ZNCC + GD for a head and neck tumor; TRE = 0.12 ± 0.07 mm and AE = 0.16 ± 0.07° with ZNCC for a pelvic tumor; and TRE = 1.19 ± 0.78 mm and AE = 0.83 ± 0.61° with ZNCC for lung tumor. Calculation time was less than 7.26 s.The new registration software has been successfully installed and implemented in our treatment process. We expect that it will improve both treatment workflow and treatment accuracy.

Fully automatic initialization of two-dimensional-three-dimensional medical image registration using hybrid classifier.

X-ray video fluoroscopy along with two-dimensional-three-dimensional (2D-3D) registration techniques is widely used to study joints in vivo kinematic behaviors. These techniques, however, are generally very sensitive to the initial alignment of the 3-D model. We present an automatic initialization method for 2D-3D registration of medical images. The contour of the knee bone or implant was first automatically extracted from a 2-D x-ray image. Shape descriptors were calculated by normalized elliptical Fourier descriptors to represent the contour shape. The optimal pose was then determined by a hybrid classifier combining [Formula: see text]-nearest neighbors and support vector machine. The feasibility of the method was first validated on computer synthesized images, with 100% successful estimation for the femur and tibia implants, 92% for the femur and 95% for the tibia. The method was further validated on fluoroscopic x-ray images with all the poses of the testing cases successfully estimated. Finally, the method was evaluated as an initialization of a feature-based 2D-3D registration. The initialized and uninitialized registrations had success rates of 100% and 50%, respectively. The proposed method can be easily utilized for 2D-3D image registration on various medical objects and imaging modalities.

WE-AB-BRA-07: Quantitative Evaluation of 2D-2D and 2D-3D Image Guided Radiation Therapy for Clinical Trial Credentialing, NRG Oncology/RTOG

Purpose: 2D-2D kV image guided radiation therapy (IGRT) credentialing evaluation for clinical trial qualification was historically qualitative through submitting screen captures of the fusion process. However, as quantitative DICOM 2D-2D and 2D-3D image registration tools are implemented in clinical practice for better precision, especially in centers that treat patients with protons, better IGRT credentialing techniques are needed. The aim of this work is to establish methodologies for quantitatively reviewing IGRT submissions based on DICOM 2D-2D and 2D-3D image registration and to test the methodologies in reviewing 2D-2D and 2D-3D IGRT submissions for RTOG/NRG Oncology clinical trials qualifications. Methods: DICOM 2D-2D and 2D-3D automated and manual image registration have been tested using the Harmony tool in MIM software. 2D kV orthogonal portal images are fused with the reference digital reconstructed radiographs (DRR) in the 2D-2D registration while the 2D portal images are fused with DICOM planning CT image in the 2D-3D registration. The Harmony tool allows alignment of the two images used in the registration process and also calculates the required shifts. Shifts calculated using MIM are compared with those submitted by institutions for IGRT credentialing. Reported shifts are considered to be acceptable if differences are less than 3mm. Results: Several tests have been performed on the 2D-2D and 2D-3D registration. The results indicated good agreement between submitted and calculated shifts. A workflow for reviewing these IGRT submissions has been developed and will eventually be used to review IGRT submissions. Conclusion: The IROC Philadelphia RTQA center has developed and tested a new workflow for reviewing DICOM 2D-2D and 2D-3D IGRT credentialing submissions made by different cancer clinical centers, especially proton centers. NRG Center for Innovation in Radiation Oncology (CIRO) and IROC RTQA center continue their collaborative efforts to enhance quality assurance services and to be consistently adaptive to the new advances in radiation therapy. This project was supported by NCI grants U10CA180868, U10CA180822, U24CA180803, U24CA12014 and PA CURE Grant.

WE-AB-BRA-01: 3D-2D Image Registration for Target Localization in Spine Surgery: Comparison of Similarity Metrics Against Robustness to Content Mismatch

Purpose: In image-guided spine surgery, mapping 3D preoperative images to 2D intraoperative images via 3D-2D registration can provide valuable assistance in target localization. However, the presence of surgical instrumentation, hardware implants, and soft-tissue resection/displacement causes mismatches in image content, confounding existing registration methods. Manual/semi-automatic methods to mask such extraneous content is time consuming, user-dependent, error prone, and disruptive to clinical workflow. We developed and evaluated 2 novel similarity metrics within a robust registration framework to overcome such challenges in target localization. Methods: An IRB-approved retrospective study in 19 spine surgery patients included 19 preoperative 3D CT images and 50 intraoperative mobile radiographs in cervical, thoracic, and lumbar spine regions. A neuroradiologist provided truth definition of vertebral positions in CT and radiography. 3D-2D registration was performed using the CMA-ES optimizer with 4 gradient-based image similarity metrics: (1) gradient information (GI); (2) gradient correlation (GC); (3) a novel variant referred to as gradient orientation (GO); and (4) a second variant referred to as truncated gradient correlation (TGC). Registration accuracy was evaluated in terms of the projection distance error (PDE) of the vertebral levels. Results: Conventional similarity metrics were susceptible to gross registration error and failure modes associated with the presence of surgical instrumentation: for GI, the median PDE and interquartile range was 33.0±43.6 mm; similarly for GC, PDE = 23.0±92.6 mm respectively. The robust metrics GO and TGC, on the other hand, demonstrated major improvement in PDE (7.6 ±9.4 mm and 8.1± 18.1 mm, respectively) and elimination of gross failure modes. Conclusion: The proposed GO and TGC similarity measures improve registration accuracy and robustness to gross failure in the presence of strong image content mismatch. Such registration capability could offer valuable assistance in target localization without disruption of clinical workflow. G. Kleinszig and S. Vogt are employees of Siemens Healthcare.

Automatic localization of target vertebrae in spine surgery: clinical evaluation of the LevelCheck registration algorithm.

A 3-dimensional-2-dimensional (3D-2D) image registration algorithm, "LevelCheck," was used to automatically label vertebrae in intraoperative mobile radiographs obtained during spine surgery. Accuracy, computation time, and potential failure modes were evaluated in a retrospective study of 20 patients. To measure the performance of the LevelCheck algorithm using clinical images acquired during spine surgery. In spine surgery, the potential for wrong level surgery is significant due to the difficulty of localizing target vertebrae based solely on visual impression, palpation, and fluoroscopy. To remedy this difficulty and reduce the risk of wrong-level surgery, our team introduced a program (dubbed LevelCheck) to automatically localize target vertebrae in mobile radiographs using robust 3D-2D image registration to preoperative computed tomographic (CT) scan. Twenty consecutive patients undergoing thoracolumbar spine surgery, for whom both a preoperative CT scan and an intraoperative mobile radiograph were available, were retrospectively analyzed. A board-certified neuroradiologist determined the "true" vertebra levels in each radiograph. Registration of the preoperative CT scan to the intraoperative radiograph was calculated via LevelCheck, and projection distance errors were analyzed. Five hundred random initializations were performed for each patient, and algorithm settings (viz, the number of robust multistarts, ranging 50-200) were varied to evaluate the trade-off between registration error and computation time. Failure mode analysis was performed by individually analyzing unsuccessful registrations (>5 mm distance error) observed with 50 multistarts. At 200 robust multistarts (computation time of ∼26 s), the registration accuracy was 100% across all 10,000 trials. As the number of multistarts (and computation time) decreased, the registration remained fairly robust, down to 99.3% registration accuracy at 50 multistarts (computation time ∼7 s). The LevelCheck algorithm correctly identified target vertebrae in intraoperative mobile radiographs of the thoracolumbar spine, demonstrating acceptable computation time, compatibility with routinely obtained preoperative CT scans, and warranting investigation in prospective studies. N/A.

Self-Calibration of Cone-Beam CT Geometry Using 3D-2D Image Registration: Development and Application to Task-Based Imaging with a Robotic C-Arm.

Robotic C-arm systems are capable of general noncircular orbits whose trajectories can be driven by the particular imaging task. However obtaining accurate calibrations for reconstruction in such geometries can be a challenging problem. This work proposes a method to perform a unique geometric calibration of an arbitrary C-arm orbit by registering 2D projections to a previously acquired 3D image to determine the transformation parameters representing the system geometry. Experiments involved a cone-beam CT (CBCT) bench system, a robotic C-arm, and three phantoms. A robust 3D-2D registration process was used to compute the 9 degree of freedom (DOF) transformation between each projection and an existing 3D image by maximizing normalized gradient information with a digitally reconstructed radiograph (DRR) of the 3D volume. The quality of the resulting "self-calibration" was evaluated in terms of the agreement with an established calibration method using a BB phantom as well as image quality in the resulting CBCT reconstruction. The self-calibration yielded CBCT images without significant difference in spatial resolution from the standard ("true") calibration methods (p-value >0.05 for all three phantoms), and the differences between CBCT images reconstructed using the "self" and "true" calibration methods were on the order of 10-3 mm-1. Maximum error in magnification was 3.2%, and back-projection ray placement was within 0.5 mm. The proposed geometric "self" calibration provides a means for 3D imaging on general non-circular orbits in CBCT systems for which a geometric calibration is either not available or not reproducible. The method forms the basis of advanced "task-based" 3D imaging methods now in development for robotic C-arms.

Registration of 2D C-Arm and 3D CT Images for a C-Arm Image-Assisted Navigation System for Spinal Surgery.

C-Arm image-assisted surgical navigation system has been broadly applied to spinal surgery. However, accurate path planning on the C-Arm AP-view image is difficult. This research studies 2D-3D image registration methods to obtain the optimum transformation matrix between C-Arm and CT image frames. Through the transformation matrix, the surgical path planned on preoperative CT images can be transformed and displayed on the C-Arm images for surgical guidance. The positions of surgical instruments will also be displayed on both CT and C-Arm in the real time. Five similarity measure methods of 2D-3D image registration including Normalized Cross-Correlation, Gradient Correlation, Pattern Intensity, Gradient Difference Correlation, and Mutual Information combined with three optimization methods including Powell's method, Downhill simplex algorithm, and genetic algorithm are applied to evaluate their performance in converge range, efficiency, and accuracy. Experimental results show that the combination of Normalized Cross-Correlation measure method with Downhill simplex algorithm obtains maximum correlation and similarity in C-Arm and Digital Reconstructed Radiograph (DRR) images. Spine saw bones are used in the experiment to evaluate 2D-3D image registration accuracy. The average error in displacement is 0.22 mm. The success rate is approximately 90% and average registration time takes 16 seconds.

Pore scale imaging and modelling of coal properties

A key parameter in determining the productivity and commercial success of coal seam gas wells is the permeability of individual seams. Laboratory testing of core plugs is commonly used as an indicator of potential seam permeability. The highly heterogeneous and stress-dependent nature of coal makes laboratory measurements difficult to perform and the results difficult to interpret. Consequently, permeability in coal is poorly understood. The permeability of coal at the core scale depends on the geometry, topology, connectivity, mineralisation and spatial distribution of the cleat system, and a better understanding of coal permeability, that and the factors that control this depends on having a better understanding and detailed characterisation of the system. The authors used high resolution micro-focus X-ray computed tomography and 2D-3D image registration techniques to overlay tomograms of the same core plug, with and without X-ray attenuating fluids present in the pore space, with 2D scanning electron microscope images to produce detailed 3D visualisations of the geometry and topology of the cleat systems in the coal plugs. Novel filtering algorithms were used to produce segmentations that preserve cleat aperture dimensions and together with large-scale fluid flow simulations, they performed directly on the images and were used to compute porosities and permeabilities. Image resolution and segmentation sensitivity studies are also described, which show that the core scale permeability is controlled by a small number of well-connected percolating cleats. The results of this study demonstrate the potential of simple image-based analysis techniques to provide rapid estimates of core plug permeabilities.

Non-rigid registration of medical images based on estimation of deformation states

A unified framework for automatic non-rigid 3D–3D and 3D–2D registration of medical images with static and dynamic deformations is proposed in this paper. The problem of non-rigid image registration is approached as a classical state estimation problem using a generic deformation model for the soft tissue. The registration technique employs a dynamic linear elastic continuum mechanics model of the tissue deformation, which is discretized using the finite element method. In the proposed method, the registration is achieved through a Kalman-like filtering process, which incorporates information from the deformation model and a vector of observation prediction errors computed from an intensity-based similarity/distance metric between images. With this formulation, single and multiple-modality, 3D–3D and 3D–2D image registration problems can all be treated within the same framework. The performance of the proposed registration technique was evaluated in a number of different registration scenarios. First, 3D magnetic resonance (MR) images of uncompressed and compressed breast tissue were co-registered. 3D MR images of the uncompressed breast tissue were also registered to a sequence of simulated 2D interventional MR images of the compressed breast. Finally, the registration algorithm was employed to dynamically track a target sub-volume inside the breast tissue during the process of the biopsy needle insertion based on registering pre-insertion 3D MR images to a sequence of real-time simulated 2D interventional MR images. Registration results indicate that the proposed method can be effectively employed for the registration of medical images in image-guided procedures, such as breast biopsy in which the tissue undergoes static and dynamic deformations.

Interventional digital tomosynthesis from a standard fluoroscopy system using 2D-3D registration.

Interventional fluoroscopy provides guidance in a variety of minimally invasive procedures. However, three-dimensional (3D) clinically relevant information is projected onto a two-dimensional (2D) image which can make image interpretation difficult. Moreover, vasculature visualisation requires the use of iodinated contrast media which is nephrotoxic and is the primary cause of renal complications. In this article, we demonstrate how digital tomosynthesis slices can be produced on standard fluoroscopy equipment by registering the preoperative CT volume and the intraoperative fluoroscopy images using 2D-3D image registration. The proposed method automatically reconstructs patient-anatomy-specific slices and removes clutter resulting from bony anatomy. Such slices could provide additional intraoperative information which cannot be provided by the preoperative CT volume alone, such as the deformed aorta position offering improved guidance precision. Image acquisition would fit with interventional clinical work-flow and would not require a high X-ray dose. Experiments are carried out using one phantom and four clinical datasets. Phantom results showed a 3351% contrast-to-noise improvement compared to standard fluoroscopy. Patient results showed our method enabled visualization of clinically relevant features: outline of the aorta, the aortic bifurcation and some aortic calcifications.