Image-based Registration Method Research Articles

3D Image-Guided Robotic System for Bone Fracture Reduction

To overcome possible drawbacks of bone fracture reduction such as risks of malrotation, radiation exposure, as well as laborious traction, image-guided robotic surgery systems have been proposed. However, the use of optical tracking systems (OTS) creates inherent line-of-sight problems that cause frequent interruptions during surgery. We propose an OTS-free image-guided bone fracture reduction system utilizing a Stewart robot platform to solve the problem of conventional OTS-based robotic systems. The system applies inverse kinematics to compute the relative positions between the broken bone fragments. Each fragment is pre-operatively registered in the robotic system with a semi-automatic image-based registration method using an attachable jig designed for the proposed platform. This approach is particularly effective when the image features are not clearly detected in the fluoroscopic images. The accuracy of the proposed system was evaluated via pre- and post-operative computed tomography (CT) scans of femoral phantoms. Ex-vivo experiments were also performed on caprine legs to assess the clinical feasibility. In the phantom and ex-vivo experiments, the mean rotational errors of the reduction were 1.79° and 1.76°, respectively. The mean time for the reduction was approximately 3 min. This letter proposes a new method to compute the relative positions between the bone fragments using inverse kinematics and semi-automatic robot-patient registration without requiring a C-arm and an OTS. The OTS-free robotic surgery system has the potential advantages to enhance the accuracy of fracture reduction and reduce the surgery time and radiation exposure.

A novel bone registration method using impression molding and structured-light 3D scanning technology.

Accurate bone registration is critical for computer navigation and robotic surgery. Existing registration systems are expensive, cumbersome, limited in accuracy and/or require intraoperative radiation. We recently reported a novel method of registration utilizing an inexpensive, compact, and X-ray-free structured-light 3D scanner. However, this technique is not always practical in a real surgical setting where soft tissue and blood can obstruct the continuous line-of-sight required for structured-light technology. We sought to remedy these limitations using a novel technique using rapid-setting impression molding to capture bone surface features and scan the undersurface of the mold with a structured-light scanner. The photonegative of this mold is compared to the preoperative computed tomography (CT)-scan to register the bone. A registration accuracy study was conducted on 36 CT-scanned femur sawbones, simulating typical exposure in hip/knee arthroplasty and bone tumor surgery. A cadaver experiment was also conducted to evaluate the feasibility of using the impression molding in a more realistic operating room setting. The registration accuracy of the proposed technique was 0.50 ± 0.19 mm. This was close to the reported accuracy of 0.43 ± 0.18 mm using a structured-light scanner without impression molding (p = 0.085). In comparison, historical values for "paired-point" and intraoperative CT image-based registration methods currently used in modern robotic/computer-navigation systems were 0.68 ± 0.14 mm (p = 0.004) and 0.86 ± 0.38 mm, respectively. The registration accuracy of the cadaver experiment was consistent with that of sawbone experiments. Although future studies are needed to extend to human subjects, this study shows that the impression molding method can produce comparable or better registration accuracy than the existing techniques.

Fully convolutional network-based registration for augmented assembly systems

Image-based registration methods have been widely used in augmented assembly systems. However, many challenges remain to be solved, such as low robustness and poor timeliness during registration. This paper presents a deep learning approach for registration. To reduce the workload of data collection, an automatic picture generation method is offered for deep learning algorithm, and a dataset is built for detecting the keypoints of assembly objects. We propose a fully convolutional network (FCN) model for detecting keypoints from a single RGB image. The FCN model, which consists of 13 layers, can accurately detect the location of keypoints with a low computational resource consumption. The detected keypoints are used to solve the camera position and orientation for augmented assembly registration. The experiments demonstrate that our method is robust against different rotation angles of the assembly object and against background interference. The detection accuracy is high under different camera motion blurs; to be specific, the pixel error in different directions of 640 × 480 images was only 0.9 pixels. The FCN-based registration approach was shown to be fast during augmented assembly experiments, achieving up to 30 FPS.

Cost-function testing methodology for image-based registration of endoscopy to CT images in the head and neck

One of the largest geometric uncertainties in designing radiotherapy treatment plans for squamous cell cancers of the head and neck is contouring the gross tumor volume. We have previously described a method of projecting mucosal disease contours, visible on endoscopy, to volumetrically reconstructed planning computed tomography (CT) datasets, using electromagnetic (EM) tracking of a flexible endoscope, enabling rigid registration between endoscopic and CT images.However, to achieve better accuracy for radiotherapy planning, we propose refining this initial registration with image-based registration methods. In this paper, several types of cost functions are evaluated based on accuracy and robustness. Three phantoms and eight clinical cases are used to test each cost function, with initial registration of endoscopy to CT provided by the pose of the flexible endoscope recovered from EM tracking. Cost function classes include: cross correlation, mutual information and gradient methods. For each test case, a ground truth virtual camera pose was first defined by manual registration of anatomical features visible in both real and virtual endoscope images. A new set of evenly spaced fiducial points and a sample contour were created and projected onto the CT image to be used in assessing image registration quality. A new set of 5000 displaced poses was generated by random sampling displacements along each translational and rotational dimension. At each pose, fiducial and contour points in the real image were again projected on the CT image. The cost function, fiducial registration error and contouring error values were then calculated.While all cost functions performed well in select cases, only the normalized gradient field function consistently had registration errors less than 2 mm, which is the accuracy needed if this application of registering mucosal disease identified on optical image to CT images is to be used in the clinical practice of radiation treatment planning.(Registration: ClinicalTrials.gov NCT02704169)

BAND TO BAND REGISTRATION OF MULTI-SPECTRAL AERIAL IMAGERY – RELIEF DISPLACEMENT AND MISS-REGISTRATION ERROR

Abstract. With the development of UAVs in agricultural applications, multi-spectral cameras are designed to be installed on these platforms in small size and weight to help users for applications. These cameras are designed in multi lenses structure each lens considered for each spectral band. Band miss-registration of low altitude multispectral images acquired via UAVs is challenging due to multi view image sensors and optics structure of the multispectral cameras. In this study, an image based registration method is proposed to reduce the effect of relief displacement on miss-registration error. For this purpose, three different data sets with different level of relief displacement are considered and three different strategies are proposed. In the proposed method, by dividing the image into patches, selecting the appropriate local window, and local matching in each window, an attempt has been made to obtain the proper distribution of points in the image. Also, by limiting the search space, the matching speed is also increased. To evaluate the proposed method, the comparison with the ENVI and ArcGIS software has been made where results clearly present the superiority of the proposed method by average of 0.28 pixel in accuracy.

Real-time, image-based slice-to-volume registration for ultrasound-guided spinal intervention

Real-time fusion of magnetic resonance (MR) and ultrasound (US) images could facilitate safe and accurate needle placement in spinal interventions. We develop an entirely image-based registration method (independent of or complementary to surgical trackers) that includes an efficient US probe pose initialization algorithm. The registration enables the simultaneous display of 2D ultrasound image slices relative to 3D pre-procedure MR images for navigation.A dictionary-based 3D–2D pose initialization algorithm was developed in which likely probe positions are predefined in a dictionary with feature encoding by Haar wavelet filters. Feature vectors representing the 2D US image are computed by scaling and translating multiple Haar basis filters to capture scale, location, and relative intensity patterns of distinct anatomical features. Following pose initialization, fast 3D–2D registration was performed by optimizing normalized cross-correlation between intra- and pre-procedure images using Powell’s method. Experiments were performed using a lumbar puncture phantom and a fresh cadaver specimen presenting realistic image quality in spinal US imaging. Accuracy was quantified by comparing registration transforms to ground truth motion imparted by a computer-controlled motion system and calculating target registration error (TRE) in anatomical landmarks.Initialization using a 315-length feature vector yielded median translation accuracy of 2.7 mm (3.4 mm interquartile range, IQR) in the phantom and 2.1 mm (2.5 mm IQR) in the cadaver. By comparison, storing the entire image set in the dictionary and optimizing correlation yielded a comparable median accuracy of 2.1 mm (2.8 mm IQR) in the phantom and 2.9 mm (3.5 mm IQR) in the cadaver. However, the dictionary-based method reduced memory requirements by 47× compared to storing the entire image set. The overall 3D error after registration measured using 3D landmarks was 3.2 mm (1.8 mm IQR) mm in the phantom and 3.0 mm (2.3 mm IQR) mm in the cadaver. The system was implemented in a 3D Slicer interface to facilitate translation to clinical studies.Haar feature based initialization provided accuracy and robustness at a level that was sufficient for real-time registration using an entirely image-based method for ultrasound navigation. Such an approach could improve the accuracy and safety of spinal interventions in broad utilization, since it is entirely software-based and can operate free from the cost and workflow requirements of surgical trackers.

A ROBUST REGISTRATION ALGORITHM FOR POINT CLOUDS FROM UAV IMAGES FOR CHANGE DETECTION

Landslides are among the major threats to urban landscape and manmade infrastructure. They often cause economic losses, property damages, and loss of lives. Temporal monitoring data of landslides from different epochs empowers the evaluation of landslide progression. Alignment of overlapping surfaces from two or more epochs is crucial for the proper analysis of landslide dynamics. The traditional methods for point-cloud-based landslide monitoring rely on using a variation of the Iterative Closest Point (ICP) registration procedure to align any reconstructed surfaces from different epochs to a common reference frame. However, sometimes the ICP-based registration can fail or may not provide sufficient accuracy. For example, point clouds from different epochs might fit to local minima due to lack of geometrical variability within the data. Also, manual interaction is required to exclude any non-stable areas from the registration process. In this paper, a robust image-based registration method is introduced for the simultaneous evaluation of all registration parameters. This includes the Interior Orientation Parameters (IOPs) of the camera and the Exterior Orientation Parameters (EOPs) of the involved images from all available observation epochs via a bundle block adjustment with self-calibration. Next, a semi-global dense matching technique is implemented to generate dense 3D point clouds for each epoch using the images captured in a particular epoch separately. The normal distances between any two consecutive point clouds can then be readily computed, because the point clouds are already effectively co-registered. A low-cost DJI Phantom II Unmanned Aerial Vehicle (UAV) was customised and used in this research for temporal data collection over an active soil creep area in Lethbridge, Alberta, Canada. The customisation included adding a GPS logger and a Large-Field-Of-View (LFOV) action camera which facilitated capturing high-resolution geo-tagged images in two epochs over the period of one year (i.e., May 2014 and May 2015). Note that due to the coarse accuracy of the on-board GPS receiver (e.g., +/- 5-10 m) the geo-tagged positions of the images were only used as initial values in the bundle block adjustment. Normal distances, signifying detected changes, varying from 20 cm to 4 m were identified between the two epochs. The accuracy of the co-registered surfaces was estimated by comparing non-active patches within the monitored area of interest. Since these non-active sub-areas are stationary, the computed normal distances should theoretically be close to zero. The quality control of the registration results showed that the average normal distance was approximately 4 cm, which is within the noise level of the reconstructed surfaces.

A ROBUST REGISTRATION ALGORITHM FOR POINT CLOUDS FROM UAV IMAGES FOR CHANGE DETECTION

Abstract. Landslides are among the major threats to urban landscape and manmade infrastructure. They often cause economic losses, property damages, and loss of lives. Temporal monitoring data of landslides from different epochs empowers the evaluation of landslide progression. Alignment of overlapping surfaces from two or more epochs is crucial for the proper analysis of landslide dynamics. The traditional methods for point-cloud-based landslide monitoring rely on using a variation of the Iterative Closest Point (ICP) registration procedure to align any reconstructed surfaces from different epochs to a common reference frame. However, sometimes the ICP-based registration can fail or may not provide sufficient accuracy. For example, point clouds from different epochs might fit to local minima due to lack of geometrical variability within the data. Also, manual interaction is required to exclude any non-stable areas from the registration process. In this paper, a robust image-based registration method is introduced for the simultaneous evaluation of all registration parameters. This includes the Interior Orientation Parameters (IOPs) of the camera and the Exterior Orientation Parameters (EOPs) of the involved images from all available observation epochs via a bundle block adjustment with self-calibration. Next, a semi-global dense matching technique is implemented to generate dense 3D point clouds for each epoch using the images captured in a particular epoch separately. The normal distances between any two consecutive point clouds can then be readily computed, because the point clouds are already effectively co-registered. A low-cost DJI Phantom II Unmanned Aerial Vehicle (UAV) was customised and used in this research for temporal data collection over an active soil creep area in Lethbridge, Alberta, Canada. The customisation included adding a GPS logger and a Large-Field-Of-View (LFOV) action camera which facilitated capturing high-resolution geo-tagged images in two epochs over the period of one year (i.e., May 2014 and May 2015). Note that due to the coarse accuracy of the on-board GPS receiver (e.g., +/- 5-10 m) the geo-tagged positions of the images were only used as initial values in the bundle block adjustment. Normal distances, signifying detected changes, varying from 20 cm to 4 m were identified between the two epochs. The accuracy of the co-registered surfaces was estimated by comparing non-active patches within the monitored area of interest. Since these non-active sub-areas are stationary, the computed normal distances should theoretically be close to zero. The quality control of the registration results showed that the average normal distance was approximately 4 cm, which is within the noise level of the reconstructed surfaces.

An enhanced method for registration of dental surfaces partially scanned by a 3D dental laser scanning

In this paper, we propose the fast and accurate registration method of partially scanned dental surfaces in a 3D dental laser scanning. To overcome the multiple point correspondence problems of conventional surface registration methods, we propose the novel depth map-based registration method to register 3D surface models. First, we convert a partially scanned 3D dental surface into a 2D image by generating the 2D depth map image of the surface model by applying a 3D rigid transformation into this model. Then, the image-based registration method using 2D depth map images accurately estimates the initial transformation between two consequently acquired surface models. To further increase the computational efficiency, we decompose the 3D rigid transformation into out-of-plane (i.e. x-, y-rotation, and z-translation) and in-plane (i.e. x-, y-translation, and z-rotation) transformations. For the in-plane transformation, we accelerate the transformation process by transforming the 2D depth map image instead of transforming the 3D surface model. For the more accurate registration of 3D surface models, we enhance iterative closest point (ICP) method for the subsequent fine registration. Our initial depth map-based registration well aligns each surface model. Therefore, our subsequent ICP method can accurately register two surface models since it is highly probable that the closest point pairs are the exact corresponding point pairs. The experimental results demonstrated that our method accurately registered partially scanned dental surfaces. Regarding the computational performance, our method delivered about 1.5 times faster registration than the conventional method. Our method can be successfully applied to the accurate reconstruction of 3D dental objects for orthodontic and prosthodontic treatment.

Computer assisted determination of acetabular cup orientation using 2D–3D image registration

2D-3D image-based registration methods have been developed to measure acetabular cup orientation after total hip arthroplasty (THA). These methods require registration of both the prosthesis and the CT images to 2D radiographs and compute implant position with respect to a reference. The application of these methods is limited in clinical practice due to two limitations: (1) the requirement of a computer-aided design (CAD) model of the prosthesis, which may be unavailable due to the proprietary concerns of the manufacturer, and (2) the requirement of either multiple radiographs or radiograph-specific calibration, usually unavailable for retrospective studies. In this paper, we propose a new method to address these limitations. A new formulation for determination of post-operative cup orientation, which couples a radiographic measurement with 2D-3D image matching, was developed. In our formulation, the radiographic measurement can be obtained with known methods so that the challenge lies in the 2D-3D image matching. To solve this problem, a hybrid 2D-3D registration scheme combining a landmark-to-ray 2D-3D alignment with a robust intensity-based 2D-3D registration was used. The hybrid 2D-3D registration scheme allows computing both the post-operative cup orientation with respect to an anatomical reference and the pelvic tilt and rotation with respect to the X-ray imaging table/plate. The method was validated using 2D adult cadaver hips. Using the hybrid 2D-3D registration scheme, our method showed a mean accuracy of 1.0 +/- 0.7 degrees (range from 0.1 degrees to 2.0 degrees) for inclination and 1.7 +/- 1.2 degrees (range from 0.0 degrees to 3.9 degrees) for anteversion, taking the measurements from post-operative CT images as ground truths. Our new solution formulation and the hybrid 2D-3D registration scheme facilitate estimation of post-operative cup orientation and measurement of pelvic tilt and rotation.

A quantitative comparison of two methods to correct eddy current-induced distortions in DT-MRI

Eddy current-induced geometric distortions of single-shot, diffusion-weighted, echo-planar (DW-EP) images are a major confounding factor to the accurate determination of water diffusion parameters in diffusion tensor MRI (DT-MRI). Previously, it has been suggested that these geometric distortions can be removed from brain DW-EP images using affine transformations determined from phantom calibration experiments using iterative cross-correlation (ICC). Since this approach was first described, a number of image-based registration methods have become available that can also correct eddy current-induced distortions in DW-EP images. However, as yet no study has investigated whether separate eddy current calibration or image-based registration provides the most accurate way of removing these artefacts from DT-MRI data. Here we compare how ICC phantom calibration and affine FLIRT (http://www.fmrib.ox.ac.uk), a popular image-based multi-modal registration method that can correct both eddy current-induced distortions and bulk subject motion, perform when registering DW-EP images acquired with different slice thicknesses (2.8 and 5 mm) and b-values (1000 and 3000 s/mm(2)). With the use of consistency testing, it was found that ICC was a more robust algorithm for correcting eddy current-induced distortions than affine FLIRT, especially at high b-value and small slice thickness. In addition, principal component analysis demonstrated that the combination of ICC phantom calibration (to remove eddy current-induced distortions) with rigid body FLIRT (to remove bulk subject motion) provided a more accurate registration of DT-MRI data than that achieved by affine FLIRT.