3D Registration Methods Research Articles

A Registration Method Based on Planar Features Between BIM Model and Point Cloud

Abstract In the digitization process of the construction industry, it is frequently necessary to use BIM as a digital model carrier, and the registration between BIM and point cloud is a crucial step in BIM applications. Most existing 3D registration methods, such as the ICP algorithm, are capable of aligning point clouds. However, these classical methods are subject to the influence of numerous points contained within the point cloud, relying heavily on point-to-point correlations. Consequently, it is challenging to extend these algorithms to register with other forms of spatial information, such as BIM models, beyond the point cloud. Considering that planes are basic geometric elements in building BIM models, this paper proposes a novel method for aligning BIM models with point clouds by matching planar features. The method extracts planes from point clouds using an enhanced region growing algorithm, directly parsing the planar geometric information from BIM in IFC format. After completing the matching of plane groups, the optimal solution is calculated using a weighted least squares method. The experimental results indicate that the proposed method can achieve successful registration between the BIM model and point cloud, with a lower RMSE of 5.73mm compared to the registration method using RANSAC+ICP on the same dataset.

3D Data Processing and Entropy Reduction for Reconstruction from Low-Resolution Spatial Coordinate Clouds in a Technical Vision System.

This paper proposes an advancement in the application of a Technical Vision System (TVS), which integrates a laser scanning mechanism with a single light sensor to measure 3D spatial coordinates. In this application, the system is used to scan and digitalize objects using a rotating table to explore the potential of the system for 3D scanning at reduced resolutions. The experiments undertaken searched for optimal scanning windows and used statistical data filtering techniques and regression models to find a method to generate a 3D scan that was still recognizable with the least amount of 3D points, balancing the number of points scanned and time, while at the same time reducing effects caused by the particularities of the TVS, such as noise and entropy in the form of natural distortion in the resulting scans. The evaluation of the experimentation results uses 3D point registration methods, joining multiple faces from the original volume scanned by the TVS and aligning it to the ground truth model point clouds, which are based on a commercial 3D camera to verify that the reconstructed 3D model retains substantial detail from the original object. This research finds it is possible to reconstruct sufficiently detailed 3D models obtained from the TVS, which contain coarsely scanned data or scans that initially lack high definition or are too noisy.

Addressing the generalization of 3D registration methods with a featureless baseline and an unbiased benchmark

Recent 3D registration methods are mostly learning-based that either find correspondences in feature space and match them, or directly estimate the registration transformation from the given point cloud features. Therefore, these feature-based methods have difficulties with generalizing onto point clouds that differ substantially from their training data. This issue is not so apparent because of the problematic benchmark definitions that cannot provide any in-depth analysis and contain a bias toward similar data. Therefore, we propose a methodology to create a 3D registration benchmark, given a point cloud dataset, that provides a more informative evaluation of a method w.r.t. other benchmarks. Using this methodology, we create a novel FAUST-partial (FP) benchmark, based on the FAUST dataset, with several difficulty levels. The FP benchmark addresses the limitations of the current benchmarks: lack of data and parameter range variability, and allows to evaluate the strengths and weaknesses of a 3D registration method w.r.t. a single registration parameter. Using the new FP benchmark, we provide a thorough analysis of the current state-of-the-art methods and observe that the current method still struggle to generalize onto severely different out-of-sample data. Therefore, we propose a simple featureless traditional 3D registration baseline method based on the weighted cross-correlation between two given point clouds. Our method achieves strong results on current benchmarking datasets, outperforming most deep learning methods. Our source code is available on github.com/DavidBoja/exhaustive-grid-search.

Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning.

Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing approaches. The Learn2Reg challenge addresses these limitations by providing a multi-task medical image registration data set for comprehensive characterisation of deformable registration algorithms. A continuous evaluation will be possible at https:// learn2reg.grand-challenge.org. Learn2Reg covers a wide range of anatomies (brain, abdomen, and thorax), modalities (ultrasound, CT, MR), availability of annotations, as well as intra- and inter-patient registration evaluation. We established an easily accessible framework for training and validation of 3D registration methods, which enabled the compilation of results of over 65 individual method submissions from more than 20 unique teams. We used a complementary set of metrics, including robustness, accuracy, plausibility, and runtime, enabling unique insight into the current state-of-the-art of medical image registration. This paper describes datasets, tasks, evaluation methods and results of the challenge, as well as results of further analysis of transferability to new datasets, the importance of label supervision, and resulting bias. While no single approach worked best across all tasks, many methodological aspects could be identified that push the performance of medical image registration to new state-of-the-art performance. Furthermore, we demystified the common belief that conventional registration methods have to be much slower than deep-learning-based methods.

A unified framework for automated registration of point clouds, mesh surfaces and 3D models by using planar surfaces

AbstractRegistration of 3D spatial data and models is a fundamental task in applications such as mapping, positioning and virtual/augmented reality. Most of the existing 3D registration methods such as iterative closest point (ICP) and recent learning‐based methods are dedicated to point cloud registration, and rely heavily on point‐wise correspondences, which limits their ability to address registration problems across different data types. Since man‐made objects and buildings usually contain many planar surfaces, it is possible to use the planes for accurate registration of different data and models. In this paper, a unified registration framework is proposed consisting of a plane extraction module, which can extract planes from various forms of spatial data such as point clouds or surface‐based 3D models, and a registration module, which performs automatic registration based on the extracted planes. Tests show that the proposed method can handle small‐overlap registration across all these data types with high success rates. The result of point cloud registration also indicates that the method achieves better accuracy as compared to ICP.

Augmented reality registration algorithm based on T-AKAZE features

Three-dimensional (3D) registration plays a pivotal step in augmented reality (AR) systems. Traditional 3D registration methods have the disadvantages of poor accuracy and robustness. This paper proposes a novel registration method, we believe, for AR systems based on the AKAZE and Tanimoto similarity measurement method. In this paper, the image feature points are extracted and matched by combining the AKAZE algorithm and the Tanimoto similarity measurement method. Then, the camera pose is estimated by calculating the constraint relationship of the feature points. Finally, the 3D registration and real-time tracking of the virtual objects are realized by the Lucas-Kanade (LK) optical flow tracking algorithm. We use Tanimoto to determine the similarity of feature points to improve the matching accuracy of the AKAZE algorithm, and this method not only retains the advantages of strong scale adaptation but also has the advantage of high-precision matching. Experiments show that the method proposed in this paper has the benefits of high registration accuracy, low time consumption, and strong robustness. Under the premise of ensuring accuracy, when the marker is rotated or blocked, it can be accurately registered. In addition, when the external environment changes, for example, the light intensity or the size of the parallax, the registration can still be stable.

3D Registration of pre-surgical prostate MRI and histopathology images via super-resolution volume reconstruction.

The use of MRI for prostate cancer diagnosis and treatment is increasing rapidly. However, identifying the presence and extent of cancer on MRI remains challenging, leading to high variability in detection even among expert radiologists. Improvement in cancer detection on MRI is essential to reducing this variability and maximizing the clinical utility of MRI. To date, such improvement has been limited by the lack of accurately labeled MRI datasets. Data from patients who underwent radical prostatectomy enables the spatial alignment of digitized histopathology images of the resected prostate with corresponding pre-surgical MRI. This alignment facilitates the delineation of detailed cancer labels on MRI via the projection of cancer from histopathology images onto MRI. We introduce a framework that performs 3D registration of whole-mount histopathology images to pre-surgical MRI in three steps. First, we developed a novel multi-image super-resolution generative adversarial network (miSRGAN), which learns information useful for 3D registration by producing a reconstructed 3D MRI. Second, we trained the network to learn information between histopathology slices to facilitate the application of 3D registration methods. Third, we registered the reconstructed 3D histopathology volumes to the reconstructed 3D MRI, mapping the extent of cancer from histopathology images onto MRI without the need for slice-to-slice correspondence. When compared to interpolation methods, our super-resolution reconstruction resulted in the highest PSNR relative to clinical 3D MRI (32.15dB vs 30.16dB for BSpline interpolation). Moreover, the registration of 3D volumes reconstructed via super-resolution for both MRI and histopathology images showed the best alignment of cancer regions when compared to (1) the state-of-the-art RAPSODI approach, (2) volumes that were not reconstructed, or (3) volumes that were reconstructed using nearest neighbor, linear, or BSpline interpolations. The improved 3D alignment of histopathology images and MRI facilitates the projection of accurate cancer labels on MRI, allowing for the development of improved MRI interpretation schemes and machine learning models to automatically detect cancer on MRI.

An application independent review of multimodal 3D registration methods

Registration is a ubiquitous operation in Visual Computing, with applications in 3D object retrieval among others. Registration is the process of overlaying two or more datasets taken from different viewpoints, at different times or by different sensors into a common reference frame. Multimodal registration is a special case where the data to be matched do not belong to the same modality and is challenging due to the diverse nature of the modalities involved which makes the creation of a distance function harder. Due to the large number of possible modality combinations and application fields, a considerable number of multimodal registration techniques have been proposed in diverse fields, including medicine and archaeology. This survey aims to unify 3D multimodal registration techniques (i.e. where at least one of the modalities is in 3D) across application domains, with the hope of providing an application-independent view and the potential for cross-fertilization. The problem of 3D multimodal registration is explicitly defined and the various methods are systematically categorized and described in terms of a number of important properties. Methods with publicly available source code have been compared on common datasets. A discussion on trends, observations and challenges for further research concludes the review.

Zero-shot fMRI decoding with three-dimensional registration based on diffusion tensor imaging

Functional magnetic resonance imaging (fMRI) acquisitions include a great deal of individual variability. This individuality often generates obstacles to the efficient use of databanks from multiple subjects. Although recent studies have suggested that inter-regional connectivity reflects individuality, conventional three-dimensional (3D) registration methods that calibrate inter-subject variability are based on anatomical information about the gray matter shape (e.g., T1-weighted). Here, we present a new registration method focusing more on the white matter structure, which is directly related to the connectivity in the brain, and apply it to subject-transfer brain decoding. Our registration method based on diffusion tensor imaging (DTI) transferred functional maps of each individual to a common anatomical space, where a decoding analysis of multi-voxel patterns was performed. The decoder trained on functional maps from other individuals in the common space showed a transfer decoding accuracy comparable to that of an individual decoder trained on single-subject functional maps. The DTI-based registration allowed more precise transformation of gray matter boundaries than a well-established T1-based method. These results suggest that the DTI-based registration is a promising tool for standardization of the brain functions, and moreover, will allow us to perform ‘zero-shot’ learning of decoders which is profitable in brain machine interface scenes.

3D registration of intravascular optical coherence tomography and cryo-image volumes for microscopic-resolution validation.

High resolution, 100 frames/sec intravascular optical coherence tomography (IVOCT) can distinguish plaque types, but further validation is needed, especially for automated plaque characterization. We developed experimental and 3D registration methods, to provide validation of IVOCT pullback volumes using microscopic, brightfield and fluorescent cryo-image volumes, with optional, exactly registered cryo-histology. The innovation was a method to match an IVOCT pull-back images, acquired in the catheter reference frame, to a true 3D cryo-image volume. Briefly, an 11-parameter, polynomial virtual catheter was initialized within the cryo-image volume, and perpendicular images were extracted, mimicking IVOCT image acquisition. Virtual catheter parameters were optimized to maximize cryo and IVOCT lumen overlap. Local minima were possible, but when we started within reasonable ranges, every one of 24 digital phantom cases converged to a good solution with a registration error of only +1.34±2.65μm (signed distance). Registration was applied to 10 ex-vivo cadaver coronary arteries (LADs), resulting in 10 registered cryo and IVOCT volumes yielding a total of 421 registered 2D-image pairs. Image overlays demonstrated high continuity between vascular and plaque features. Bland-Altman analysis comparing cryo and IVOCT lumen area, showed mean and standard deviation of differences as 0.01±0.43 mm2. DICE coefficients were 0.91±0.04. Finally, visual assessment on 20 representative cases with easily identifiable features suggested registration accuracy within one frame of IVOCT (±200μm), eliminating significant misinterpretations introduced by 1mm errors in the literature. The method will provide 3D data for training of IVOCT plaque algorithms and can be used for validation of other intravascular imaging modalities.

GPGPU computation in mobile robot applications

The paper concerns the results related with GPGPU computing applied for mobile robotics applications. The scalable implementation of the point to point and point to plane 3D data registration methods with an improvement based on regular grid decomposition is shown. 3D data is delivered by mobile robot equipped with 3D laser measurement system for INDOOR environments. Presented empirical analysis of the implementation shows the On-Line computation capability using modern graphic processor unit NVIDIA GF 580. In the paper the discussion concerning the comparison between these two methods is given. It will be shown why the point to plain ICP implementation can achieve better performance than the point to point approach. We show parallel vector computation that is used for semantic objects identifications and for loop closing detection.

WE‐E‐BRC‐11: Towards Subject Specific Head and Neck Biomechanical Models Using Multi‐Level Multi‐Resolution and Inverse Consistent Registration Algorithms for Radiotherapy

Purpose: The proposed work focuses on addressing the impact with which the patientˈs treatment delivery can be monitored and documented in real time taking into account the interfraction and intrafraction motion and effective optimizations for the treatment can be conceived and implemented. The first step towards establishing the radiation delivered to the head and neck tumor and the surrounding tissues in real‐time is the development of subject specific biomechanical head and neck models. The complexity in developing such a biomechanical model arises from the human musculoskeletal system, which includes approximately 57 articular bones and many more muscle actuators and their closed kinematic loops. Methods: We present a subject specific model, which is developed using cancer patientsˈ 3D CT scans (simulation CT, cone‐beam CT), a validated generic biomechanical model and registration algorithms that incorporate biomechanical properties from the generic model into the static patient anatomy to facilitate subject‐specific biomechanical model. For illustration purposes, we take as input a subject specific anatomy from a 3DCT acquired during the simulation stage. For analysis purposes, free‐form deformation (FFD), demons, the symmetric force demons, and optical flow (OF) registration methods are used within the multi‐level multi‐resolution (MLMR) and additional Inverse Consistency (MLMRIC) for comparing the registration accuracy. Once registered, the biomechanical generic model is warped using the registration results to closely fit the patient anatomy thereby facilitating a patient specific biomechanical model. Results: Validation studies showed that among the eight 3D registration methods, the MLMR symmetric demons registration is quantitatively more accurate as compared to the other methods and facilitates the development of a more appropriate subject specific biomechanical head and neck model. Conclusions: The development of subject specific biomechanical models is made possible by MLMR symmetric demons registration of generic validated biomechanical models to subject anatomy.

SU-GG-T-11: Dosimetric Impacts of Rotational Setup Errors on Head & Neck IMRT Treatments

Purpose: To investigate the dosimetric impact of the residual rotational setup errors on head&neck (HNC) IMRT with conventional clinical three translational setup corrections, and to evaluate the necessity of the full six degree (6D) setup correction. Methods and Materials: A total of 66 kV CBCTimage sets were acquired on the first day of treatment and weekly thereafter for 10 HNC patients. The CBCTimages were registered with the planning CTimages using two 3D rigid registration methods. Method1 determines the translational errors only and method2 determines 6D errors. To simulate the clinical setup correction, the 6D setup errors were simulated in the treatment planning system and then were corrected by using the corresponding translational data determined by method1. For each patient, dose distributions for 6 to 7 fractions with various setup uncertainties were generated, and plan sum was created to simulate the total dose distribution through an entire course and was compared with the original treatment plan. The first phase plan with prescription about 50 Gy was used in this study. Results: Without rotational corrections and with only translational corrections, for PTV, the plan sum showed that the average Vp (prescription dose coverage) was decreased by 5.9%, D95 (dose that covers 95% of target) was decreased 1.8 Gy. For CTV, though the Vp could be decreased by 8% for treatments on selected days, the plan sum showed that the average Vp was decreased by 1.7% and D98 was decreased 0.1 Gy. The average dose increase to spinal cord was 0.7 Gy. Conclusions: With conventional translational setup correction, the overall CTV prescription dose coverage decreases insignificantly because of the 5mm margin from CTV to PTV. The dose increase to spinal cord is not significant. So for HNC IMRT treatment with target margin, the six degree setup correction is not necessary.

Using ITK platform for medical image registration

Within image processing bounds, registration is a fundamental and very important tool when performing diagnosis and/or treatment of different diseases. It becomes necessary to carefully assess the performance of different available methods for each situation, which may be of great numerical complexity. In this work, some known 2D and 3D registration methods are examined (based on mutual information, least squares, B-splines, finite-elements) to be employed for both rigid and elastic registration, making use of ITK libraries and classes in a C++ environment. Our preliminary results suggest that those based on mutual information are most appropriate for brain image registration. Regarding non-rigid or elastic registration, the best results were achieved using B-splines as interpolator. The application of finite elements shows poorer performance and requires further research.

SU‐FF‐J‐35: Clinical Applications of 3D and 4D Deformable Image Registration for Image Guided Radiotherapy

Purpose: To develop a framework of 3D and 4D deformable image registration and investigate its applications in image guided radiotherapy (IGRT). Method and Materials: We have developed 3D and 4D deformable image registration methods to register fan beam CT (FBCT) and cone beam CT (CBCT) images. Applications of the 3D and 4D deformable registration include: (1) Inter‐subject atlas‐based image segmentation to automatically contour normal tissue regions of interest (ROI) in treatment planning CT images; (2) Intra‐subject atlas‐based image segmentation to delineate daily ROIs by registering daily FBCT or CBCT images to treatment planning CT images. This method is an ideal ROI delineation method for online plan adjustments; (3) 4D deformable image registration to extract average voxel trajectories for real‐time tumor tracking and for 4D tomotherapy treatment planning; (4) Deformable fusion of 4D‐PET images, which is based on deformable registration of 4D‐CT images, to minimize motion blurring in PET images without increase of committed doses or data acquisition time. Results: The deformable image registration system was tested using clinical images. Registrations of inter‐ and intra‐subject images show acceptable results. The corresponding results are reported in our supporting document. Quantitative validation of intra‐subject atlas‐based image segmentation showed agreements with physicians' manual contours. 4D registration yields voxel trajectories smooth in both spatial and temporal spaces. Problems were noticed in the cases where the registration image pairs are intrinsically different, such as gas bubbles in rectums, or deformations are extremely large, such as empty vs. full bladders. Conclusions: We have developed a general framework of applying 3D & 4D deformable image registration in image guided radiotherapy. Our clinical applications showed the positive and convincing results.

Evaluation of a 3D object registration method for analysis of humeral kinematics

In 3D image-based studies of joint kinematics, 3D registration methods should be automatic, insensitive to segmentation inconsistencies and use coordinate systems that have clinically relevant orientations and locations because this is important for analyzing rotation angles and translation directions. We developed and evaluated a registration method, which is based on the cylindrical geometry of the humerus shaft and an analysis of the inertia moments of the humerus head, in order to consistently and automatically orient the humerus coordinate system according to its anatomy. Registration techniques must be thoroughly evaluated. In this study we used a well-detectable marker as reference, from which coordinate system determination errors of a 3D object could be measured. This allowed us to quantify by means of unique error analysis the translational and rotational errors in terms of how much and about/along which humeral axis errors occurred. The evaluation experiments were performed using virtual rotations of 3D humeral binary image, a humerus model and a 3D image of a volunteer's shoulder. They indicated that the humeral coordinate system determination errors primarily originated from segmentation inconsistencies, which influenced mostly the humeral transverse axes orientation. The error analysis revealed that the developed registration method reduced the effect of manual segmentation inconsistencies on the orientation of the humeral transverse axes up to 37%, in comparison to the commonly used inertia registration.

Rotational knee laxity measurement device: reliability assessment in a cadaver model

In 3D image-based studies of joint kinematics, 3D registration methods should be automatic, insensitive to segmentation inconsistencies and use coordinate systems that have clinically relevant orientations and locations because this is important for analyzing rotation angles and translation directions. We developed and evaluated a registration method, which is based on the cylindrical geometry of the humerus shaft and an analysis of the inertia moments of the humerus head, in order to consistently and automatically orient the humerus coordinate system according to its anatomy.Registration techniques must be thoroughly evaluated. In this study we used a well-detectable marker as reference, from which coordinate system determination errors of a 3D object could be measured. This allowed us to quantify by means of unique error analysis the translational and rotational errors in terms of how much and about/along which humeral axis errors occurred. The evaluation experiments were performed using virtual rotations of 3D humeral binary image, a humerus model and a 3D image of a volunteer's shoulder. They indicated that the humeral coordinate system determination errors primarily originated from segmentation inconsistencies, which influenced mostly the humeral transverse axes orientation. The error analysis revealed that the developed registration method reduced the effect of manual segmentation inconsistencies on the orientation of the humeral transverse axes up to 37%, in comparison to the commonly used inertia registration.

A novel four-dimensional image registration method for radionuclide therapy dosimetry

A novel method for registering sequential SPECT scans (4DRRT) is described, whereby all sequential scans acquired in the course of a therapy or a pre-therapy tracer study may be registered in one pass. The method assumes that a mono-exponential decay function can be fitted to the series of sequential SPECT scans. Multiple volumes, presenting with different decay rates, are fitted with different mono-exponential functions. The MSSE (mean sum of squared errors in the least-squares fit algorithm), over the volume used for registration, is the cost function minimized at registration. Simulated data were used to assess the effect of thresholding, smoothing, noise and the multi-exponential nature of the four-dimensional (4D) SPECT studies on the performance of 4DRRT, resulting in three-dimensional (3D) residual registration errors <3.5 mm. The 4DRRT method was then compared to the following 3D registration methods: the correlation coefficient, the sum of absolute differences, the variance of image ratios and the mutual information. The comparisons, using both simulated and clinical data, were based on the standard deviation of the effective decay time distribution, generated from the registered 4D dataset, and showed that image registration using 4DRRT is simpler and more robust compared to the 3D techniques, especially when multiple tumour sites with different decay rates are present.