Conventional Registration Methods Research Articles

A fast object registration method for augmented reality assembly with simultaneous determination of multiple 2D-3D correspondences

Abstract A fast and robust object registration is one of the fundamental steps of an augmented reality (AR) assembly guidance system. To manage the registration of texture-less objects from monocular images, this paper presents a three-dimensional (3D) object registration solution in the view-based framework, which incorporates a new image feature named as chain-of-lines feature (COLF). The COLF, constructed by several directed line segments with geometric constraints, enhances the robustness of the registration by establishing multiple correspondences between the two-dimensional (2D) image and 3D model simultaneously. In order to save on computational cost and improve the efficiency of the COLF-based registration, a compression algorithm based on the local breadth-first search (LBFS) strategy is also proposed; this adaptively reduces the number of keyframes according to the target model. In the experimental evaluation, the combination of COLF and LBFS-based keyframe compression shows higher registration success rate and faster speed in comparison with conventional registration methods. Moreover, an AR assembly guidance prototype system for reductors is introduced to illustrate the application of the proposed registration method.

Memory-efficient 2.5D convolutional transformer networks for multi-modal deformable registration with weak label supervision applied to whole-heart CT and MRI scans.

PURPOSE: Despite its potential for improvements through supervision, deep learning-based registration approaches are difficult to train for large deformations in 3D scans due to excessive memory requirements. METHODS: We propose a new 2.5D convolutional transformer architecture that enables us to learn a memory-efficient weakly supervised deep learning model for multi-modal image registration. Furthermore, we firstly integrate a volume change control term into the loss function of a deep learning-based registration method to penalize occurring foldings inside the deformation field. RESULTS: Our approach succeeds at learning large deformations across multi-modal images. We evaluate our approach on 100 pair-wise registrations of CT and MRI whole-heart scans and demonstrate considerably higher Dice Scores (of 0.74) compared to a state-of-the-art unsupervised discrete registration framework (deeds with Dice of 0.71). CONCLUSION: Our proposed memory-efficient registration method performs better than state-of-the-art conventional registration methods. By using a volume change control term in the loss function, the number of occurring foldings can be considerably reduced on new registration cases.

Dynamic inspection of a rail profile under affine distortion based on the reweighted-scaling iterative closest point method

The precision and efficiency of rail inspection are of vital importance to the operation and safety of railway systems. However, because of vibrations that occur in the rail inspection vehicle and installation error of the line laser sensor, the collected rail profile is distorted. As a result, the accuracy and robustness of conventional registration and inspection methods cannot be guaranteed. In this paper, an accurate and robust calibration method for measuring rail profile distortion is proposed. In this study, the impact of vibration on the rail profile, which was obtained with a laser sensor, was analyzed. Then, the collected rail profile was divided into two segments according to the actual condition where only the upper part of the rail was subjected to wear. An accurate and robust reweighted-scaling iterative closest point algorithm was proposed to calibrate the distorted rail profile, and the upper and lower boundaries of the scale ratio were used as constraints to solve the registration problem effectively and efficiently. Finally, the Hausdorff distance method was introduced to clearly visualize the rectified rail profile and wear. The experimental results demonstrated that the proposed method can realize the affine transformation of the distorted rail profile with a high precision and robustness. Additionally, it can achieve an online dynamic calibration and inspection of the rail by comparing the data with the conventional iterative closest point, scaling iterative closest point, and Calipri.

Adversarial learning for deformable registration of brain MR image using a multi-scale fully convolutional network

Background and objectiveDeformable registration is very significant for various clinical image applications. Unfortunately, existing conventional medical image registration approaches, which involve time-consuming iterative optimization, have not reached the level of routine clinical practice in terms of registration time and robustness. The aim of this study is to propose a tuning-free 3D image registration model based on adversarial deep network, and to achieve rapid and high-accurate registration. MethodsWe propose a fully convolutional network (FCN) to regress the 3D dense deformation field in one shot from the to-be-registered image pair. To precisely regress the complex deformation and produce optimal registration, we design the FCN as a novel multi-scale frame to capture the complementary multi-scale image features and effectively characterize the spatial correspondence between the image pair. Moreover, we learn a discriminator network simultaneously to discriminate the registered two images, where the discrimination loss helps further update the FCN. Thus by the adversarial training strategy, the registration network is urged to produce well-registered two images that are indistinguishable for the discriminator. ResultsWe perform registration experiments on four different brain MR datasets using the model trained by ANDI database. Compared with some state-of-the-art registration algorithms including other newest deeplearning-based methods, the proposed method provides a considerable increase of large than 4% in terms of Dice similarity coefficient (DSC). Moreover, our model also obtains comparable distance errors. More significantly, our model can achieve a high-accurate 3D registration result in average 0.74 s, with roughly hundred speed-up over conventional registration methods. ConclusionsThe proposed model shows consistent high performance for various registration tasks under a second without any additional parameter tuning, which proves its potential for real-time clinical applications.

A Novel Noninvasive Patient-Specific Navigation Method for Orbital Reconstructive Surgery: A Phantom Study Using Patient Data.

The correction of orbital deformities is an ongoing challenge in maxillofacial surgery. Computer-assisted navigation can improve surgical outcomes. However, conventional registration methods for navigation are not appropriate for orbital reconstructive surgery. This study proposes an accurate, noninvasive, patient-specific navigation method and demonstrates its feasibility. A noninvasive, patient-specific registration frame based on the external auditory canals and upper front teeth was designed using software developed in-house. A three-dimensional craniofacial model was segmented from patient computed tomographic data for the registration frame. A customized craniofacial phantom was also made using this three-dimensional model, with 20 embedded target points on the orbital model and 21 landmark points on the reference standard model. The proposed method was compared with two conventional registration methods: the dental splint-based method and the invasive marker frame-based method. Twenty trials were conducted for evaluation. Target registration error and surface registration error were computed to measure accuracy. The proposed method showed a target registration error of 1.05 ± 0.52 mm, with greater accuracy than conventional methods (dental splint, 2.10 ± 0.63 mm; invasive marker frame, 1.22 ± 0.46 mm). The proposed method yielded the best results for surface registration error, with 0.38 mm of deviation (dental splint, 0.82 mm; invasive marker frame, 0.60 mm). The proposed noninvasive patient-specific registration method demonstrated superior results for both target registration error and surface registration error compared with other conventional registration methods for computer-assisted navigation in orbital reconstructive surgery. Therapeutic, V.

Registration of Dental Tomographic Volume Data and Scan Surface Data Using Dynamic Segmentation

Over recent years, computer-aided design (CAD) has become widely used in the dental industry. In dental CAD applications using both volumetric computed tomography (CT) images and 3D optical scanned surface data, the two data sets need to be registered. Previous works have registered volume data and surface data by segmentation. Volume data can be converted to surface data by segmentation and the registration is achieved by the iterative closest point (ICP) method. However, the segmentation needs human input and the results of registration can be poor depending on the segmented surface. Moreover, if the volume data contains metal artifacts, the segmentation process becomes more complex since post-processing is required to remove the metal artifacts, and initially positioning the registration becomes more challenging. To overcome these limitations, we propose a modified iterative closest point (MICP) process, an automatic segmentation method for volume data and surface data. The proposed method uses a bundle of edge points detected along an intensity profile defined by points and normal of surface data. Using this dynamic segmentation, volume data becomes surface data which can be applied to the ICP method. Experimentally, MICP demonstrates fine results compared to the conventional registration method. In addition, the registration can be completed within 10 s if down sampling is applied.

Improved accuracy using a modified registration method of ROSA in deep brain stimulation surgery

OBJECTIVE The aim of this study was to determine whether a modified registration method could reduce registration error and postoperative electrode vector error and to analyze the method's clinical significance in deep brain stimulation (DBS) surgery. METHODS The first part of the study involved a skull model, in which three registration methods were tested using the ROSA (robotic stereotactic assistance) system. In the second part, four registration methods were clinically tested in patients undergoing DBS surgery using the ROSA system. Thirty-three patients (65 sides, group I) underwent the conventional registration method 2E, and registration errors were recorded. Thirty-eight patients (75 sides, group II) underwent four types of modified registration methods including 2A, 2B, 2C, and 2D. Registration and electrode vector errors, intraoperative electrophysiological signal length (IESL), and DBS power-on voltage were recorded. The primary measure of efficacy was the change in the Unified Parkinson's Disease Rating Scale (UPDRS) and UPDRS Part III scores from baseline to 10 weeks after surgery. RESULTS In the skull model, the registration error (mean ± SD) was 0.56 ± 0.11 mm for method 1A, 0.35 ± 0.11 mm for method 1B (vs. 1A, p < 0.001), and 0.90 ± 0.15 mm for method 1C (vs. 1A, p < 0.001). In the clinical study, method 2C was selected for DBS surgery in group II since it had the smallest registration error among the four methods tested. The registration error was 0.62 ± 0.22 mm (mean ± SD) for group I and 0.27 ± 0.07 mm for group II (p < 0.001). Postoperative electrode vector error was 0.97 ± 0.31 mm for group I and 0.65 ± 0.23 mm for group II (p < 0.001). There was a positive correlation between registration error and electrode vector error in both groups (group I: r = 0.69, p < 0.001; group II: r = 0.71, p < 0.001). The mean IESL was 5.0 ± 0.9 mm in group I and 5.8 ± 0.7 mm in group II (p < 0.001). The mean DBS power-on voltage was 1.63 ± 0.44 V in group I and 1.48 ± 0.38 V in group II (p = 0.027). In the UPDRS score, group I showed 50% ± 16% improvement and group II showed 52% ± 18% improvement (p = 0.724); there was no statistically significant difference in improvement on the UPDRS. CONCLUSIONS In DBS surgery assisted by the ROSA system, registration error and electrode vector error showed a positive correlation. The modified registration method could reduce the registration error and electrode vector error, but the long-term effects need to be further observed and evaluated.

Functional MRI registration with tissue-specific patch-based functional correlation tensors.

Population studies of brain function with resting-state functional magnetic resonance imaging (rs-fMRI) rely on accurate intersubject registration of functional areas. This is typically achieved through registration using high-resolution structural images with more spatial details and better tissue contrast. However, accumulating evidence has suggested that such strategy cannot align functional regions well because functional areas are not necessarily consistent with anatomical structures. To alleviate this problem, a number of registration algorithms based directly on rs-fMRI data have been developed, most of which utilize functional connectivity (FC) features for registration. However, most of these methods usually extract functional features only from the thin and highly curved cortical grey matter (GM), posing great challenges to accurate estimation of whole-brain deformation fields. In this article, we demonstrate that additional useful functional features can also be extracted from the whole brain, not restricted to the GM, particularly the white-matter (WM), for improving the overall functional registration. Specifically, we quantify local anisotropic correlation patterns of the blood oxygenation level-dependent (BOLD) signals using tissue-specific patch-based functional correlation tensors (ts-PFCTs) in both GM and WM. Functional registration is then performed by integrating the features from different tissues using the multi-channel large deformation diffeomorphic metric mapping (mLDDMM) algorithm. Experimental results show that our method achieves superior functional registration performance, compared with conventional registration methods.

Issues in quantification of registered respiratory gated PET/CT in the lung

PET/CT quantification of lung tissue is limited by several difficulties: the lung density and local volume changes during respiration, the anatomical mismatch between PET and CT and the relative contributions of tissue, air and blood to the PET signal (the tissue fraction effect). Air fraction correction (AFC) has been shown to improve PET image quantification in the lungs. Methods to correct for the movement and anatomical mismatch involve respiratory gating and image registration techniques. While conventional registration methods only account for spatial mismatch, the Jacobian determinant of the deformable registration transformation field can be used to estimate local volume changes and could therefore potentially be used to correct (i.e. Jacobian Correction, JC) the PET signal for changes in concentration due to local volume changes. This work aims to investigate the relationship between variations in the lung due to respiration, specifically density, tracer concentration and local volume changes. In particular, we study the effect of AFC and JC on PET quantitation after registration of respiratory gated PET/CT patient data. Six patients suffering from lung cancer with solitary pulmonary nodules underwent F-FDG PET/cine-CT. The PET data were gated into six respiratory gates using displacement gating based on a real-time position management (RPM) signal and reconstructed with matched gated CT. The PET tracer concentration and tissue density were extracted from registered gated PET and CT images before and after corrections (AFC or JC) and compared to the values from the reference images. Before correction, we observed a linear correlation between the PET tracer concentration values and density. Across all gates and patients, the maximum relative change in PET tracer concentration before (after) AFC was found to be 16.2% (4.1%) and the maximum relative change in tissue density and PET tracer concentration before (after) JC was found to be 17.1% (5.5%) and 16.2% (6.8%) respectively. Overall our results show that both AFC or JC largely explain the observed changes in PET tracer activity over the respiratory cycle. We also speculate that a second order effect is related to change in fluid content but this needs further investigation. Consequently, either AFC or JC is recommended when combining lung PET images from different gates to reduce noise.

An InSAR Fine Registration Algorithm Using Uniform Tie Points Based on Voronoi Diagram

Interferometric synthetic aperture radar (InSAR) image coregistration is a nontrivial task for its skew and distorted image pair, especially in severe decorrelated areas. In this letter, a new InSAR coregistration method, which considers both the coherence of the reference points and their topographical distribution, is proposed to perform accurate coregistration. First, a conventional cross-correlation registration method is performed, and a number of high coherent points are extracted as the reference points. Then, some well-distributed reference points are selected by utilizing the Voronoi diagram-based distribution optimization algorithm. Their subpixel correspondences are selected by finding the maximum values of the cross-correlation functions. Some singular correspondences are rejected by the random sample consensus method. Based on these accurate correspondences, the parameters of polynomial mapping function are estimated via the weighted least squares method. It improves the accuracy of geometrical mapping functions and overcomes the limitation of the conventional registration method when the reference points locate in low coherent area. C-band airborne repeat-pass acquired SAR data with 0.5-m resolution are used to validate the proposed algorithm by comparing with the conventional registration algorithm in accuracy. Experimental results prove the effectiveness of the proposed algorithm.

Dual-core steered non-rigid registration for multi-modal images via bi-directional image synthesis.

In prostate cancer radiotherapy, computed tomography (CT) is widely used for dose planning purposes. However, because CT has low soft tissue contrast, it makes manual contouring difficult for major pelvic organs. In contrast, magnetic resonance imaging (MRI) provides high soft tissue contrast, which makes it ideal for accurate manual contouring. Therefore, the contouring accuracy on CT can be significantly improved if the contours in MRI can be mapped to CT domain by registering MRI with CT of the same subject, which would eventually lead to high treatment efficacy. In this paper, we propose a bi-directional image synthesis based approach for MRI-to-CT pelvic image registration. First, we use patch-wise random forest with auto-context model to learn the appearance mapping from CT to MRI domain, and then vice versa. Consequently, we can synthesize a pseudo-MRI whose anatomical structures are exactly same with CT but with MRI-like appearance, and a pseudo-CT as well. Then, our MRI-to-CT registration can be steered in a dual manner, by simultaneously estimating two deformation pathways: 1) one from the pseudo-CT to the actual CT and 2) another from actual MRI to the pseudo-MRI. Next, a dual-core deformation fusion framework is developed to iteratively and effectively combine these two registration pathways by using complementary information from both modalities. Experiments on a dataset with real pelvic CT and MRI have shown improved registration performance of the proposed method by comparing it to the conventional registration methods, thus indicating its high potential of translation to the routine radiation therapy.

Two and Three Dimensional Image Registration Based on B-Spline Composition and Level Sets

AbstractAmethod for non-rigid image registration that is suitable for large deformations is presented. Conventional registration methods embed the image in a B-spline object, and the image is evolved by deforming the B-spline object. In this work, we represent the image using B-spline and deform the image using a composition approach. We also derive a computationally efficient algorithm for calculating the B-spline coefficients and gradients of the image by adopting ideas from signal processing using image filters. We demonstrate the application of our method on several different types of 2D and 3D images and compare it with existing methods.

Improving Functional MRI Registration Using Whole-Brain Functional Correlation Tensors.

Population studies of brain function with resting-state functional magnetic resonance imaging (rs-fMRI) largely rely on the accurate inter-subject registration of functional areas. This is typically achieved through registration of the corresponding T1-weighted MR images with more structural details. However, accumulating evidence has suggested that such strategy cannot well-align functional regions which are not necessarily confined by the anatomical boundaries defined by the T1-weighted MR images. To mitigate this problem, various registration algorithms based directly on rs-fMRI data have been developed, most of which have utilized functional connectivity (FC) as features for registration. However, most of the FC-based registration methods usually extract the functional features only from the thin and highly curved cortical grey matter (GM), posing a great challenge in accurately estimating the whole-brain deformation field. In this paper, we demonstrate that the additional useful functional features can be extracted from brain regions beyond the GM, particularly, white-matter (WM) based on rs-fMRI, for improving the overall functional registration. Specifically, we quantify the local anisotropic correlation patterns of the blood oxygenation level-dependent (BOLD) signals, modeled by functional correlation tensors (FCTs), in both GM and WM. Functional registration is then performed based on multiple components of the whole-brain FCTs using a multichannel Large Deformation Diffeomorphic Metric Mapping (mLDDMM) algorithm. Experimental results show that our proposed method achieves superior functional registration performance, compared with other conventional registration methods.

Learning-based deformable image registration for infant MR images in the first year of life.

Many brain development studies have been devoted to investigate dynamic structural and functional changes in the first year of life. To quantitatively measure brain development in such a dynamic period, accurate image registration for different infant subjects with possible large age gap is of high demand. Although many state-of-the-art image registration methods have been proposed for young and elderly brain images, very few registration methods work for infant brain images acquired in the first year of life, because of (a) large anatomical changes due to fast brain development and (b) dynamic appearance changes due to white-matter myelination. To address these two difficulties, we propose a learning-based registration method to not only align the anatomical structures but also alleviate the appearance differences between two arbitrary infant MR images (with large age gap) by leveraging the regression forest to predict both the initial displacement vector and appearance changes. Specifically, in the training stage, two regression models are trained separately, with (a) one model learning the relationship between local image appearance (of one development phase) and its displacement toward the template (of another development phase) and (b) another model learning the local appearance changes between the two brain development phases. Then, in the testing stage, to register a new infant image to the template, we first predict both its voxel-wise displacement and appearance changes by the two learned regression models. Since such initializations can alleviate significant appearance and shape differences between new infant image and the template, it is easy to just use a conventional registration method to refine the remaining registration. We apply our proposed registration method to align 24 infant subjects at five different time points (i.e., 2-week-old, 3-month-old, 6-month-old, 9-month-old, and 12-month-old), and achieve more accurate and robust registration results, compared to the state-of-the-art registration methods. The proposed learning-based registration method addresses the challenging task of registering infant brain images and achieves higher registration accuracy compared with other counterpart registration methods.

A 3-D point clouds scanning and registration methodology for automatic object digitization

The article presents a robot 3-D scanning system for generation of 3-D point clouds of an object by using multi-view 3-D scanning and novel data registration. Our approach mainly comprises two important elements in the determination of next best probe pose and multiple-view point clouds registration. A novel technique is proposed to register 3-D object scene with overlapped or stacked condition. Under this scenario, conventional registration methods such as the iterative closest point algorithm usually fail to converge to a global minimum when a good initial estimate for image registration does not exist. Our proposed technique uses a 3-D scanner to be mounted on a six degree of freedom-articulated industrial robot. It keeps moving probe continuously in the working range against the object and autonomously varying the probe with various gestures required for complete object scanning and for achieving best 3-D sensing accuracy. The robot scanning path is determined through a proposed algorithm using information from the latest scanning data and registered result of the object. The developed method has been verified through experimental tests for its feasibility test. It confirms that the registration accuracy with one standard deviation can be controlled within 0.323 mm when the objects underlying reconstruction are in range of hundreds of millimeters.

Nonrigid Registration Regularized by Shape Information: Application to Atlas Construction of Cardiac CT Images.

Cardiac atlases play an important role in the computer-aided diagnosis of cardiovascular diseases, in particular they need to deal with large and highly variable image datasets. In this paper, we propose a new nonrigid registration algorithm incorporating shape information, to produce comprehensive atlases. For one thing, the multiscale gradient orientation features of images are combined to form the construction of multifeature mutual information. Additionally, the shape information of multiple-objects in images is incorporated into the cost function for registration. We demonstrate the merits of the new registration algorithm on the 3D data sets of 15 patients. The experimental results show that the new registration algorithm can outperform the conventional intensity-based registration method. The obtained atlas can represent the cardiac structures more accurately.

Automatic motion estimation using flow parameters for dynamic contrast-enhanced ultrasound

Dynamic contrast-enhanced ultrasound (DCE-US) sequences are subject to motion which can disturb functional flow quantification. This can make estimated parameters more variable or unreliable. Methods that compensate for motion are therefore desirable. The most commonly used motion correction techniques in DCE-US register the images in the sequence with respect to a user-selected reference image. However, this image may not include all features that are representative of the whole sequence. Moreover, image-based registration neglects pertinent, functional-flow information contained in the DCE-US sequence.An operator-free method is proposed that combines the motion estimation and flow-parameter quantification (M/Q method) in a single mathematical framework. This method is based on a realistic multiplicative model of the DCE-US noise. By computing likelihood in this model, motion and flow parameters are both estimated iteratively. First, the maximization is accomplished by estimating functional and motion parameters. Then, a final registration based on a non-parametric temporal smoothing of the sequence is performed.This method is compared to a conventional (mutual information) registration method where all the images of the sequence are registered with respect to a reference image chosen by an expert. The two methods are evaluated on simulated sequences and DCE-US sequences acquired in patients (N = 15). The M/Q method demonstrates significantly (p < 0.05) lower Dice coefficients and Hausdorff distance than the conventional method on the simulated data sets. On the in vivo sequences analysed, the M/Q methods outperformed the conventional method in terms of mean Dice and Hausdorff distance on 80% of the sequences, and in terms of standard deviation of Dice and Hausdorff distance on 87% of the sequences.

Optical displacement metrology using alternating direction Moire

We develop a new double exposure Moire method for an optical registration metrology system in photolithography. Our method enables us to achieve at least a factor of 10 improvements in precise displacement metrology using a conventional optical sensor. We utilize a new registration mark printed to the photoresist on a bare silicon wafer using a double exposure of the gratings. The mark consists of two types of Moire with opposite phases. The two types of Moire are oriented in alternate directions. Displacement is measured from the distance between the positions of the two types of Moire in analogy with the conventional registration method. This concept is called alternating direction Moire. Performance is experimentally confirmed using an i-line wafer exposure apparatus. Precision is improved by up to 32 times as compared with the conventional method and can be applied to other Moire metrologies.

Automatic Line Segment Registration Using Gaussian Mixture Model and Expectation-Maximization Algorithm

Line segment registration (LSR) for image pairs is a challenging task but plays an important role in remote sensing and photogrammetry. This paper proposes a line segment registration method using Gaussian Mixture Models (GMMs) and Expectation-Maximization (EM) algorithm. Comparing to the conventional registration methods which consider the local appearance of points or line segments, the proposed method of LSR uses only the spatial relations between the line segments detected from an image pair, and it does not require the corresponding line segments sharing the same start points and end points. Although the proposed method is not confined to the transformation model between the image pair, the affine model, which is a simple and fast registration model and widely used in remote sensing, is taken to verify the proposed method. Various images including aerial images, satellite images and GIS data are used to test the algorithm, and test results show that the method is robust to different conditions, including rotation, noise and illumination. The results of the proposed method are compared with those of other line segment matching methods, and it is shown that the proposed method is superior in matching precision and performs better in less-texture or no-texture case.

A Joint-Constraint Model-Based System for Reconstructing Total Knee Motion

Comprehending knee motion is an essential requirement for studying the causes of knee disorders. In this paper, we propose a new 2-D-3-D registration system based on joint-constraint model for reconstructing total knee motion. The proposed model that contains bone geometries and an articulated joint mechanism is first constructed from multipostural magnetic resonance volumetric images. Then, the bone segments of the model are hierarchically registered to each frame of the given single-plane fluoroscopic video that records the knee activity. The bone posture is iteratively optimized using a modified chamfer matching algorithm to yield the simulated radiograph which is the best fit to the underlying fluoroscopic image. Unlike conventional registration methods computing posture parameters for each bone independently, the proposed femorotibial and patellofemoral joint models properly maintain the articulations between femur, tibia, and patella during the registration processes. As a result, we can obtain a sequence of registered knee postures showing smooth and reasonable physiologic patterns of motion. The proposed system also provides joint-space interpolation to densely generate intermediate postures for motion animation. The effectiveness of the proposed method was validated by computer simulation, animal cadaver, and in vivo knee testing. The mean target registration errors for femur, tibia, and patella were less than 1.5 mm. In particular, small out-of-plane registration errors [less than 1 mm (translation) and 2° (rotation)] were achieved in animal cadaver assessments.