Existing Registration Algorithms Research Articles

Real-Time Registration of Unmanned Aerial Vehicle Hyperspectral Remote Sensing Images Using an Acousto-Optic Tunable Filter Spectrometer

Differences in field of view may occur during unmanned aerial remote sensing imaging applications with acousto-optic tunable filter (AOTF) spectral imagers using zoom lenses. These differences may stem from image size deformation caused by the zoom lens, image drift caused by AOTF wavelength switching, and drone platform jitter. However, they can be addressed using hyperspectral image registration. This article proposes a new coarse-to-fine remote sensing image registration framework based on feature and optical flow theory, comparing its performance with that of existing registration algorithms using the same dataset. The proposed method increases the structure similarity index by 5.2 times, reduces the root mean square error by 3.1 times, and increases the mutual information by 1.9 times. To meet the real-time processing requirements of the AOTF spectrometer in remote sensing, a development environment using VS2023+CUDA+OPENCV was established to improve the demons registration algorithm. The registration algorithm for the central processing unit+graphics processing unit (CPU+GPU) achieved an acceleration ratio of ~30 times compared to that of a CPU alone. Finally, the real-time registration effect of spectral data during flight was verified. The proposed method demonstrates that AOTF hyperspectral imagers can be used in real-time remote sensing applications on unmanned aerial vehicles.

Robust Point Cloud Registration Network for Complex Conditions.

Point cloud registration is widely used in autonomous driving, SLAM, and 3D reconstruction, and it aims to align point clouds from different viewpoints or poses under the same coordinate system. However, point cloud registration is challenging in complex situations, such as a large initial pose difference, high noise, or incomplete overlap, which will cause point cloud registration failure or mismatching. To address the shortcomings of the existing registration algorithms, this paper designed a new coarse-to-fine registration two-stage point cloud registration network, CCRNet, which utilizes an end-to-end form to perform the registration task for point clouds. The multi-scale feature extraction module, coarse registration prediction module, and fine registration prediction module designed in this paper can robustly and accurately register two point clouds without iterations. CCRNet can link the feature information between two point clouds and solve the problems of high noise and incomplete overlap by using a soft correspondence matrix. In the standard dataset ModelNet40, in cases of large initial pose difference, high noise, and incomplete overlap, the accuracy of our method, compared with the second-best popular registration algorithm, was improved by 7.0%, 7.8%, and 22.7% on the MAE, respectively. Experiments showed that our CCRNet method has advantages in registration results in a variety of complex conditions.

Anisotropic Generalized Bayesian Coherent Point Drift for Point Set Registration

Registration is highly demanded in many real-world scenarios such as robotics and automation. Registration is challenging partly due to the fact that the acquired data is usually noisy and has many outliers. In addition, in many practical applications, one point set (PS) usually only covers a partial region of the other PS. Thus, most existing registration algorithms cannot guarantee theoretical convergence. This article presents a novel, robust, and accurate three-dimensional (3D) rigid point set registration (PSR) method, which is achieved by generalizing the state-of-the-art (SOTA) Bayesian coherent point drift (BCPD) theory to the scenario that high-dimensional point sets (PSs) are aligned and the anisotropic positional noise is considered. The high-dimensional point sets typically consist of the positional vectors and normal vectors. On one hand, with the normal vectors, the proposed method is more robust to noise and outliers, and the point correspondences can be found more accurately. On the other hand, incorporating the registration into the BCPD framework will guarantee the algorithm’s theoretical convergence. Our contributions in this article are three folds. First, the problem of rigidly aligning two general PSs with normal vectors is incorporated into a variational Bayesian inference framework, which is solved by generalizing the BCPD approach while the anisotropic positional noise is considered. Second, the updated parameters during the algorithm’s iterations are given in closed-form or with iterative solutions. Third, extensive experiments have been done to validate the proposed approach and its significant improvements over the BCPD. Note to Practitioners—This paper was motivated by the problem of 3D rigid PSR for computer-assisted surgery (CAS), especially in orthopedic applications. The proposed algorithm is also suitable for other scenarios where the initial coarse registration is conducted. The traditional registration methods are susceptible to noise (especially anisotropic noise), outliers, and incomplete partial data. This paper generalizes the recently proposed BCPD method to the six-dimensional scenario where anisotropic positional noise is considered and normal vectors are incorporated. The proposed noise model is decomposed into three parts to be solved alternately: the membership probability of mixture distributions, the soft correspondence estimation, and the model parameters (i.e., the rotation matrix, translation vector, the covariance matrix with the anisotropic positional error, and the concentration parameter with the estimation of the normal vectors). Especially, the convergence is guaranteed at the theoretical level using the variational inference theory. The experimental results demonstrate the superiority of our algorithm on registration accuracy, convergence speed, and robustness to noise, outliers, and partial data.

Image Point Set Registration: Phased Gaussian Mixture Model

The introduction of probability model into point set matching has achieved great success. Because the image point set has more outliers and the image has complex transformation, the traditional Gaussian Mixture Model (GMM) does not perform well in image point set matching. In this paper, we propose a Phased Gaussian Mixture Model (PGMM) for point set registration between images. Different from the existing registration algorithms based on GMM, PGMM uses Locality Sensitive Hashing to distinguish the feature reliability of feature points. The initial transformation relation is obtained in the segment with high feature reliability and vector constraint is applied to other segments. The average Hamming distance is used to give different weights to each Gaussian component, which is related to the characteristics between data points and model points. The proposed method has good performance in the case of large image transformation scale.

Multi-strategy mutual learning network for deformable medical image registration

Deformable medical image registration plays a vital role in clinical diagnosis, monitoring treatment, and postoperative recovery. Nevertheless, the existing registration algorithms rely on a single network or training strategy to complete the registration task. Even the most advanced registration algorithms cannot capture sufficiently compelling feature information by a single network or strategy.This paper proposes a new unsupervised and deformable medical image registration network framework to get the compelling feature. The network uses 2D sub-images of 3D images as additional constraint information to supplement the training process. Therefore, the network’s two-dimensional and three-dimensional data images can interactively learn each other’s characteristic information. In addition, we propose to perform secondary registration on the concentrated part of the registration area according to the characteristics of the input image. It enables the complete image and critical regions to learn their characteristic information interactively. This paper uses the image pyramid to integrate the two mutual learning strategies, thus proposing a multi-strategy mutual learning network(MMLN) and conducting many evaluation tests on the public data set OASIS and LPBA40. The test results show that the network can achieve better registration performance than other learning-based methods and traditional algorithms.

A Transformer-Based Coarse-to-Fine Wide-Swath SAR Image Registration Method under Weak Texture Conditions

As an all-weather and all-day remote sensing image data source, SAR (Synthetic Aperture Radar) images have been widely applied, and their registration accuracy has a direct impact on the downstream task effectiveness. The existing registration algorithms mainly focus on small sub-images, and there is a lack of available accurate matching methods for large-size images. This paper proposes a high-precision, rapid, large-size SAR image dense-matching method. The method mainly includes four steps: down-sampling image pre-registration, sub-image acquisition, dense matching, and the transformation solution. First, the ORB (Oriented FAST and Rotated BRIEF) operator and the GMS (Grid-based Motion Statistics) method are combined to perform rough matching in the semantically rich down-sampled image. In addition, according to the feature point pairs, a group of clustering centers and corresponding images are obtained. Subsequently, a deep learning method based on Transformers is used to register images under weak texture conditions. Finally, the global transformation relationship can be obtained through RANSAC (Random Sample Consensus). Compared with the SOTA algorithm, our method’s correct matching point numbers are increased by more than 2.47 times, and the root mean squared error (RMSE) is reduced by more than 4.16%. The experimental results demonstrate that our proposed method is efficient and accurate, which provides a new idea for SAR image registration.

Iterative automatic global registration algorithm for multi-view point cloud of underground tunnel space

Aiming at the narrow and long tunnel structure, few internal features, and a large amount of point cloud data, the existing registration algorithms and commercial software registration results are not ideal, an iterative global registration algorithm is proposed for massive underground tunnel point cloud registration, which is composed of local initial pose acquisition and global adjustment. Firstly, the feature point coordinates in the point cloud are extracted, and then the station-by-station registration is performed according to the Rodrigues matrix. Finally, the registration result is considered as the initial value of the parameter, and the global adjustment of all observations is carried out. The observation values are weighted by the selection weight iteration method and the weights are constantly modified in the iteration process until the threshold conditions are met and the iteration stops. In this paper, the experimental data, made up of 85 stations of point cloud data, are from the Xiamen subway tunnel, which is about 1300 m long. When the accumulated error of station-to-station registration is too large, several stations are regarded as partial wholes, and the optimal registration is achieved through multiple global adjustments, and the registration accuracy is within 5 mm. Experimental results confirm the feasibility and effectiveness of the algorithm, which provides a new method for point cloud registration of underground space tunnel.

A Coarse-to-Fine Deformable Transformation Framework for Unsupervised Multi-Contrast MR Image Registration with Dual Consistency Constraint.

Multi-contrast magnetic resonance (MR) image registration is useful in the clinic to achieve fast and accurate imaging-based disease diagnosis and treatment planning. Nevertheless, the efficiency and performance of the existing registration algorithms can still be improved. In this paper, we propose a novel unsupervised learning-based framework to achieve accurate and efficient multi-contrast MR image registration. Specifically, an end-to-end coarse-to-fine network architecture consisting of affine and deformable transformations is designed to improve the robustness and achieve end-to-end registration. Furthermore, a dual consistency constraint and a new prior knowledge-based loss function are developed to enhance the registration performances. The proposed method has been evaluated on a clinical dataset containing 555 cases, and encouraging performances have been achieved. Compared to the commonly utilized registration methods, including VoxelMorph, SyN, and LT-Net, the proposed method achieves better registration performance with a Dice score of 0.8397± 0.0756 in identifying stroke lesions. With regards to the registration speed, our method is about 10 times faster than the most competitive method of SyN (Affine) when testing on a CPU. Moreover, we prove that our method can still perform well on more challenging tasks with lacking scanning information data, showing the high robustness for the clinical application.

Cardiac motion estimation from medical images: a regularisation framework applied on pairwise image registration displacement fields

Accurate cardiac motion estimation from medical images such as ultrasound is important for clinical evaluation. We present a novel regularisation layer for cardiac motion estimation that will be applied after image registration and demonstrate its effectiveness. The regularisation utilises a spatio-temporal model of motion, b-splines of Fourier, to fit to displacement fields from pairwise image registration. In the process, it enforces spatial and temporal smoothness and consistency, cyclic nature of cardiac motion, and better adherence to the stroke volume of the heart. Flexibility is further given for inclusion of any set of registration displacement fields. The approach gave high accuracy. When applied to human adult Ultrasound data from a Cardiac Motion Analysis Challenge (CMAC), the proposed method is found to have 10% lower tracking error over CMAC participants. Satisfactory cardiac motion estimation is also demonstrated on other data sets, including human fetal echocardiography, chick embryonic heart ultrasound images, and zebrafish embryonic microscope images, with the average Dice coefficient between estimation motion and manual segmentation at 0.82–0.87. The approach of performing regularisation as an add-on layer after the completion of image registration is thus a viable option for cardiac motion estimation that can still have good accuracy. Since motion estimation algorithms are complex, dividing up regularisation and registration can simplify the process and provide flexibility. Further, owing to a large variety of existing registration algorithms, such an approach that is usable on any algorithm may be useful.

A Point Cloud Registration Algorithm Based on Feature Extraction and Matching

The existing registration algorithms suffer from low precision and slow speed when registering a large amount of point cloud data. In this paper, we propose a point cloud registration algorithm based on feature extraction and matching; the algorithm helps alleviate problems of precision and speed. In the rough registration stage, the algorithm extracts feature points based on the judgment of retention points and bumps, which improves the speed of feature point extraction. In the registration process, FPFH features and Hausdorff distance are used to search for corresponding point pairs, and the RANSAC algorithm is used to eliminate incorrect point pairs, thereby improving the accuracy of the corresponding relationship. In the precise registration phase, the algorithm uses an improved normal distribution transformation (INDT) algorithm. Experimental results show that given a large amount of point cloud data, this algorithm has advantages in both time and precision.

Image Registration Algorithm Based on Copula Distribution Estimation

In order to solve the problem that the measure function is easy to fall into the local extremum because of much local extremes in the mutual information registration method, this paper designs an optimization algorithm applied to image registration. One measure function based on mutual information and gradient similarity is constructed combining with evolutional copula estimation of distribution algorithm (ECEDA). New algorithm fully pays attention to the correlation between multidimensional variables joining with the linear weighted based on nonparametric estimation method to overcome the randomness of nonparametric estimation method. Experimental results show that comparing with the traditional copula estimation of distribution algorithm and other existing registration algorithms, the proposed algorithm has higher accurate rate and robustness.

A Characteristic Statistic Based Femoral Contour Point-set Registration Algorithm

The point-set registration technology has been widely applied in the field of medical image registration. This paper proposed a characteristic statistic based femoral contour point-set registration algorithm, in order to overcome the flaw of getting into local extrema and of poor global performance in the iterative optimization process in existing registration algorithms. First, it uses the Gaussian mixture model to represent the correspondence of points between two point-sets, and selects the global parameter associated with the objective function as the characteristic statistical items. Then, it leverages the statistical variation in the optimization process to guide the optimal search, yielding the global optimal solution of statistical items, and realizing the registration of two point-sets. In the end, this algorithm is applied to solve the registration problem of femoral contour characteristic point-set. The experimental results reveal that the proposed registration method performs more effective, faster, and more accurate compared with Coherent Point Drift (CPD) algorithm.

Improved image registration by sparse patch-based deformation estimation

Despite intensive efforts for decades, deformable image registration is still a challenging problem due to the potential large anatomical differences across individual images, which limits the registration performance. Fortunately, this issue could be alleviated if a good initial deformation can be provided for the two images under registration, which are often termed as the moving subject and the fixed template, respectively. In this work, we present a novel patch-based initial deformation prediction framework for improving the performance of existing registration algorithms. Our main idea is to estimate the initial deformation between subject and template in a patch-wise fashion by using the sparse representation technique. We argue that two image patches should follow the same deformation toward the template image if their patch-wise appearance patterns are similar. To this end, our framework consists of two stages, i.e., the training stage and the application stage. In the training stage, we register all training images to the pre-selected template, such that the deformation of each training image with respect to the template is known. In the application stage, we apply the following four steps to efficiently calculate the initial deformation field for the new test subject: (1) We pick a small number of key points in the distinctive regions of the test subject; (2) for each key point, we extract a local patch and form a coupled appearance-deformation dictionary from training images where each dictionary atom consists of the image intensity patch as well as their respective local deformations; (3) a small set of training image patches in the coupled dictionary are selected to represent the image patch of each subject key point by sparse representation. Then, we can predict the initial deformation for each subject key point by propagating the pre-estimated deformations on the selected training patches with the same sparse representation coefficients; and (4) we employ thin-plate splines (TPS) to interpolate a dense initial deformation field by considering all key points as the control points. Thus, the conventional image registration problem becomes much easier in the sense that we only need to compute the remaining small deformation for completing the registration of the subject to the template. Experimental results on both simulated and real data show that the registration performance can be significantly improved after integrating our patch-based deformation prediction framework into the existing registration algorithms.

Message Passing Matching Dynamics for Overlapping Point Identification

Existing registration algorithms usually converge to a local minimum due to inaccurate evaluation of the tentative correspondences established. In this paper, we move a step further and instead estimate the extent to which a point lies in the overlapping area. To this end, we regard the registration problem as an exchange network and develop a matching dynamics to characterize the interaction inside. Then we propose a novel algorithm based on the powerful message passing scheme derived from the matching dynamics for the optimization of the overlapping point weight. The novel algorithm penalizes in the process of deterministic annealing those tentative correspondences that violate the properties of the matching dynamics. The rigid transformation that brings the two overlapping shapes into alignment is finally estimated in the weighted least squares sense. Our experiments use both synthetic and real data to show that our proposed algorithm is more likely to converge to the global minimum than four selected state of the art ones for more accurate and robust results.

3D-3D Registration of partial capitate bones using spin-images.

It is often necessary to register partial objects in medical imaging. Due to limited FOV, the entirety of an object cannot always be imaged. This study presents a novel application of an existing registration algorithm to this problem. The spin-image algorithm [1] creates pose-invariant representations of global shape with respect to individual mesh vertices. These 'spin-images,' are then compared for two different poses of the same object to establish correspondences and subsequently determine relative orientation of the poses. In this study, the spin-image algorithm is applied to 4DCT-derived capitate bone surfaces to assess the relative accuracy of registration with various amounts of geometry excluded. The limited longitudinal coverage under the 4DCT technique (38.4mm, [2]), results in partial views of the capitate when imaging wrist motions. This study assesses the ability of the spin-image algorithm to register partial bone surfaces by artificially restricting the capitate geometry available for registration. Under IRB approval, standard static CT and 4DCT scans were obtained on a patient. The capitate was segmented from the static CT and one phase of 4DCT in which the whole bone was available. Spin-image registration was performed between the static and 4DCT. Distal portions of the 4DCT capitate (10-70%) were then removed and registration was repeated. Registration accuracy was evaluated by angular errors and the percentage of sub-resolution fitting. It was determined that 60% of the distal capitate could be omitted without appreciable effect on registration accuracy using the spin-image algorithm (angular error < 1.5 degree, sub-resolution fitting > 98.4%).

A general fast registration framework by learning deformation-appearance correlation.

In this paper, we propose a general framework for performance improvement of the current state-of-the-art registration algorithms in terms of both accuracy and computation time. The key concept involves rapid prediction of a deformation field for registration initialization, which is achieved by a statistical correlation model learned between image appearances and deformation fields. This allows us to immediately bring a template image as close as possible to a subject image that we need to register. The task of the registration algorithm is hence reduced to estimating small deformation between the subject image and the initially warped template image, i.e., the intermediate template (IT). Specifically, to obtain a good subject-specific initial deformation, support vector regression is utilized to determine the correlation between image appearances and their respective deformation fields. When registering a new subject onto the template, an initial deformation field is first predicted based on the subject's image appearance for generating an IT. With the IT, only the residual deformation needs to be estimated, presenting much less challenge to the existing registration algorithms. Our learning-based framework affords two important advantages: 1) by requiring only the estimation of the residual deformation between the IT and the subject image, the computation time can be greatly reduced; 2) by leveraging good deformation initialization, local minima giving suboptimal solution could be avoided. Our framework has been extensively evaluated using medical images from different sources, and the results indicate that, on top of accuracy improvement, significant registration speedup can be achieved, as compared with the case where no prediction of initial deformation is performed.

SU-GG-I-111: A Novel Stochastic Optimization Approach for Multimodal Image Registration

Purpose: The task of image registration appears in many different areas of radiation therapy, e.g. comparing images during the treatment planning process. However, the complexity of the existing registration algorithms requires extensive computation time which limits their applications. Therefore, we propose a fast and robust algorithm for mono and multimodal registration. Method and Materials: We use the Lie Group structure to describe the set of rigid transformations and we generate the set of admissible deformations with a cubic B-spline basis in the case of non-rigid deformations. In order to solve the optimization problem that arises from the registration task, we developed a stochastic Gauss-Newton method combining the advantages of the (stochastic gradient) Robbins-Monro algorithms with the well-known deterministic Gauss-Newton method. To incorporate multimodal image registration as well, we also present a novel image distance measure. For test purposes we implemented our algorithms in Matlab. Results: A comparison of the new measure with the classical mutual information measure in the case of rigid registration showed that both measures detect the position of a template image in a reference image with the same quality, but the numerical costs for one optimization step are reduced significant with the new measure. The stochastic algorithm led to a considerable reduction of computation time, compared to the deterministic Gauss-Newton method. The final algorithm was tested for CT-MR and CT-PET and CT-CT registration. Especially the last case allows to detect huge deformations which appear e.g. in patient-to-patient registration. Conclusion: We have developed a novel, fast and robust registration algorithm. It combines the advantages of the mutual information measure with the superior convergence properties of a quasi adaptive stochastic algorithm.

Accurate and robust brain image alignment using boundary-based registration

The fine spatial scales of the structures in the human brain represent an enormous challenge to the successful integration of information from different images for both within- and between-subject analysis. While many algorithms to register image pairs from the same subject exist, visual inspection shows that their accuracy and robustness to be suspect, particularly when there are strong intensity gradients and/or only part of the brain is imaged. This paper introduces a new algorithm called Boundary-Based Registration, or BBR. The novelty of BBR is that it treats the two images very differently. The reference image must be of sufficient resolution and quality to extract surfaces that separate tissue types. The input image is then aligned to the reference by maximizing the intensity gradient across tissue boundaries. Several lower quality images can be aligned through their alignment with the reference. Visual inspection and fMRI results show that BBR is more accurate than correlation ratio or normalized mutual information and is considerably more robust to even strong intensity inhomogeneities. BBR also excels at aligning partial-brain images to whole-brain images, a domain in which existing registration algorithms frequently fail. Even in the limit of registering a single slice, we show the BBR results to be robust and accurate.

A Nonlinear Least Square Technique for Simultaneous Image Registration and Super-Resolution

This paper proposes a new algorithm to integrate image registration into image super-resolution (SR). Image SR is a process to reconstruct a high-resolution (HR) image by fusing multiple low-resolution (LR) images. A critical step in image SR is accurate registration of the LR images or, in other words, effective estimation of motion parameters. Conventional SR algorithms assume either the estimated motion parameters by existing registration methods to be error-free or the motion parameters are known a priori. This assumption, however, is impractical in many applications, as most existing registration algorithms still experience various degrees of errors, and the motion parameters among the LR images are generally unknown a priori. In view of this, this paper presents a new framework that performs simultaneous image registration and HR image reconstruction. As opposed to other current methods that treat image registration and HR reconstruction as disjoint processes, the new framework enables image registration and HR reconstruction to be estimated simultaneously and improved progressively. Further, unlike most algorithms that focus on the translational motion model, the proposed method adopts a more generic motion model that includes both translation as well as rotation. An iterative scheme is developed to solve the arising nonlinear least squares problem. Experimental results show that the proposed method is effective in performing image registration and SR for simulated as well as real-life images.

Nonrigid registration of brain MRI using NURBS

The focus of medical image registration has shifted to nonrigid registration. It is also a core component in computational neuroanatomy. In this paper, we propose an efficient registration framework which has feature of multiresolution and multigrid. In contrast to the existing registration algorithms. Free-Form Deformations based NURBS (Nonuniform Rational B Spline) are used to acquire nonrigid transformation. This can provide a competitive alternative to Free-Form Deformations based B spline on flexibility and accuracy. To our knowledge, it is the first report of the use of this mathematical tool for medical image registration. Subdivision of NURBS is extended to 3D and is used in hierarchical optimization to speed up the registration and avoid local minima. The performance of this method is numerically evaluated on simulated images and real images. Compared with the registration method using uniform Free-Form Deformations based B spline, our method can successfully register images with improved performance.