Conventional Registration Methods Research Articles

ACSwinNet: A Deep Learning-Based Rigid Registration Method for Head-Neck CT-CBCT Images in Image-Guided Radiotherapy.

Accurate and precise rigid registration between head-neck computed tomography (CT) and cone-beam computed tomography (CBCT) images is crucial for correcting setup errors in image-guided radiotherapy (IGRT) for head and neck tumors. However, conventional registration methods that treat the head and neck as a single entity may not achieve the necessary accuracy for the head region, which is particularly sensitive to radiation in radiotherapy. We propose ACSwinNet, a deep learning-based method for head-neck CT-CBCT rigid registration, which aims to enhance the registration precision in the head region. Our approach integrates an anatomical constraint encoder with anatomical segmentations of tissues and organs to enhance the accuracy of rigid registration in the head region. We also employ a Swin Transformer-based network for registration in cases with large initial misalignment and a perceptual similarity metric network to address intensity discrepancies and artifacts between the CT and CBCT images. We validate the proposed method using a head-neck CT-CBCT dataset acquired from clinical patients. Compared with the conventional rigid method, our method exhibits lower target registration error (TRE) for landmarks in the head region (reduced from 2.14 ± 0.45 mm to 1.82 ± 0.39 mm), higher dice similarity coefficient (DSC) (increased from 0.743 ± 0.051 to 0.755 ± 0.053), and higher structural similarity index (increased from 0.854 ± 0.044 to 0.870 ± 0.043). Our proposed method effectively addresses the challenge of low registration accuracy in the head region, which has been a limitation of conventional methods. This demonstrates significant potential in improving the accuracy of IGRT for head and neck tumors.

A landmark-supervised registration framework for multi-phase CT images with cross-distillation

Objective. Multi-phase computed tomography (CT) has become a leading modality for identifying hepatic tumors. Nevertheless, the presence of misalignment in the images of different phases poses a challenge in accurately identifying and analyzing the patient’s anatomy. Conventional registration methods typically concentrate on either intensity-based features or landmark-based features in isolation, so imposing limitations on the accuracy of the registration process. Method. We establish a nonrigid cycle-registration network that leverages semi-supervised learning techniques, wherein a point distance term based on Euclidean distance between registered landmark points is introduced into the loss function. Additionally, a cross-distillation strategy is proposed in network training to further improve registration performance which incorporates response-based knowledge concerning the distances between feature points. Results. We conducted experiments using multi-centered liver CT datasets to evaluate the performance of the proposed method. The results demonstrate that our method outperforms baseline methods in terms of target registration error. Additionally, Dice scores of the warped tumor masks were calculated. Our method consistently achieved the highest scores among all the comparing methods. Specifically, it achieved scores of 82.9% and 82.5% in the hepatocellular carcinoma and the intrahepatic cholangiocarcinoma dataset, respectively. Significance. The superior registration performance indicates its potential to serve as an important tool in hepatic tumor identification and analysis.

Local geometric edge features based registration for textureless object in augmented reality assisted assembly

Image-based methods have been widely used in augmented reality (AR) assistant assembly systems. However, due to the lack of sufficient texture information on the surface of assembly part, traditional image feature matching methods still face challenges. This paper proposes a coarse-to-fine AR registration method for textureless assembly part. In the first stage, a new feature matching method which is called line neighborhood edge descriptor (LNED) is presented to find the coarse camera pose from textureless image. The LNED take the contour line of assembly part as the description object, and use local geometric edge of assembly part to describe the contour line. During the image matching, the binary encoding is used to reduce the computational consumption for LNED. In the second stage, spatial points in the CAD model of assembly part are reverse projected to the textureless image based on the coarse camera pose. And the bundle adjustment method based on the edge distance of the textureless image is adopted to iteratively calculate the precise camera pose. In the experimental evaluation, the proposed registration method shows high accuracy and fast speed in comparison with conventional registration methods, which demonstrates that our method can effectively solve the problem of AR registration for textureless assembly part.

Deep learning-based registration of two-dimensional dental images with edge specific loss.

Image registration is a very common procedure in dental applications for aligning images. Registration between pairs of images taken from different angles can improve diagnosis. Our study presents an edge-enhanced unsupervised deep learning (DL)-based deformable registration framework for aligning two-dimensional (2D) pairs of dental x-ray images. The proposed neural network is based on the combination of a U-Net like structure, which produces a displacement field, combined with spatial transformer networks, which produce the transformed image. The proposed structure is trained end-to-end by minimizing a weighted loss function consisting of three parts corresponding to image similarity, edge similarity, and registration restrictions. In this regard, the proposed edge specific loss enhances the unsupervised training of the registration framework without the need of supervision through anatomical structures. The proposed framework was applied to two datasets, a set of 104 x-ray images of mandibles, arranged in 2600 pairs for training and testing and a set of 17 pairs of pre- and post-operative reconstructed panoramic images. The proposed model outperformed both conventional registration methods and DL-based techniques for both qualitative and quantitative assessment, in most of the compared metrics concerning intensity similarity and edge distances. The proposed framework achieved accurate and fast deformable alignment of pairs of 2D dental radiographic images. The edge-based module of the loss function enhances the unsupervised learning by directing the network toward deformations that take into consideration the edges of the depicted objects (teeth, bone, and tissue), which are crucial in diagnosis.

Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning.

Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing approaches. The Learn2Reg challenge addresses these limitations by providing a multi-task medical image registration data set for comprehensive characterisation of deformable registration algorithms. A continuous evaluation will be possible at https:// learn2reg.grand-challenge.org. Learn2Reg covers a wide range of anatomies (brain, abdomen, and thorax), modalities (ultrasound, CT, MR), availability of annotations, as well as intra- and inter-patient registration evaluation. We established an easily accessible framework for training and validation of 3D registration methods, which enabled the compilation of results of over 65 individual method submissions from more than 20 unique teams. We used a complementary set of metrics, including robustness, accuracy, plausibility, and runtime, enabling unique insight into the current state-of-the-art of medical image registration. This paper describes datasets, tasks, evaluation methods and results of the challenge, as well as results of further analysis of transferability to new datasets, the importance of label supervision, and resulting bias. While no single approach worked best across all tasks, many methodological aspects could be identified that push the performance of medical image registration to new state-of-the-art performance. Furthermore, we demystified the common belief that conventional registration methods have to be much slower than deep-learning-based methods.

Fast Deformable Image Registration for Real-Time Target Tracking During Radiation Therapy Using Cine MRI and Deep Learning

We developed a deep learning (DL) model for fast deformable image registration using 2-dimensional sagittal cine magnetic resonance imaging (MRI) acquired during radiation therapy and evaluated its potential for real-time target tracking compared with conventional image registration methods. Our DL model uses a pair of cine MRI images as input and provides a motion vector field (MVF) as output. The MVF is then applied to align the input images. A retrospective study was conducted to train and evaluate our model using cine MRI data from patients undergoing treatment for abdominal and thoracic tumors. For each treatment fraction, MR-linear accelerator delivery log files, tracking videos, and cine image files were analyzed. Individual MRI frames were temporally sampled to construct a large set of image registration pairs used to evaluate multiple methods. The DL model was optimized using 5-fold cross validation, and model outputs (transformed images and MVFs) using test set images were saved for comparison with 3 conventional registration methods (affine, b-spline, and demons). Evaluation metrics were 3-fold: (1) registration error, (2) MVF stability (both spatial and temporal), and (3) average computation time. We analyzed >21 hours of cine MRI (>629,000 frames) acquired during 86 treatment fractions from 21 patients. In a test set of 10,320 image registration pairs, DL registration outperformed conventional methods in both registration error (affine, b-spline, demons, DL; root mean square error: 0.067, 0.040, 0.036, 0.032; paired t test demons vs DL: t[20]=4.2, P < .001) and computation time per frame (51, 1150, 4583, 8 ms). Among deformable methods, spatial stability of resulting MVFs was comparable; however, the DL model had significantly improved temporal consistency. DL-based image registration can leverage large-scale MR cine data sets to outperform conventional registration methods and is a promising solution for real-time deformable motion estimation in radiation therapy.

Real-Time Internal Motion Monitoring during Radiation Therapy Using Cine MRI and Deep Learning

<h3>Purpose/Objective(s)</h3> Fast and accurate internal motion estimation is fundamental to enabling real-time MRI-guided radiation therapy (RT). We hypothesized that a deep learning (DL) model for fast deformable image registration using 2D sagittal cine MRI is feasible. Herein, we report performance of a DL-based image registration model as compared to conventional image registration methods. <h3>Materials/Methods</h3> Our DL model uses a pair of cine MRI images acquired using an MR-LINAC as inputs and outputs a motion vector field (MVF) which is applied to align the images. Cine delivery data for patients (pts) undergoing treatment for abdominal and thoracic tumors were retrospectively analyzed. Cine MRI scans were sampled, constructing a large set of image registration pairs capturing pt respiratory motion to train the DL model. Data were partitioned by pt using five-fold cross validation. Model outputs (transformed images and MVFs) on test set images were saved for comparison with three conventional registration methods (affine, b-spline, and demons). Performance was quantified via average registration error and computation time. Paired t-tests determined whether differences in performance were statistically significant. <h3>Results</h3> We analyzed >21 hours of cine MRI (>629,000 frames) acquired during 86 treatment fractions from 21 pts. In a test set of 10,320 image registration pairs, DL registration outperformed conventional methods in both registration error (affine, b-spline, demons vs. DL; RMSE: 0.067, 0.040, 0.036 vs. 0.032; paired t-test demons vs DL: t(20) = 4.2, p<0.001) and computation time per frame (51 ms, 1150 ms, 4583 ms vs. 8 ms). <h3>Conclusion</h3> DL-based image registration can leverage large-scale cine MRI datasets to learn to predict pt internal motion fields using a single reference and a single moving frame as inputs. Low registration error and fast computation speed indicate that DL-based registration is a promising solution for real-time deformable motion tracking in RT. Further inter-institutional validation is underway.

The application of multiple metrics in deformable image registration for target volume delineation of breast tumor bed.

For postoperative breast cancer patients, deformable image registration (DIR) is challenged due to the large deformations and non-correspondence caused by tumor resection and clip insertion. To deal with it, three metrics (fiducial-, region-, and intensity-based) were jointly used in DIR algorithm for improved accuracy. Three types of metrics were combined to form a single-objective function in DIR algorithm. Fiducial-based metric was used to minimize the distance between the corresponding point sets of two images. Region-based metric was used to improve the overlap between the corresponding areas of two images. Intensity-based metric was used to maximize the correlation between the corresponding voxel intensities of two images. The two CT images, one before surgery and the other after surgery, were acquired from the same patient in the same radiotherapy treatment position. Twenty patients who underwent breast-conserving surgery and postoperative radiotherapy were enrolled in this study. For target registration error, the difference between the proposed and the conventional registration methods was statistically significant for soft tissue (2.06 vs. 7.82, p=0.00024<0.05) and body boundary (3.70 vs. 6.93, p=0.021<0.05). For visual assessment, the proposed method achieved better matching result for soft tissue and body boundary. Comparing to the conventional method, the registration accuracy of the proposed method was significantly improved. This method provided a feasible way for target volume delineation of tumor bed in postoperative radiotherapy of breast cancer patients.

Biomechanical model registration for monitoring and simulating large orthodontic tooth movements in the maxilla and mandible.

Superimposition of digital dental-arch models allows quantification of orthodontic tooth movements (OTM). Currently, this procedure requires stable reference surfaces usually only present in the maxilla. This study aimed to investigate the accuracy of anovel superimposition approach based on biomechanical principles of OTM and the equilibrium of forces and moments (EFM)-applicable in both jaws-for monitoring and simulating large OTM. The study included 7patients who had undergone extraction of the first (PM1-Ex) or second (PM2-Ex) premolar in each quadrant. Digital models taken at start and end of the T‑Loop treatment phase were superimposed by applying 3 EFM variants differing in the number of teeth used for registration. Maxillary OTM results for EFM were validated against those for aconventional surface registration method (SRM). In an additional case study, OTM were simulated for PM1-Ex, PM2-Ex and non-extraction treatment strategies. The EFM variant that included all teeth of the dental arch achieved the highest accuracy, with median translational and rotational OTM deviations from SRM of only 0.37 mm and 0.56°, respectively. On average, retracted canines and first premolars were distalized by 3.0 mm, accompanied by 6.2° distal crown tipping and 12.2° distorotation. The share of space closure by molar mesialization was 19.4% for PM1-Ex quadrants and 34.5% for PM2-Ex quadrants. EFM allows accurate OTM quantification relative to the maxillary and mandibular bases even in challenging situations involving large OTM. Superimposition of malocclusion and setup models enables realistic simulation of final tooth positions. This may greatly enhance the value of digital setups for decision-making in orthodontic treatment planning.

Efficient Pairwise 3-D Registration of Urban Scenes via Hybrid Structural Descriptors

Automatic registration of point clouds captured by terrestrial laser scanning (TLS) plays an important role in many fields including remote sensing (e.g., transportation management, 3-D reconstruction in large-scale urban areas and environment monitoring), computer vision, and virtual reality and robotics. However, noise, outliers, nonuniform point density, and small overlaps are inevitable when collecting multiple views of data, which poses great challenges to 3-D registration of point clouds. Since conventional registration methods aim to find point correspondences and estimate transformation parameters directly in the original point space, the traditional way to address these difficulties is to introduce many restrictions during the scanning process (e.g., more scanning and careful selection of scanning positions), thus making the data acquisition more difficult. In this article, we present a novel 3-D registration framework that performs in a &#x201C;middle-level structural space&#x201D; and is capable of robustly and efficiently reconstructing urban, semiurban, and indoor scenes, despite disturbances introduced in the scanning process. The new structural space is constructed by extracting multiple types of middle-level geometric primitives (planes, spheres, cylinders, and cones) from the 3-D point cloud. We design a robust method to find effective primitive combinations corresponding to the 6-D poses of the raw point clouds and then construct hybrid-structure-based descriptors. By matching descriptors and computing rotation and translation parameters, successful registration is achieved. Note that the whole process of our method is performed in the structural space, which has the advantages of capturing geometric structures (the relationship between primitives) and semantic features (primitive types and parameters) in larger fields. Experiments show that our method achieves state-of-the-art performance in several point cloud registration benchmark datasets at different scales and even obtains good registration results for data without overlapping areas.

Raw Scanned Point Cloud Registration with Repetition for Aircraft Fuel Tank Inspection

The aircraft fuel tank, as the source of fuel supply, is one of the key components for an aircraft. It is fastened to the fuselage with thousands of rivets, in which each rivet is sealed carefully to prevent the leakage. Thus, the sealing quality of the aircraft fuel tank needs to be precisely measured to ensure its normal function. To achieve the measurement, the complete 3D point clouds captured by 3D scanning are usually utilized. Therefore, it is necessary to align the different views of the aircraft fuel tank point clouds, to obtain the complete data. However, the aircraft fuel tank consists of abundant repetitive structures, such as the regular-distributed rivets, which can lead to mismatches in the aircraft fuel tank point clouds if using the conventional registration methods. In order to address the challenges, we propose a new registration framework, which is able to handle the point cloud match under the repetitive and complex scenes. First, a novel 3D descriptor called LP-PPF is constructed with the point and line features from the aircraft fuel tank point clouds, so as to identify the repetitive structures correctly. Then, according to the proposed descriptor, the accurate registration can be achieved between the neighboring point cloud pairs. Finally, based on the results of pairwise registration, the complete aircraft fuel tank point cloud can be obtained via the multi-view registration method. Experiments demonstrate that our method achieves favorable results on both synthetic and raw scanned point cloud data.

CNN-based lung CT registration with multiple anatomical constraints.

Deep-learning-based registration methods emerged as a fast alternative to conventional registration methods. However, these methods often still cannot achieve the same performance as conventional registration methods because they are either limited to small deformation or they fail to handle a superposition of large and small deformations without producing implausible deformation fields with foldings inside. In this paper, we identify important strategies of conventional registration methods for lung registration and successfully developed the deep-learning counterpart. We employ a Gaussian-pyramid-based multilevel framework that can solve the image registration optimization in a coarse-to-fine fashion. Furthermore, we prevent foldings of the deformation field and restrict the determinant of the Jacobian to physiologically meaningful values by combining a volume change penalty with a curvature regularizer in the loss function. Keypoint correspondences are integrated to focus on the alignment of smaller structures. We perform an extensive evaluation to assess the accuracy, the robustness, the plausibility of the estimated deformation fields, and the transferability of our registration approach. We show that it achieves state-of-the-art results on the COPDGene dataset compared to conventional registration method with much shorter execution time. In our experiments on the DIRLab exhale to inhale lung registration, we demonstrate substantial improvements (TRE below 1.2 mm) over other deep learning methods. Our algorithm is publicly available at https://grand-challenge.org/algorithms/deep-learning-based-ct-lung-registration/.

PHOTOGRAMMETRY AND COMPUTED TOMOGRAPHY POINT CLOUD REGISTRATION USING VIRTUAL CONTROL POINTS

Abstract. In this paper we propose a virtual control point based method for the registration of photogrammetry and computed tomography (CT) data. Because of the fundamentally different two data sources, conventional registration methods, such as manual control points registration or 3D local feature-based registration, are not suitable. The registration objective of our application is about 3D reconstructions of gyroscopes, which contain abundant geometric primitives to be fitted in the point clouds. In the first place, photogrammetry and CT scanning are applied, respectively, for 3D reconstructions. Secondly, our workflow implements a segmentation after obtaining the surface point cloud from the complete CT volumetric data. Then geometric primitives are fitted in this point cloud benefitting from the less complex cluster segments. In the next step, intersection operations of the parametrized primitives generates virtual points, which are utilized as control points for the transformation parameters estimation. A random sample consensus (RANSAC) method is applied to find the correspondences of both virtual control point sets using corresponding descriptors and calculates the transformation matrix as an initial alignment for further refining the registration. The workflow is invariant to pose, resolution, completeness and noise within our validation process.

Three-dimensional analysis of the accuracy of conventional and completely digital interocclusal registration methods

Statement of problemCompared with the frequent investigations into the accuracy of digital intraoral scans, studies analyzing digital determinations of jaw relationships based on intraoral scans are scarce. PurposeThe purpose of this in vitro study was to present an optical 3-dimensional method for analyzing deviations in static occlusion and to compare the accuracy of conventional and digital interocclusal registrations. Material and methodsA Frasaco jaw model was duplicated, articulated, and scanned with a high-precision industrial scanner, and the data were stored in a virtual standard tessellation language (STL) format, which served as the reference model. Fifteen paired mandibular and maxillary models were scanned with a digital intraoral scanner in the completely digital workflow (IOS group). Forty-five paired gypsum casts were poured from polyvinyl siloxane (PVS) impressions and associated with 2 different PVS registration materials. These casts were digitized with a laboratory scanner and grouped as follows (n=15/group): PVS group, conventional Futar D interocclusal record; sPVS group, conventional Futar Scan interocclusal record; and the AIR group, partially digital antagonist scan of the Futar Scan interocclusal record. The axes (X, Y, Z, and XYZ) of each paired model were aligned to those of the reference model by 3-dimensional superimposition, and deviations were calculated. To determine the ideal zero position, a best-fit over the mandibular teeth between the reference model and the actual model was estimated. Next, a second best-fit was determined between the maxillary models to determine the actual position of the mandibular model. The different registration methods were compared with the Mann–Whitney U test (α=.05). ResultsIn the IOS group, the interocclusal registration caused a mandibular deviation of 0.05 ±0.04 mm (mean ±standard deviation). This fit was better than those of conventional registrations with inserted interocclusal registration materials (PVS group and sPVS group), which caused mean z-axis deviations of 0.41 ±0.46 mm and 0.44 ±0.32 mm (P<.001), with the deviations leading to elevation of the mandibular model. The partially digital workflow with a scannable registration material (AIR group) showed significantly larger deviations in the x-axis (0.15 ±0.08 mm; P=.042) compared with the IOS group. No significant difference was observed in the total deviation between the IOS and the AIR groups. Both groups showed significantly smaller deviations than the conventional registration methods (P<.001 for the IOS group and P=.023 for the AIR group). ConclusionsIn comparison with maxillary and mandibular alignment using conventional interocclusal registration materials, digital interocclusal registrations showed greater accuracy in evaluating complete jaw models and can be recommended for clinical use. Additionally, the partially digital workflow with an antagonist scan of the interocclusal record provided acceptable results.

A coarse-to-fine registration method for three-dimensional MR images.

Three-dimensional (3D) multimodal magnetic resonance (MR) image registration aims to align similar things in different MR images spatially. Such a technology is useful in auxiliary disease diagnosis and surgical treatment. However, inconsistent intensity correspondence and large initial displacement contribute to the difficulty in registering multimodal MR volumes. A coarse-to-fine method is proposed in this study for pairwise 3D MR image rigid registration. Firstly, the proposed method extracts image feature points to form unregistered point sets and performs coarse registration based on point set registration to reduce the initial displacements of offset images effectively. Then, this method calculates a grey histogram based on voxels in the adaptive region of interest and further improves registration accuracy by maximizing mutual information of coarse-registered images. Some representative registration methods are compared on the basis of three MR image datasets to evaluate the performance of the proposed method. Experimental results show that the proposed method improved more in registration success rate and accuracy compared with conventional registration methods, especially when initial displacements are large.

Non-Rigid Respiratory Motion Estimation of Whole-Heart Coronary MR Images Using Unsupervised Deep Learning.

Non-rigid motion-corrected reconstruction has been proposed to account for the complex motion of the heart in free-breathing 3D coronary magnetic resonance angiography (CMRA). This reconstruction framework requires efficient and accurate estimation of non-rigid motion fields from undersampled images at different respiratory positions (or bins). However, state-of-the-art registration methods can be time-consuming. This article presents a novel unsupervised deep learning-based strategy for fast estimation of inter-bin 3D non-rigid respiratory motion fields for motion-corrected free-breathing CMRA. The proposed 3D respiratory motion estimation network (RespME-net) is trained as a deep encoder-decoder network, taking pairs of 3D image patches extracted from CMRA volumes as input and outputting the motion field between image patches. Using image warping by the estimated motion field, a loss function that imposes image similarity and motion smoothness is adopted to enable training without ground truth motion field. RespME-net is trained patch-wise to circumvent the challenges of training a 3D network volume-wise which requires large amounts of GPU memory and 3D datasets. We perform 5-fold cross-validation with 45 CMRA datasets and demonstrate that RespME-net can predict 3D non-rigid motion fields with subpixel accuracy (0.44 ± 0.38 mm) within ~10 seconds, being ~20 times faster than a GPU-implemented state-of-the-art non-rigid registration method. Moreover, we perform non-rigid motion-compensated CMRA reconstruction for 9 additional patients. The proposed RespME-net has achieved similar motion-corrected CMRA image quality to the conventional registration method regarding coronary artery length and sharpness.

Visual Sensor Fusion With Error Compensation Strategy Toward a Rapid and Low-Cost 3D Scanning System for the Lower Residual Limb

Rapid and low-cost 3D scanning system becomes a common solution for prosthetic designers and engineers to acquire accurate 3D geometrical model of the involved patient. Multiple Kinect sensor fusion solutions have been proposed but still, there is a lack of fusion scheme within a single workstation and a systematic reconstruction error analysis. The objective of the present study was to develop a new Kinect sensor fusion scheme running on a single workstation and an error compensation strategy. Architecture of a 3D scanning system for the lower residual limb was proposed. Four Kinect sensors were used to develop the fusion scheme. An error compensation approach based on linear regression method was performed. Then, an iterative rigid multi-set registration method was proposed. Different primitive geometries (cylindrical objects, box) with well-known physical dimensions were designed, fabricated and used for the evaluation process. Obtained results showed that the error compensation strategy leads to a reduction of around 40% of 3D geometrical reconstruction error for the residual stump-like structure. The total scanning and processing time is 25.4 ± 2.5 seconds. Moreover, our novel iterative rigid multi-set registration method leads to an improvement of accuracy of 60% compared to conventional registration method. Our proposed 3D scanning system using multiple Kinect cameras with error compensation strategy is expected to reduce the design cost and improve the design accuracy. As perspective, this 3D scanning system with fusion scheme will be integrated into a Computer-Aided Design (CAD) system with soft tissue deformation feedbacks for lower limb prostheses.

Progressively Trained Convolutional Neural Networks for Deformable Image Registration.

Deep learning-based methods for deformable image registration are attractive alternatives to conventional registration methods because of their short registration times. However, these methods often fail to estimate larger displacements in complex deformation fields, for which a multi-resolution strategy is required. In this article, we propose to train neural networks progressively to address this problem. Instead of training a large convolutional neural network on the registration task all at once, we initially train smaller versions of the network on lower resolution versions of the images and deformation fields. During training, we progressively expand the network with additional layers that are trained on higher resolution data. We show that this way of training allows a network to learn larger displacements without sacrificing registration accuracy and that the resulting network is less sensitive to large misregistrations compared to training the full network all at once. We generate a large number of ground truth example data by applying random synthetic transformations to a training set of images, and test the network on the problem of intrapatient lung CT registration. We analyze the learned representations in the progressively growing network to assess how the progressive learning strategy influences training. Finally, we show that a progressive training procedure leads to improved registration accuracy when learning large and complex deformations.

Multimodal image registration using histogram of oriented gradient distance and data-driven grey wolf optimizer

Multimodal image registration is becoming increasingly important in remote sensing. However, due to the significant nonlinear intensity differences between multimodal images, conventional registration methods tend to get trapped into local optima. To address this issue, we present a new approach for multimodal image registration using histogram of oriented gradient distance (HOGD) and data-driven grey wolf optimizer (DDGWO). First, we propose a novel similarity measure for area-based registration methods that is HOGD. We investigate the performance of HOGD by analyzing its similarity curve. HOGD has a large range of values, which is helpful to find the global optimum. Second, we use GWO to optimize the transformation parameters. Since it is time-consuming to calculate HOGD, we propose DDGWO to minimize HOGD. In DDGWO, the iterations are divided into two parts: the training and prediction iterations. A support vector machine (SVM) regression model is trained by the historical HOGD computed in the training iterations. The trained SVM model predicts HOGD instead of calculating in the prediction iterations, which can reduce the computational time. Finally, we test the proposed approach that uses HOGD as the similarity measure and DDGWO as the search algorithm on 12 real and four simulated image pairs. Extensive experiments demonstrate that our approach saves up to 83.35-84.15% of computational time and outperforms the state-of-the-art algorithms in terms of registration accuracy.

A Deformable 3D-3D Registration Framework Using Discrete Periodic Spline Wavelet and Edge Position Difference

Neck pain is one of the most common symptoms of cervical spine disease, and segmenting neck muscles to create volumetric measurements may assist clinical diagnosis. While image registration is used to segment medical images, registration is highly challenging for neck muscles due to their tight proximity, shape and size variations among subjects, and similar appearance. These challenges cause conventional multi resolution-based registration methods to be trapped in local minima due to their low degree of freedom geometrical transforms. This article presents a novel object-constrained hierarchical registration framework for aligning inter-subject neck muscles. First, to handle large scale local minima, the proposed framework uses a coarse registration technique, which optimizes the new edge position difference (EPD) similarity measure, to align large mismatches. Also, a new transformation based on the discrete periodic spline wavelet (DPSW), affine and free-form-deformation (FFD) transformations are exploited. Second, to avoid monotonous nature of using transformations in multiple stages, a fine registration technique is designed for aligning small mismatches. This technique uses a double-pushing system by changing edges in the EPD and switching transformation resolutions. The EPD helps in both coarse and fine techniques to implement object-constrained registration via controlling edges, which is not possible when using traditional similarity measures. Experiments are performed on clinical 3D magnetic resonance imaging (MRI) scans of the neck, with the results showing that the EPD is more effective than the mutual information (MI) and sum of squared difference (SSD) measure in terms of volumetric dice similarity coefficient (DSC). Additionally, the proposed method is compared with the diffeomorphic Demons and SyN state-of-the-art approaches with ablation studies in inter-subject deformable registration. The proposed method achieves better accuracy, robustness and consistency than the reference methods, with an average volumetric DSC of 0.7029 compared to 0.6654 and 0.6606 for the Demons and SyN methods, respectively.