Three-dimensional Image Registration Research Articles

Automated three-dimensional image registration for longitudinal photoacoustic imaging.

Photoacoustic tomography (PAT) has great potential in monitoring disease progression and treatment response in breast cancer. However, due to variations in breast repositioning, there is a chance of geometric misalignment between images. Further, poor repositioning can affect light fluence distribution and imaging field-of-view, making images different from one another. The net effect is that it becomes challenging to distinguish between image changes due to repositioning effects and those due to true biological variations. The aim is to develop a three-dimensional image registration framework for geometrically aligning repeated PAT volumetric images, which are potentially affected by repositioning effects such as misalignment, changed radiant exposure conditions, and different fields-of-view. The proposed framework involves the use of a coordinate-based neural network to represent the displacement field between pairs of PAT volumetric images. A loss function based on normalized cross correlation and Frangi vesselness feature extraction at multiple scales was implemented. We refer to our image registration framework as MUVINN-reg, which stands for multiscale vesselness-based image registration using neural networks. The approach was tested on a longitudinal dataset of healthy volunteer breast PAT images acquired with the hybrid photoacoustic-ultrasound Photoacoustic Mammoscope 3 imaging system. The registration performance was also tested under unfavorable repositioning conditions such as intentional mispositioning, and variation in breast-supporting cup size between measurements. A total of 13 pairs of repeated PAT scans were included in this study. MUVINN-reg showed excellent performance in co-registering each pair of images. The proposed framework was shown to be robust to image intensity shifts and field-of-view changes. Furthermore, MUVINN-reg could align vessels at imaging depths greater than 4cm. The proposed framework will enable the use of PAT for quantitative and reproducible monitoring of disease progression and treatment response.

A Multistage Rigid-Affine-Deformable Network for Three-Dimensional Multimodal Medical Image Registration

Multimodal image registration is an important component of medical image processing, allowing the integration of complementary information from various imaging modalities to improve clinical applications like diagnosis and treatment planning. We proposed a novel multistage neural network for three-dimensional multimodal medical image registration, which addresses the challenge of larger rigid deformations commonly present in medical images due to variations in patient positioning in different scanners and rigid anatomical structures. This multistage network combines rigid, affine and deformable transformations in three stages. The network was trained unsupervised with Mutual Information and Gradient L2 loss. We compared the results of our proposed multistage network with a rigid-affine-deformable registration with the classical registration method NiftyReg as a baseline and a multistage network, which combines affine and deformable transformation, as a benchmark. To evaluate the performance of the proposed multistage network, we used four three-dimensional multimodal in vivo datasets: three renal MR datasets consisting of T1-weighted and T2-weighted MR scans and one liver dataset containing CT and T1-weighted MR scans. Experimental results showed that combining rigid, affine and deformable transformations in a multistage network leads to registration results with a high structural similarity, overlap of the corresponding structures (Dice: 76.7 ± 12.5, 61.1 ± 14.0, 64.8 ± 16.2, 68.1 ± 24.6 for the four datasets) and a low level of image folding (|J| ≤ 0: less than or equal to 1.1%), resulting in a medical plausible registration result.

A multi-stack registration technique to improve measurement accuracy and precision across longitudinal HR-pQCT scans.

Recent applications of high-resolution peripheral quantitative computed tomography (HR-pQCT) have demonstrated that changes in local bone remodelling can be quantified in vivo using longitudinal three-dimensional image registration. However, certain emerging applications, such as fracture healing and joint analysis, require larger multi-stack scan regions that can result in stack shift image artifacts. These artifacts can be detrimental to the accurate alignment of the bone structure across multiple timepoints. The purpose of this study was to establish a multi-stack registration protocol for evaluating longitudinal HR-pQCT images and to assess the accuracy and precision error in comparison with measures obtained using previously established three-dimensional longitudinal registration. Three same day multi-stack HR-pQCT scans of the radius (2 stacks in length) and tibia (3 stacks in length) were obtained from 39 healthy individuals who participated in a previous reproducibility study. A fully automated multi-stack registration algorithm was developed to re-align stacks within a scan by leveraging slight offsets between longitudinal scans. Stack shift severity before and after registration was quantified using a newly proposed stack-shift severity score. The false discovery rate for bone remodelling events and precision error of bone morphology and micro-finite element analysis parameters were compared between longitudinally registered scans with and without the addition of multi-stack registration. Most scans (82%) improved in stack alignment or maintained the lowest stack shift severity score when multi-stack registration was implemented. The false discovery rate of bone remodelling events significantly decreased after multi-stack registration, resulting in median false detection of bone formation and resorption fractions between 3.2 to 7.5% at the radius and 3.4 to 5.3% at the tibia. Further, precision error was significantly reduced or remained unchanged in all standard bone morphology and micro-finite element analysis parameters, except for total and trabecular cross-sectional areas. Multi-stack registration is an effective strategy for accurately aligning multi-stack HR-pQCT scans without modification of the image acquisition protocol. The algorithm presented here is a viable approach for performing accurate morphological analysis on multi-stack HR-pQCT scans, particularly for advanced application investigating local bone remodelling in vivo.

3D Biological/Biomedical Image Registration with enhanced Feature Extraction and Outlier Detection.

In various applications, such as computer vision, medical imaging and robotics, three-dimensional (3D) image registration is a significant step. It enables the alignment of various datasets into a single coordinate system, consequently providing a consistent perspective that allows further analysis. By precisely aligning images we can compare, analyze, and combine data collected in different situations. This paper presents a novel approach for 3D or z-stack microscopy and medical image registration, utilizing a combination of conventional and deep learning techniques for feature extraction and adaptive likelihood-based methods for outlier detection. The proposed method uses the Scale-invariant Feature Transform (SIFT) and the Residual Network (ResNet50) deep neural learning network to extract effective features and obtain precise and exhaustive representations of image contents. The registration approach also employs the adaptive Maximum Likelihood Estimation SAmple Consensus (MLESAC) method that optimizes outlier detection and increases noise and distortion resistance to improve the efficacy of these combined extracted features. This integrated approach demonstrates robustness, flexibility, and adaptability across a variety of imaging modalities, enabling the registration of complex images with higher precision. Experimental results show that the proposed algorithm outperforms state-of-the-art image registration methods, including conventional SIFT, SIFT with Random Sample Consensus (RANSAC), and Oriented FAST and Rotated BRIEF (ORB) methods, as well as registration software packages such as bUnwrapJ and TurboReg, in terms of Mutual Information (MI), Phase Congruency-Based (PCB) metrics, and Gradiant-based metrics (GBM), using 3D MRI and 3D serial sections of multiplex microscopy images.

A bidirectional registration neural network for cardiac motion tracking using cine MRI images

Using cine magnetic resonance imaging (cine MRI) images to track cardiac motion helps users to analyze the myocardial strain, and is of great importance in clinical applications. At present, most of the automatic deep learning-based motion tracking methods compare two images without considering temporal information between MRI frames, which easily leads to the lack of consistency of the generated motion fields. Even though a small number of works take into account the temporal factor, they are usually computationally intensive or have limitations on image length. To solve this problem, we propose a bidirectional convolution neural network for motion tracking of cardiac cine MRI images. This network leverages convolutional blocks to extract spatial features from three-dimensional (3D) image registration pairs, and models the temporal relations through a bidirectional recurrent neural network to obtain the Lagrange motion field between the reference image and other images. Compared with previous pairwise registration methods, the proposed method can automatically learn spatiotemporal information from multiple images with fewer parameters. We evaluated our model on three public cardiac cine MRI datasets. The experimental results demonstrated that the proposed method can significantly improve the motion tracking accuracy. The average Dice coefficient between estimated segmentation and manual segmentation has reached almost 0.85 on the widely used Automatic Cardiac Diagnostic Challenge (ACDC) dataset.

A Review of 3D-2D Registration Methods and Applications based on Medical Images

The registration of preoperative three-dimensional (3D) medical images with intraoperative two-dimensional (2D) data is a key technology for image-guided radiotherapy, minimally invasive surgery, and interventional procedures. In this paper, we review 3D-2D registration methods using computed tomography (CT) and magnetic resonance imaging (MRI) as preoperative 3D images and ultrasound, X-ray, and visible light images as intraoperative 2D images. The 3D-2D registration techniques are classified into intensity-based, structure-based, and gradient-based according to the different registration features. In addition, we investigated the different application scenarios of this registration technology in medical clinical treatment, which can be divided into disease diagnosis, surgical guidance and postoperative evaluation, and also investigated the evaluation method of 3D-2D registration effect.

AMNet: Adaptive multi-level network for deformable registration of 3D brain MR images.

Three-dimensional (3D) deformable image registration is a fundamental technique in medical image analysis tasks. Although it has been extensively investigated, current deep-learning-based registration models may face the challenges posed by deformations with various degrees of complexity. This paper proposes an adaptive multi-level registration network (AMNet) to retain the continuity of the deformation field and to achieve high-performance registration for 3D brain MR images. First, we design a lightweight registration network with an adaptive growth strategy to learn deformation field from multi-level wavelet sub-bands, which facilitates both global and local optimization and achieves registration with high performance. Second, our AMNet is designed for image-wise registration, which adapts the local importance of a region in accordance with the complexity degrees of its deformation, and thereafter improves the registration efficiency and maintains the continuity of the deformation field. Experimental results from five publicly-available brain MR datasets and a synthetic brain MR dataset show that our method achieves superior performance against state-of-the-art medical image registration approaches.

Deformable image registration based on single or multi-atlas methods for automatic muscle segmentation and the generation of augmented imaging datasets.

Muscle segmentation is a process relied upon to gather medical image-based muscle characterisation, useful in directly assessing muscle volume and geometry, that can be used as inputs to musculoskeletal modelling pipelines. Manual or semi-automatic techniques are typically employed to segment the muscles and quantify their properties, but they require significant manual labour and incur operator repeatability issues. In this study an automatic process is presented, aiming to segment all lower limb muscles from Magnetic Resonance (MR) imaging data simultaneously using three-dimensional (3D) deformable image registration (single inputs or multi-atlas). Twenty-three of the major lower limb skeletal muscles were segmented from five subjects, with an average Dice similarity coefficient of 0.72, and average absolute relative volume error (RVE) of 12.7% (average relative volume error of -2.2%) considering the optimal subject combinations. The multi-atlas approach showed slightly better accuracy (average DSC: 0.73; average RVE: 1.67%). Segmented MR imaging datasets of the lower limb are not widely available in the literature, limiting the potential of new, probabilistic methods such as deep learning to be used in the context of muscle segmentation. In this work, Non-linear deformable image registration is used to generate 69 manually checked, segmented, 3D, artificial datasets, allowing access for future studies to use these new methods, with a large amount of reliable reference data.

Enhancing the depth perception of DSA images with 2D-3D registration.

Today, cerebrovascular disease has become an important health hazard. Therefore, it is necessary to perform a more accurate and less time-consuming registration of preoperative three-dimensional (3D) images and intraoperative two-dimensional (2D) projection images which is very important for conducting cerebrovascular disease interventions. The 2D-3D registration method proposed in this study is designed to solve the problems of long registration time and large registration errors in 3D computed tomography angiography (CTA) images and 2D digital subtraction angiography (DSA) images. To make a more comprehensive and active diagnosis, treatment and surgery plan for patients with cerebrovascular diseases, we propose a weighted similarity measure function, the normalized mutual information-gradient difference (NMG), which can evaluate the 2D-3D registration results. Then, using a multi-resolution fusion optimization strategy, the multi-resolution fused regular step gradient descent optimization (MR-RSGD) method is presented to attain the optimal value of the registration results in the process of the optimization algorithm. In this study, we adopt two datasets of the brain vessels to validate and obtain similarity metric values which are 0.0037 and 0.0003, respectively. Using the registration method proposed in this study, the time taken for the experiment was calculated to be 56.55s and 50.8070s, respectively, for the two sets of data. The results show that the registration methods proposed in this study are both better than the Normalized Mutual (NM) and Normalized Mutual Information (NMI). The experimental results in this study show that in the 2D-3D registration process, to evaluate the registration results more accurately, we can use the similarity metric function containing the image gray information and spatial information. To improve the efficiency of the registration process, we can choose the algorithm with gradient optimization strategy. Our method has great potential to be applied in practical interventional treatment for intuitive 3D navigation.

A novel technique to quantify bone-to-implant contact of zygomatic implants: a radiographic analysis based on three-dimensional image registration and segmentation.

The purpose of this study is to establish a novel, reproducible technique to obtain the BIC area (BICA) between zygomatic implants and zygomatic bone based on post-operative cone-beam computed tomography (CBCT) images. Three-dimensional (3D) image registration and segmentation were used to eliminate the effect of metal-induced artifacts of zygomatic implants. An ex-vivo study was included to verify the feasibility of the new method. Then, the radiographic bone-to-implant contact (rBIC) of 143 implants was measured in a total of 50 patients. To obtain the BICA of zygomatic implants and the zygomatic bone, several steps were necessary, including image preprocessing of CBCT scans, identification of the position of zygomatic implants, registration, and segmentation of pre- and post-operative CBCT images, and 3D reconstruction of models. The conventional two-dimensional (2D) linear rBIC (rBICc) measurement method with post-operative CBCT images was chosen as a comparison. The mean values of rBIC and rBICc were 15.08 ± 5.92 mm and 14.77 ± 5.14 mm, respectively. A statistically significant correlation was observed between rBIC and rBICc values ([Formula: see text]=0.86, p < 0.0001). This study proposed a standardized, repeatable, noninvasive technique to quantify the rBIC of post-operative zygomatic implants in 3D terms. This technique is comparable to conventional 2D linear measurements and seems to be more reliable than these conventional measurements; thus, this method could serve as a valuable tool in the performance of clinical research protocols.

Similarity attention-based CNN for robust 3D medical image registration

In recent years, deep learning (DL)-based registration technology has significantly improved the calculation speed of medical image registration. Existing DL-based registration methods generally use raw data features to predict the deformation field. However, this strategy may not be very effective for difficult registration tasks. Hence, in this study, we propose a similarity attention-based convolutional neural network (CNN) for accurate and robust three-dimensional medical image registration. We first introduce a similarity-based local attention model as an auxiliary module for building a displacement searching space, instead of a direct displacement prediction based on raw data. The proposed model can help the network focus on spatial correspondences with high similarities and ignore those with low similarities. A multi-scale CNN is then integrated with the similarity-based local attention for providing non-local attention, lightweight network, and coarse-to-fine registration. We evaluated the proposed method for various applications, such as the registration of large-scope abdominal computerized tomography (CT) images and chest CT images acquired at different respiratory phases, and atlas registration in magnetic resonance imaging. The experimental results demonstrate that the proposed method can provide a more accurate and robust registration performance than state-of-the-art registration methods.

Multiscale Approach for Three-Dimensional Conformal Image Registration

Image registration, especially for three-dimensional (3D) image registration, is widely used in clinical medicine. Many 3D image registration models have been proposed during the last decades. All these models achieve the minimum of the cost functional with some prior regularization. In addition, the physical mesh folding phenomenon is not taken into consideration in most models. This raises a question of whether one can achieve/approach the infimum of the cost functional without a regularization term and ensure no mesh folding. To give an answer to this question, a multiscale approach for 3D conformal image registration is presented in this paper. This approach ensures no mesh folding and gets close to the infimum of the cost functional without any regularization on the 3D conformal set. The 3D multiscale approach contains a series of deformation composition processes and the convergence of the process is presented. Furthermore, a numerical algorithm for this multiscale approach is proposed and the convergence of the numerical algorithm is proved. Moreover, several numerical tests are also listed to show the good performance of the proposed algorithm.

A 3D Image Registration Method for Laparoscopic Liver Surgery Navigation

At present, laparoscopic augmented reality (AR) navigation has been applied to minimally invasive abdominal surgery, which can help doctors to see the location of blood vessels and tumors in organs, so as to perform precise surgery operations. Image registration is the process of optimally mapping one or more images to the target image, and it is also the core of laparoscopic AR navigation. The key is how to shorten the registration time and optimize the registration accuracy. We have studied the three-dimensional (3D) image registration technology in laparoscopic liver surgery navigation and proposed a new registration method combining rough registration and fine registration. First, the adaptive fireworks algorithm (AFWA) is applied to rough registration, and then the optimized iterative closest point (ICP) algorithm is applied to fine registration. We proposed a method that is validated by the computed tomography (CT) dataset 3D-IRCADb-01. Experimental results show that our method is superior to other registration methods based on stochastic optimization algorithms in terms of registration time and accuracy.

Optical coherence tomography evaluation of deep dentin crack removal techniques

BackgroundThis in vitro study used optical coherence tomography for noninvasive evaluation of the effectiveness of current clinical techniques to remove deep coronal dentin cracks. MethodsStandard dentin cracks were induced on the pulpal floor of 40 decoronated, extracted, sound human posterior teeth using a diamond disk, resembling cracks extending from marginal ridges. The specimens were randomly assigned to 1 of 5 treatment groups for crack removal using airborne-particle abrasion or a bur (fissure, small round, medium round, or tapered fine diamond). Optical coherence tomographic scans were obtained before and after the crack removal. Three-dimensional image registration analyzed the amount of dentin removed and the dimensions of cracks initiated or propagated in each treatment group. ResultsParticle abrasion resulted in the smallest crack propagation in each dimension, which was significantly different from all bur groups (Mann-Whitney, P < .01). Removed dentin depth and width differed among burs (P < .05). All burs resulted in a degree of crack formation, and there was no difference in crack dimensions among bur groups (P > .05). The amount of removed dentin during crack treatment was the largest in the particle abrasion group, which was significantly different from those of the bur groups (Mann-Whitney, P < .005). ConclusionsDental burs used with the purpose of removing pulpal floor dentin cracks induced new small cracks and extended existing cracks, and the volume of removed dentin depended on the shape and size of the bur. Air particle abrasion induced the fewest new cracks despite removing larger dentin volume.

Medical image alignment based on landmark- and approximate contour-matching.

Purpose: Our goal is to propose a landmark- and contour-matching (LCM) registration method that uses both landmark information and approximate point correspondences to boost the similarity between image pairs with sparse landmark information. Approach: A model for registering two-dimensional (2D) medical images with landmark information and contour-approximating landmarks was proposed. The model was also extended to accommodate the registration of three-dimensional (3D) cardiac images. We validated the LCM method on 2D hand x-rays and 3D porcine cardiac magnetic resonance images. The following metrics were used to assess the quality of specific aspects of the registered images: Dice similarity coefficient for the overall image overlap, target registration error for pointwise correspondence, and interior angle for local curvature. Results: Target registrations were reduced from 27.12 to 0.01mm post-LCM registration. Implementing the proposed algorithm also led to a 112% average improvement in image similarity in terms of Dice coefficients. In addition, interior angle measurements indicate that the proposed method preserved the local curvature at major reference landmarks and mitigated the appearance of deformities in the registered images. Conclusions: The proposed method addressed several issues associated with purely landmark-based techniques, such as iterative closest point registration and thin plate spline interpolation. Furthermore, it provided accurate registration results even in the presence of landmark localization errors.

A review of deep learning-based three-dimensional medical image registration methods.

Medical image registration is a vital component of many medical procedures, such as image-guided radiotherapy (IGRT), as it allows for more accurate dose-delivery and better management of side effects. Recently, the successful implementation of deep learning (DL) in various fields has prompted many research groups to apply DL to three-dimensional (3D) medical image registration. Several of these efforts have led to promising results. This review summarized the progress made in DL-based 3D image registration over the past 5 years and identify existing challenges and potential avenues for further research. The collected studies were statistically analyzed based on the region of interest (ROI), image modality, supervision method, and registration evaluation metrics. The studies were classified into three categories: deep iterative registration, supervised registration, and unsupervised registration. The studies are thoroughly reviewed and their unique contributions are highlighted. A summary is presented following a review of each category of study, discussing its advantages, challenges, and trends. Finally, the common challenges for all categories are discussed, and potential future research topics are identified.

An Orthopedic Auxiliary Diagnosis System Based on Image Recognition Technology.

There are many kinds of orthopedic diseases with complex professional background, and it is easy to miss diagnosis and misdiagnosis. The computer-aided diagnosis system of orthopedic diseases based on the key technology of medical image processing can locate and display the lesion location area by visualization, measuring and providing disease diagnosis indexes. It is of great significance to assist orthopedic doctors to diagnose orthopedic diseases from the perspective of visual vision and quantitative indicators, which can improve the diagnosis rate and accuracy of orthopedic diseases, reduce the pain of patients, and shorten the treatment time of diseases. To solve the problem of possible spatial inconsistency of medical images of orthopedic diseases, we propose an image registration method based on volume feature point selection and Powell. Through the linear search strategy of golden section method and Powell algorithm optimization, the best spatial transformation parameters are found, which maximizes the normalized mutual information between images to be registered, thus ensuring the consistency of two-dimensional spatial positions. According to the proposed algorithm, a computer-aided diagnosis system of orthopedic diseases is developed and designed independently. The system consists of five modules, which can complete many functions such as medical image input and output, algorithm processing, and effect display. The experimental results show that the system developed in this paper has good results in cartilage tissue segmentation, bone and urate agglomeration segmentation, urate agglomeration artifact removal, two-dimensional and three-dimensional image registration, and visualization. The system can be applied to clinical gout and cartilage defect diagnosis and evaluation, providing sufficient basis to assist doctors in making diagnosis decisions.

A 3D adult zebrafish brain atlas (AZBA) for the digital age.

Zebrafish have made significant contributions to our understanding of the vertebrate brain and the neural basis of behavior, earning a place as one of the most widely used model organisms in neuroscience. Their appeal arises from the marriage of low cost, early life transparency, and ease of genetic manipulation with a behavioral repertoire that becomes more sophisticated as animals transition from larvae to adults. To further enhance the use of adult zebrafish, we created the first fully segmented three-dimensional digital adult zebrafish brain atlas (AZBA). AZBA was built by combining tissue clearing, light-sheet fluorescence microscopy, and three-dimensional image registration of nuclear and antibody stains. These images were used to guide segmentation of the atlas into over 200 neuroanatomical regions comprising the entirety of the adult zebrafish brain. As an open source, online (azba.wayne.edu), updatable digital resource, AZBA will significantly enhance the use of adult zebrafish in furthering our understanding of vertebrate brain function in both health and disease.

Uncertainty quantification in non-rigid image registration via stochastic gradient Markov chain Monte Carlo

We develop a new Bayesian model for non-rigid registration of three-dimensional medical images, with a focus on uncertainty quantification. Probabilistic registration of large images with calibrated uncertainty estimates is difficult for both computational and modelling reasons. To address the computational issues, we explore connections between the Markov chain Monte Carlo by backpropagation and the variational inference by backpropagation frameworks, in order to efficiently draw samples from the posterior distribution of transformation parameters. To address the modelling issues, we formulate a Bayesian model for image registration that overcomes the existing barriers when using a dense, high-dimensional, and diffeomorphic transformation parametrisation. This results in improved calibration of uncertainty estimates. We compare the model in terms of both image registration accuracy and uncertainty quantification to VoxelMorph, a state-of-the-art image registration model based on deep learning.

An Abdominal Registration Technology for Integration of Nanomaterial Imaging-Aided Diagnosis and Treatment.

Image registration technology is a key technology used in the process of nanomaterial imaging-aided diagnosis and targeted therapy effect monitoring for abdominal diseases. Recently, the deep-learning based methods have been increasingly used for large-scale medical image registration, because their iteration is much less than those of traditional ones. In this paper, a coarse-to-fine unsupervised learning-based three-dimensional (3D) abdominal CT image registration method is presented. Firstly, an affine transformation was used as an initial step to deal with large deformation between two images. Secondly, an unsupervised total loss function containing similarity, smoothness, and topology preservation measures was proposed to achieve better registration performances during convolutional neural network (CNN) training and testing. The experimental results demonstrated that the proposed method severally obtains the average MSE, PSNR, and SSIM values of 0.0055, 22.7950, and 0.8241, which outperformed some existing traditional and unsupervised learning-based methods. Moreover, our method can register 3D abdominal CT images with shortest time and is expected to become a real-time method for clinical application.