Accurate Image Registration Research Articles

Deep learning based super-resolution for CBCT dose reduction in radiotherapy.

Cone-beam computed tomography (CBCT) is a crucial daily imaging modality in image-guided and adaptive radiotherapy. However, the use of ionizing radiation in CBCT imaging increases the risk of secondary cancers, which is particularly concerning for pediatric patients. Deep learning super-resolution has shown promising results in enhancing the resolution of photographic and medical images but has not yet been explored in the context of CBCT dose reduction. To facilitate CBCT imaging dose reduction, we propose using an enhanced super-resolution generative adversarial network (ESRGAN) in both the projection and image domains to restore the image quality of low-dose CBCT. An extensive projection database, containing 2997 CBCT scans from head and neck cancer patients, was used to train two different ESRGAN models to generate super-resolution CBCTs. One model operated in the projection domain, using pairs of simulated low-resolution (low-dose) and original high-resolution (high-dose) projections and yielded CBCTSRpro. The other model operated in the image domain, using pairs of axial slices from reconstructed low-resolution and high-resolution CBCTs (CBCTLR and CBCTHR) and resulted in CBCTSRimg. Super-resolution CBCTs were evaluated in terms of image similarity (MAE, ME, PSNR, and SSIM), noise characteristics, spatial resolution, and registration accuracy, using the original CBCT as a reference. To test the perceptual difference between the original and super-resolution CBCT, we performed a visual Turing test. Visually, both super-resolution approaches in the projection and image domains improved the image quality of low-dose CBCTs. This was confirmed by the visual Turing test, that showed low accuracy, sensitivity, and specificity, indicating almost no perceptual difference between CBCTHR and the super-resolution CBCTs. CBCTSRimg (accuracy: 0.55, sensitivity: 0.59, specificity: 0.50) performed slightly better than CBCTSRpro (accuracy: 0.59, sensitivity: 0.61, specificity: 0.57). Image similarity metrics were affected by varying noise levels and did not reflect the visual improvements, with MAE/ME/PSNR/SSIM values of 110.4HU/2.9HU/40.4dB/0.82 for CBCTLR, 136.6HU/-0.4HU/38.6dB/0.77 for CBCTSRpro, and 128.2HU/1.9HU/39.0dB/0.80 for CBCTSRimg. In terms of spatial resolution, quantified by calculating 10% levels of the task transfer function, both CBCTSRpro and CBCTSRimg outperformed CBCTLR and nearly matched the reference CBCTHR (CBCTLR: 0.66lp/mm, CBCTSRpro: 0.88lp/mm, CBCTSRimg: 0.95lp/mm, CBCTHR: 1.01lp/mm). Noise characteristics of CBCTSRimg and CBCTSRpro were comparable to the reference CBCTHR. Registration parameters showed negligible differences for all CBCTs (CBCTLR, CBCTSRpro, CBCTSRimg), with average absolute differences in registration parameters being below 0.4° for rotations and below 0.06mm for translations (CBCTHR as reference). This study demonstrates that deep learning can be a valuable tool for CBCT dose reduction in CBCT-guided radiotherapy by acquiring low-dose CBCTs and restoring the image quality using deep learning super-resolution. The results suggest that higher quality images can be generated when super-resolution is performed in the image domain compared to the projection domain.

Enhanced multi-modal abdominal image registration via structural awareness and region-specific optimization

Multi-modal abdominal image registration is a critical step in achieving accurate diagnosis and treatment. However, the large field of view, complex organ structures, and significant anatomical variability in the abdominal region pose substantial challenges to precise multi-modal registration. To address these challenges, we propose a novel framework based on structural awareness and region-specific optimization, which progressively enhances registration accuracy through a hierarchical strategy. The core innovation lies in introducing the Modality Independent Structural Awareness (MISA) loss, which comprises two key components: first, it ensures structural consistency across modalities by deeply understanding the critical anatomical structures; second, it employs a region-specific weighting strategy to prioritize key anatomical regions, such as organ boundaries, thereby achieving high-precision, region-specific constraints in multi-modal registration. Experimental results demonstrate that our method outperforms state-of-the-art approaches on two public multi-modal abdominal datasets, particularly in accurately capturing complex anatomical structures and maintaining tissue consistency. Further analysis confirms that the MISA loss and its two innovative components are pivotal in enhancing multi-modal abdominal image registration accuracy.

A vessel bifurcation liver CT landmark pair dataset for evaluating deformable image registration algorithms.

Evaluating deformable image registration (DIR) algorithms is vital for enhancing algorithm performance and gaining clinical acceptance. However, there is a notable lack of dependable DIR benchmark datasets for assessing DIR performance except for lung images. To address this gap, we aim to introduce our comprehensive liver computed tomography (CT) DIR landmark dataset library. This dataset is designed for efficient and quantitative evaluation of various DIR methods for liver CTs, paving the way for more accurate and reliable image registration techniques. Forty CT liver image pairs were acquired from several publicly available image archives and authors' institutions under institutional review board (IRB) approval. The images were processed with a semi-automatic procedure to generate landmark pairs: (1) for each case, liver vessels were automatically segmented on one image; (2) landmarks were automatically detected at vessel bifurcations; (3) corresponding landmarks in the second image were placed using two deformable image registration methods to avoid algorithm-specific biases; (4) a comprehensive validation process based on quantitative evaluation and manual assessment was applied to reject outliers and ensure the landmarks' positional accuracy. This workflow resulted in an average of ∼56 landmark pairs per image pair, comprising a total of 2220 landmarks for 40 cases. The general landmarking accuracy of this procedure was evaluated using digital phantoms and manual landmark placement. The landmark pair target registration errors (TRE) on digital phantoms were 0.37±0.26 and 0.55±0.34mm respectively for the two selected DIR algorithms used in our workflow, with 97% of landmark pairs having TREs below 1.5mm. The distances from the calculated landmarks to the averaged manual placement were 1.27±0.79mm. All data, including image files and landmark information, are publicly available at Zenodo (https://zenodo.org/records/13738577). Instructions for using our data can be found on our GitHub page at https://github.com/deshanyang/Liver-DIR-QA. The landmark dataset generated in this work is the first collection of large-scale liver CT DIR landmarks prepared on real patient images. This dataset can provide researchers with a dense set of ground truth benchmarks for the quantitative evaluation of DIR algorithms within the liver.

Research on registration method for enface image using multi-feature fusion

Objective.The purpose of this work is to accurately and quickly register the Optical coherence tomography (OCT) projection (enface) images at adjacent time points, and to solve the problem of interference caused by CNV lesions on the registration features.Approach.In this work, a multi-feature registration strategy was proposed, in which a combined feature (com-feature) containing 3D information, intersection information and SURF feature was designed. Firstly, the coordinates of all feature points were extracted as combined features, and then these feature coordinates were added to the initial vascular coordinate set simplified by the Douglas-Peucker algorithm as the point set for registration. Finally, the coherent point drift registration algorithm was used to register the enface coordinate point sets of adjacent time series.Main results.The newly designed features significantly improve the success rate of global registration of vascular networks in enface images, while the simplification step greatly improves the registration speed on the basis of preserving vascular features. The MSE, DSC and time complexity of the proposed method are 0.07993, 0.9693 and 42.7016 s, respectively.Significance.CNV is a serious retinal disease in ophthalmology. The registration of OCT enface images at adjacent time points can timely monitor the progress of the disease and assist doctors in making diagnoses. The proposed method not only improves the accuracy of OCT enface image registration, but also significantly reduces the time complexity. It has good registration results in clinical routine and provides a more efficient method for clinical diagnosis and treatment.

A novel framework for elucidating the effect of mechanical loading on the geometry of ovariectomized mouse tibiae using principal component analysis.

Murine models are used to test the effect of anti-osteoporosis treatments as they replicate some of the bone phenotypes observed in osteoporotic (OP) patients. The effect of disease and treatment is typically described as changes in bone geometry and microstructure over time. Conventional assessment of geometric changes relies on morphometric scalar parameters. However, being correlated with each other, these parameters do not describe separate fractions of variations and offer only a moderate insight into temporal changes. The current study proposes a novel image-based framework that employs deformable image registration on in vivo longitudinal images of bones and Principal Component Analysis (PCA) for improved quantification of geometric effects of OP treatments. This PCA-based model and a novel post-processing of score changes provide orthogonal modes of shape variations temporally induced by a course of treatment (specifically in vivo mechanical loading). Errors associated with the proposed framework are rigorously quantified and it is shown that the accuracy of deformable image registration in capturing the bone shapes (∼1 voxel = 10.4μm) is of the same order of magnitude as the relevant state-of-the-art evaluation studies. Applying the framework to longitudinal image data from the midshaft section of ovariectomized mouse tibia, two mutually orthogonal mode shapes are reliably identified to be an effect of treatment. The mode shapes captured changes of the tibia geometry due to the treatment at the anterior crest (maximum of 0.103mm) and across the tibia midshaft section and the posterior (0.030mm) and medial (0.024mm) aspects. These changes agree with those reported previously but are now described in a compact fashion, as a vector field of displacements on the bone surface. The proposed framework enables a more detailed investigation of the effect of disease and treatment on bones in preclinical studies and boosts the precision of such assessments.

High-precision geometric positioning for optical remote sensing satellite in dynamic imaging

ABSTRACT Dynamic imaging of optical remote sensing satellites refers to the active acquisition of images while the satellite is maneuvering at a high angular velocity, which significantly enhances the efficiency and application value of remote sensing imagery. However, due to the influence of rapid maneuvering, the random error of star sensors significantly increases, resulting in a decrease in the geometric positioning accuracy of dynamic imaging remote sensing images. This paper proposes a dynamic fusion method for multisource attitude measurement data based on the noise adaptive estimation and bidirectional filter to achieve high-precision attitude determination and geometric positioning in dynamic imaging. Based on the measurement error model of star sensors, the weights of the star sensor and gyroscope are adaptively adjusted in multisource data fusion to reduce the impact of star sensor measurement errors on the gyroscope. Moreover, a bidirectional fusion filter that includes a low-velocity maneuvering stage is proposed to realize the optimal estimation of the satellite attitude parameter. The simulation data and onboard data of the Luojia3-01 (LJ3–01) satellite were tested to verify the effectiveness of the proposed method. The geometric positioning accuracy of the staring images of LJ3–01 improved from 10.048 m to 7.538 m. The registration accuracy of the sequential images improved from 3.568 pixels to 1.179 pixels. The proposed method can significantly improve the attitude determination accuracy and the geometric positioning accuracy of LJ3–01 satellite staring images. Moreover, for simulation data with various angular velocities, the attitude determination accuracies of the proposed method are better than 0.93”. The experimental results show that the proposed method can achieve high-precision attitude determination in dynamic imaging, reaching the accuracy in the traditional passive imaging.

OS-PSO: A Modified Ratio of Exponentially Weighted Averages-Based Optical and SAR Image Registration.

Optical and synthetic aperture radar (SAR) images exhibit non-negligible intensity differences due to their unique imaging mechanisms, which makes it difficult for classical SIFT-based algorithms to obtain sufficiently correct correspondences when processing the registration of these two types of images. To tackle this problem, an accurate optical and SAR image registration algorithm based on the SIFT algorithm (OS-PSO) is proposed. First, a modified ratio of exponentially weighted averages (MROEWA) operator is introduced to resolve the sudden dark patches in SAR images, thus generating more consistent gradients between optical and SAR images. Next, we innovatively construct the Harris scale space to replace the traditional difference in the Gaussian (DoG) scale space, identify repeatable key-points by searching for local maxima, and perform localization refinement on the identified key-points to improve their accuracy. Immediately after that, the gradient location orientation histogram (GLOH) method is adopted to construct the feature descriptors. Finally, we propose an enhanced matching method. The transformed relation is obtained in the initial matching stage using the nearest neighbor distance ratio (NNDR) and fast sample consensus (FSC) methods. And the re-matching takes into account the location, scale, and main direction of key-points to increase the number of correctly corresponding points. The proposed OS-PSO algorithm has been implemented on the Gaofen and Sentinel series with excellent results. The superior performance of the designed registration system can also be applied in complex scenarios, including urban, suburban, river, farmland, and lake areas, with more efficiency and accuracy than the state-of-the-art methods based on the WHU-OPT-SAR dataset and the BISTU-OPT-SAR dataset.

Longitudinal registration of T1-weighted breast MRI: A registration algorithm (FLIRE) and clinical application

PurposeMRI is commonly used to aid breast cancer diagnosis and treatment evaluation. For patients with breast cancer, neoadjuvant chemotherapy aims to reduce the tumor size and extent of surgery necessary. The current clinical standard to measure breast tumor response on MRI uses the longest tumor diameter. Radiologists also account for other tissue properties including tumor contrast or pharmacokinetics in their assessment. Accurate longitudinal image registration of breast tissue is critical to properly compare response to treatment at different timepoints. MethodsIn this study, a deformable Fast Longitudinal Image Registration (FLIRE) algorithm was optimized for breast tissue. FLIRE was then compared to the publicly available software packages with high accuracy (DRAMMS) and fast runtime (Elastix). Patients included in the study received longitudinal T1-weighted MRI without fat saturation at two to six timepoints as part of asymptomatic screening (n = 27) or throughout neoadjuvant chemotherapy treatment (n = 32). T1-weighted images were registered to the first timepoint with each algorithm. ResultsAlignment and runtime performance were compared using two-way repeated measure ANOVAs (P < 0.05). Across all patients, Pearson's correlation coefficient across the entire image volume was slightly higher with statistical significance and had less variance for FLIRE (0.98 ± 0.01 stdev) compared to DRAMMS (0.97 ± 0.03 stdev) and Elastix (0.95 ± 0.03 stdev). Additionally, FLIRE runtime (10.0 mins) was 9.0 times faster than DRAMMS (89.6 mins) and 1.5 times faster than Elastix (14.5 mins) on a Linux workstation. ConclusionFLIRE demonstrates promise for time-sensitive clinical applications due to its accuracy, robustness across patients and timepoints, and speed.

NCNet: Deformable medical image registration network based on neighborhood cross-attention combined with multi-resolution constraints

Objective. Existing registration networks based on cross-attention design usually divide the image pairs to be registered into patches for input. The division and merging operations of a series of patches are difficult to maintain the topology of the deformation field and reduce the interpretability of the network. Therefore, our goal is to develop a new network architecture based on a cross-attention mechanism combined with a multi-resolution strategy to improve the accuracy and interpretability of medical image registration. Approach. We propose a new deformable image registration network NCNet based on neighborhood cross-attention combined with multi-resolution strategy. The network structure mainly consists of a multi-resolution feature encoder, a multi-head neighborhood cross-attention module and a registration decoder. The hierarchical feature extraction capability of our encoder is improved by introducing large kernel parallel convolution blocks; the cross-attention module based on neighborhood calculation is used to reduce the impact on the topology of the deformation field and double normalization is used to reduce its computational complexity. Main result. We performed atlas-based registration and inter-subject registration tasks on the public 3D brain magnetic resonance imaging datasets LPBA40 and IXI respectively. Compared with the popular VoxelMorph method, our method improves the average DSC value by 7.9% and 3.6% on LPBA40 and IXI. Compared with the popular TransMorph method, our method improves the average DSC value by 4.9% and 1.3% on LPBA40 and IXI. Significance. We proved the advantages of the neighborhood attention calculation method compared to the window attention calculation method based on partitioning patches, and analyzed the impact of the pyramid feature encoder and double normalization on network performance. This has made a valuable contribution to promoting the further development of medical image registration methods.

A Review of Medical Image Registration for Different Modalities.

Medical image registration has become pivotal in recent years with the integration of various imaging modalities like X-ray, ultrasound, MRI, and CT scans, enabling comprehensive analysis and diagnosis of biological structures. This paper provides a comprehensive review of registration techniques for medical images, with an in-depth focus on 2D-2D image registration methods. While 3D registration is briefly touched upon, the primary emphasis remains on 2D techniques and their applications. This review covers registration techniques for diverse modalities, including unimodal, multimodal, interpatient, and intra-patient. The paper explores the challenges encountered in medical image registration, including geometric distortion, differences in image properties, outliers, and optimization convergence, and discusses their impact on registration accuracy and reliability. Strategies for addressing these challenges are highlighted, emphasizing the need for continual innovation and refinement of techniques to enhance the accuracy and reliability of medical image registration systems. The paper concludes by emphasizing the importance of accurate medical image registration in improving diagnosis.

MSRA-Net: multi-channel semantic-aware and residual attention mechanism network for unsupervised 3D image registration

Objective. Convolutional neural network (CNN) is developing rapidly in the field of medical image registration, and the proposed U-Net further improves the precision of registration. However, this method may discard certain important information in the process of encoding and decoding steps, consequently leading to a decline in accuracy. To solve this problem, a multi-channel semantic-aware and residual attention mechanism network (MSRA-Net) is proposed in this paper. Approach. Our proposed network achieves efficient information aggregation by cleverly extracting the features of different channels. Firstly, a context-aware module (CAM) is designed to extract valuable contextual information. And the depth-wise separable convolution is employed in the CAM to alleviate the computational burden. Then, a new multi-channel semantic-aware module (MCSAM) is designed for more comprehensive fusion of up-sampling features. Additionally, the residual attention module is introduced in the up-sampling process to extract more semantic information and minimize information loss. Main results. This study utilizes Dice score, average symmetric surface distance and negative Jacobian determinant evaluation metrics to evaluate the influence of registration. The experimental results demonstrate that our proposed MSRA-Net has the highest accuracy compared to several state-of-the-art methods. Moreover, our network has demonstrated the highest Dice score across multiple datasets, thereby indicating that the superior generalization capabilities of our model. Significance. The proposed MSRA-Net offers a novel approach to improve medical image registration accuracy, with implications for various clinical applications. Our implementation is available at https://github.com/shy922/MSRA-Net.

Spatial-aware contrastive learning for cross-domain medical image registration.

With the rapid advancement of medical imaging technologies, precise image analysis and diagnosis play a crucial role in enhancing treatment outcomes and patient care. Computed tomography (CT) and magnetic resonance imaging (MRI), as pivotal technologies in medical imaging, exhibit unique advantages in bone imaging and soft tissue contrast, respectively. However, cross-domain medical image registration confronts significant challenges due to the substantial differences in contrast, texture, and noise levels between different imaging modalities. The purpose of this study is to address the major challenges encountered in the field of cross-domain medical image registration by proposing a spatial-aware contrastive learning approach that effectively integrates shared information from CT and MRI images. Our objective is to optimize the feature space representation by employing advanced reconstruction and contrastive loss functions, overcoming the limitations of traditional registration methods when dealing with different imaging modalities. Through this approach, we aim to enhance the model's ability to learn structural similarities across domain images, improve registration accuracy, and provide more precise imaging analysis tools for clinical diagnosis and treatmentplanning. With prior knowledge that different domains of images (CT and MRI) share same content-style information, we extract equivalent feature spaces from both images, enabling accurate cross-domain point matching. We employ a structure resembling that of an autoencoder, augmented with designed reconstruction and contrastive losses to fulfill our objectives. We also propose region mask to solve the conflict between spatial correlation and distinctiveness, to obtain a better representation space. Our research results demonstrate the significant superiority of the proposed spatial-aware contrastive learning approach in the domain of cross-domain medical image registration. Quantitatively, our method achieved an average Dice similarity coefficient (DSC) of 85.68%, target registration error (TRE) of 1.92 mm, and mean Hausdorff distance (MHD) of 1.26 mm, surpassing current state-of-the-art methods. Additionally, the registration processing time was significantly reduced to 2.67 s on a GPU, highlighting the efficiency of our approach. The experimental outcomes not only validate the effectiveness of our method in improving the accuracy of cross-domain image registration but also prove its adaptability across different medical image analysis scenarios, offering robust support for enhancing diagnostic precision and patient treatmentoutcomes. The spatial-aware contrastive learning approach proposed in this paper introduces a new perspective and solution to the domain of cross-domain medical image registration. By effectively optimizing the feature space representation through carefully designed reconstruction and contrastive loss functions, our method significantly improves the accuracy and stability of registration between CT and MRI images. The experimental results demonstrate the clear advantages of our approach in enhancing the accuracy of cross-domain image registration, offering significant application value in promoting precise diagnosis and personalized treatment planning. In the future, we look forward to further exploring the application of this method in a broader range of medical imaging datasets and its potential integration with other advanced technologies, contributing more innovations to the field of medical image analysis andprocessing.

SAR Image Registration: The Combination of Nonlinear Diffusion Filtering, Hessian Features and Edge Points.

Synthetic aperture radar (SAR) image registration is an important process in many applications, such as image stitching and remote sensing surveillance. The registration accuracy is commonly affected by the presence of speckle noise in SAR images. When speckle noise is intense, the number of image features acquired by single-feature-based methods is insufficient. An SAR image registration method that combines nonlinear diffusion filtering, Hessian features and edge points is proposed in this paper to reduce speckle noise and obtain more image features. The proposed method uses the infinite symmetric exponential filter (ISEF) for image pre-processing and nonlinear diffusion filtering for scale-space construction. These measures can remove speckle noise from SAR images while preserving image edges. Hessian features and edge points are also employed as image features to optimize the utilization of feature information. Experiments with different noise levels, geometric transformations and image scenes demonstrate that the proposed method effectively improves the accuracy of SAR image registration compared with the SIFT-OCT, SAR-SIFT, Harris-SIFT, NF-Hessian and KAZE-SAR algorithms.

Remote sensing image registration method based on synchronous atmospheric correction.

Image registration is a crucial preprocessing step in remote sensing applications, integrating information from multiple images to achieve synergistic advantages. Nevertheless, aerosols characterized by spatiotemporal heterogeneity can result in the blurring of remote-sensing images, thereby compromising the accuracy of image registration. This paper begins by analyzing the basic principles of atmospheric correction and image registration. The variations in atmospheric radiative contribution caused by aerosol changes in real-world scenarios were simulated, along with an examination of the relationship between atmospheric effects and the quantity of image features. Subsequently, addressing the challenge posed by insufficient synchronicity in aerosol parameters and the influence of atmospheric effects on remote sensing image registration, we propose a registration method based on synchronous atmospheric correction. This approach utilizes the Airborne Synchronous Monitoring Atmospheric Corrector (ASMAC) to obtain aerosol optical depth and column water vapor images for synchronous atmospheric correction of remote sensing images, along with the assessment of the registration transformation matrix. Finally, airborne experiments involving ASMAC and high-resolution cameras are conducted to validate the proposed method's improvement in remote sensing image registration accuracy. Experimental results demonstrate the effectiveness of the proposed method, showcasing an increase in the number of features and improvements in quantitative evaluation metrics. Specifically, the normalized correlation coefficient improved by up to 2.408%, while the normalized mutual information increased by a maximum of 1.395%, a maximum feature count and successfully matched features improvement of 21.1% and 38.5.

Uncertainty estimation and evaluation of deformation image registration based convolutional neural networks

Objective. Fast and accurate deformable image registration (DIR), including DIR uncertainty estimation, is essential for safe and reliable clinical deployment. While recent deep learning models have shown promise in predicting DIR with its uncertainty, challenges persist in proper uncertainty evaluation and hyperparameter optimization for these methods. This work aims to develop and evaluate a model that can perform fast DIR and predict its uncertainty in seconds. Approach. This study introduces a novel probabilistic multi-resolution image registration model utilizing convolutional neural networks to estimate a multivariate normal distributed dense displacement field (DDF) in a multimodal image registration problem. To assess the quality of the DDF distribution predicted by the model, we propose a new metric based on the Kullback–Leibler divergence. The performance of our approach was evaluated against three other DIR algorithms (VoxelMorph, Monte Carlo dropout, and Monte Carlo B-spline) capable of predicting uncertainty. The evaluation of the models included not only the quality of the deformation but also the reliability of the estimated uncertainty. Our application investigated the registration of a treatment planning computed tomography (CT) to follow-up cone beam CT for daily adaptive radiotherapy. Main results. The hyperparameter tuning of the models showed a trade-off between the estimated uncertainty’s reliability and the deformation’s accuracy. In the optimal trade-off, our model excelled in contour propagation and uncertainty estimation (p <0.05) compared to existing uncertainty estimation models. We obtained an average dice similarity coefficient of 0.89 and a KL-divergence of 0.15. Significance. By addressing challenges in DIR uncertainty estimation and evaluation, our work showed that both the DIR and its uncertainty can be reliably predicted, paving the way for safe deployment in a clinical environment.

Registration of Longitudinal Spine CTs for Monitoring Lesion Growth.

Accurate and reliable registration of longitudinal spine images is essential for assessment of disease progression and surgical outcome. Implementing a fully automatic and robust registration is crucial for clinical use, however, it is challenging due to substantial change in shape and appearance due to lesions. In this paper we present a novel method to automatically align longitudinal spine CTs and accurately assess lesion progression. Our method follows a two-step pipeline where vertebrae are first automatically localized, labeled and 3D surfaces are generated using a deep learning model, then longitudinally aligned using a Gaussian mixture model surface registration. We tested our approach on 37 vertebrae, from 5 patients, with baseline CTs and 3, 6, and 12 months follow-ups leading to 111 registrations. Our experiment showed accurate registration with an average Hausdorff distance of 0.65 mm and average Dice score of 0.92.

HGCMorph: joint discontinuity-preserving and pose-learning via GNN and capsule networks for deformable medical images registration

Objective. This study aims to enhance medical image registration by addressing the limitations of existing approaches that rely on spatial transformations through U-Net, ConvNets, or Transformers. The objective is to develop a novel architecture that combines ConvNets, graph neural networks (GNNs), and capsule networks to improve the accuracy and efficiency of medical image registration, which can also deal with the problem of rotating registration. Approach. We propose an deep learning-based approach which can be utilized in both unsupervised and semi-supervised manners, named as HGCMorph. It leverages a hybrid framework that integrates ConvNets and GNNs to capture lower-level features, specifically short-range attention, while also utilizing capsule networks (CapsNets) to model abstract higher-level features, including entity properties such as position, size, orientation, deformation, and texture. This hybrid framework aims to provide a comprehensive representation of anatomical structures and their spatial relationships in medical images. Main results. The results demonstrate the superiority of HGCMorph over existing state-of-the-art deep learning-based methods in both qualitative and quantitative evaluations. In unsupervised training process, our model outperforms the recent SOTA method TransMorph by achieving 7%/38% increase on Dice score coefficient (DSC), and 2%/7% improvement on negative jacobian determinant for OASIS and LPBA40 datasets, respectively. Furthermore, HGCMorph achieves improved registration accuracy in semi-supervised training process. In addition, when dealing with complex 3D rotations and secondary randomly deformations, our method still achieves the best performance. We also tested our methods on lung datasets, such as Japanese Society of Radiology, Montgoermy and Shenzhen. Significance. The significance lies in its innovative design to medical image registration. HGCMorph offers a novel framework that overcomes the limitations of existing methods by efficiently capturing both local and abstract features, leading to enhanced registration accuracy, discontinuity-preserving, and pose-learning abilities. The incorporation of capsule networks introduces valuable improvements, making the proposed method a valuable contribution to the field of medical image analysis. HGCMorph not only advances the SOTA methods but also has the potential to improve various medical applications that rely on accurate image registration.

RAPHIA: A deep learning pipeline for the registration of MRI and whole-mount histopathology images of the prostate

Image registration can map the ground truth extent of prostate cancer from histopathology images onto MRI, facilitating the development of machine learning methods for early prostate cancer detection. Here, we present RAdiology PatHology Image Alignment (RAPHIA), an end-to-end pipeline for efficient and accurate registration of MRI and histopathology images. RAPHIA automates several time-consuming manual steps in existing approaches including prostate segmentation, estimation of the rotation angle and horizontal flipping in histopathology images, and estimation of MRI-histopathology slice correspondences. By utilizing deep learning registration networks, RAPHIA substantially reduces computational time. Furthermore, RAPHIA obviates the need for a multimodal image similarity metric by transferring histopathology image representations to MRI image representations and vice versa. With the assistance of RAPHIA, novice users achieved expert-level performance, and their mean error in estimating histopathology rotation angle was reduced by 51% (12 degrees vs 8 degrees), their mean accuracy of estimating histopathology flipping was increased by 5% (95.3% vs 100%), and their mean error in estimating MRI-histopathology slice correspondences was reduced by 45% (1.12 slices vs 0.62 slices). When compared to a recent conventional registration approach and a deep learning registration approach, RAPHIA achieved better mapping of histopathology cancer labels, with an improved mean Dice coefficient of cancer regions outlined on MRI and the deformed histopathology (0.44 vs 0.48 vs 0.50), and a reduced mean per-case processing time (51 vs 11 vs 4.5 min). The improved performance by RAPHIA allows efficient processing of large datasets for the development of machine learning models for prostate cancer detection on MRI. Our code is publicly available at: https://github.com/pimed/RAPHIA.

Performance characteristics of the 5-ring GE Discovery MI PET/CT scanner using AAPM TG-126 report.

To report on the performance characteristics of the 5-ring GE Discovery MI PET/CT systems using the AAPM TG-126 report and compare these results to NEMA NU 2-2012 where applicable. TG-126 testing was performed on two GE 5-Rings Discovery MI scanners. Tests performed included spatial resolution, PET/CT image-registration accuracy, sensitivity, count rate performance, accuracy of corrections, image contrast, scatter/attenuation correction, and image uniformity. All acquired data were analyzed using scanner console or free software tools as described by TG-126 and the results were then compared to published NEMA NU 2-2012 values. Both scanners gave similar resolution results for TG-126 and NEMA NU 2-2012 and were within manufacturer specifications. Image-registration accuracy between PET and CT using our clinical protocol showed excellent results with values≤1mm. Sensitivity using TG-126 was 19.43cps/kBq while for NEMA the value was 20.73cps/kBq. The peak noise-equivalent counting rate was 2174 kcps at 63.1 kBq/mL and is not comparable to NEMA NU 2-2012 due to differences in phantoms and methods used to measure and calculate this parameter. The accuracy of corrections for count losses for TG-126 were expressed in SUV values and found to be within 10% of the expected SUV measurement of 1. Image contrast and scatter/attenuation correction using the TG-126 method gave acceptable results. Image uniformity assessment resulted in values within the recommended±5% limits. These results show that the 5-ring GE Discovery MI PET/CT scanner testing using TG-126 is reproducible and has similar results to NEMA NU 2-2012 tests where applicable. We hope these results start to form the basis to compare PET/CT systems using TG-126.

Deep Learning-Based Medical Image Registration Algorithm: Enhancing Accuracy with Dense Connections and Channel Attention Mechanisms

In critical clinical medical image analysis applications, such as surgical navigation and tumor monitoring, image registration is crucial. Recognizing the potential for enhanced accuracy in existing unsupervised image registration techniques for single-modal imagery, this research introduces an innovative deep learning-based image registration algorithm. Its novelty resides in integrating short and long connections to create a densely connected structure, markedly refining the feature map interconnectivity within the U-Net architecture. This advancement addresses the significant semantic gap issues arising from disparities in feature map sampling depths. Moreover, the algorithm incorporates a channel attention mechanism within the U-shaped network's decoder, significantly mitigating image noise and facilitating the generation of smoother deformation fields. This enhancement not only boosts the model's detail sensitivity but also markedly increases image registration precision, particularly evident when processing single-modal brain MRI datasets, thereby proving the algorithm's efficacy and utility. Extensive clinical application-based training and testing have underscored this algorithm's substantial contributions to medical image registration accuracy enhancement. Overall, by leveraging deep learning technologies and innovative algorithmic structures, this study addresses pivotal challenges in medical image registration, offering more precise and dependable support for clinical applications like surgical navigation and tumor surveillance.