Deformable Registration Method Research Articles

Deformable registration of magnetic resonance images using unsupervised deep learning in neuro-/radiation oncology

PurposeAccurate deformable registration of magnetic resonance imaging (MRI) scans containing pathologies is challenging due to changes in tissue appearance. In this paper, we developed a novel automated three-dimensional (3D) convolutional U-Net based deformable image registration (ConvUNet-DIR) method using unsupervised learning to establish correspondence between baseline pre-operative and follow-up MRI scans of patients with brain glioma.MethodsThis study involved multi-parametric brain MRI scans (T1, T1-contrast enhanced, T2, FLAIR) acquired at pre-operative and follow-up time for 160 patients diagnosed with glioma, representing the BraTS-Reg 2022 challenge dataset. ConvUNet-DIR, a deep learning-based deformable registration workflow using 3D U-Net style architecture as a core, was developed to establish correspondence between the MRI scans. The workflow consists of three components: (1) the U-Net learns features from pairs of MRI scans and estimates a mapping between them, (2) the grid generator computes the sampling grid based on the derived transformation parameters, and (3) the spatial transformation layer generates a warped image by applying the sampling operation using interpolation. A similarity measure was used as a loss function for the network with a regularization parameter limiting the deformation. The model was trained via unsupervised learning using pairs of MRI scans on a training data set (n = 102) and validated on a validation data set (n = 26) to assess its generalizability. Its performance was evaluated on a test set (n = 32) by computing the Dice score and structural similarity index (SSIM) quantitative metrics. The model’s performance also was compared with the baseline state-of-the-art VoxelMorph (VM1 and VM2) learning-based algorithms.ResultsThe ConvUNet-DIR model showed promising competency in performing accurate 3D deformable registration. It achieved a mean Dice score of 0.975 ± 0.003 and SSIM of 0.908 ± 0.011 on the test set (n = 32). Experimental results also demonstrated that ConvUNet-DIR outperformed the VoxelMorph algorithms concerning Dice (VM1: 0.969 ± 0.006 and VM2: 0.957 ± 0.008) and SSIM (VM1: 0.893 ± 0.012 and VM2: 0.857 ± 0.017) metrics. The time required to perform a registration for a pair of MRI scans is about 1 s on the CPU.ConclusionsThe developed deep learning-based model can perform an end-to-end deformable registration of a pair of 3D MRI scans for glioma patients without human intervention. The model could provide accurate, efficient, and robust deformable registration without needing pre-alignment and labeling. It outperformed the state-of-the-art VoxelMorph learning-based deformable registration algorithms and other supervised/unsupervised deep learning-based methods reported in the literature.

Broad-UNet-diff: Diffeomorphic Deformable Medical Image Registration based on Multi-scale Feature Learning

Introduction: To propose a medical image registration method with significant performance improvement. The spatial transformation obtained by the traditional deformable image registration technology is not smooth enough, and the calculation amount is too large to solve the optimization parameters. The network model proposed based on deep learning medical image registration technology has some limitations, which cannot guarantee the registration of topological structures, resulting in the loss of spatial features. It makes the model have topological conservation and transform reversibility, has the ability to learn more multi-scale features and complex image structures, and captures finer changes while clearly encoding global correlation. Method: Based on the core UNet model, a deformable image registration method with a new architecture Broad-UNet-diff is proposed. The model is equipped with asymmetric parallel convolution and uses diffeomorphism mapping. Result: Compared with the seven classical registration methods under the brain MRI datasets, the proposed method has significantly improved the registration performance. In particular, compared with the advanced TransMorph-diff registration method, the Dice score can be improved by 12 %, but only the 1/10 parameters are needed. Conclusion: This method confirms its convincing effectiveness and accuracy.

MF-Net: multi-scale feature extraction-integration network for unsupervised deformable registration.

Deformable registration plays a fundamental and crucial role in scenarios such as surgical navigation and image-assisted analysis. While deformable registration methods based on unsupervised learning have shown remarkable success in predicting displacement fields with high accuracy, many existing registration networks are limited by the lack of multi-scale analysis, restricting comprehensive utilization of global and local features in the images. To address this limitation, we propose a novel registration network called multi-scale feature extraction-integration network (MF-Net). First, we propose a multiscale analysis strategy that enables the model to capture global and local semantic information in the image, thus facilitating accurate texture and detail registration. Additionally, we introduce grouped gated inception block (GI-Block) as the basic unit of the feature extractor, enabling the feature extractor to selectively extract quantitative features from images at various resolutions. Comparative experiments demonstrate the superior accuracy of our approach over existing methods.

Adaptable cascaded registration for personalized maxilla completion and cleft defect volume estimation.

Cone-beam computed tomography (CBCT) images provide high-resolution insights into the underlying craniofacial anomaly in patients with cleft lip and palate (CLP), requiring non-negligible annotation costs to measure the cleft defect for the guidance of the clinical secondary alveolar bone graft procedures. Considering the cumbersome volumetric image acquisition, there is a lack of paired CLP CBCTs and normal CBCTs for learning-based anatomical structure restoration models. Nowadays, the registration-based method relieves the annotation burden, though one-shot registration and the regular mask are limited to handling fine-grained shape variations and harmony between restored bony tissues and the defectedmaxilla. This study aimed to design and evaluate a novel method for deformable partial registration of the CLP CBCTs and normal CBCTs, enabling personalized maxilla completion and cleft defect volume prediction from CLPCBCTs. We proposed an adaptable deep registration framework for personalized maxilla completion and cleft defect volume prediction from CLP CBCTs. The key ingredient was a cascaded partial registration to exploit the maxillary morphology prior and attribute transfer. Cascaded registration with coarse-to-fine registration fields handled morphological variations of cleft defects and fine-grained maxillary restoration. We designed an adaptable cleft defect mask and volumetric Boolean operators for reliable voxel filling of the defected maxilla. A total of 36 clinically obtained CLP CBCTs were used to train and validate the proposed model, among which 22 CLP CBCTs were used to generate a training dataset with 440 synthetic CBCTs by B-spline deformation-based data augmentation and the remaining for testing. The proposed model was evaluated on maxilla completion and cleft defect volume prediction from clinically obtained unilateral and bilateral CLPCBCTs. Extensive experiments demonstrated the effectiveness of the adaptable cleft defect mask and the cascaded partial registration on maxilla completion and cleft defect volume prediction. The proposed method achieved state-of-the-art performances with the Dice similarity coefficient of 0.90 0.02 on the restored maxilla and 0.84 0.04 on the estimated cleft defect, respectively. The average Hausdorff distance between the estimated cleft defect and the manually annotated ground truth was 0.30 0.08 mm. The relative volume error of the cleft defect was 0.08. The proposed model allowed for the prediction of cleft defect maps that were in line with the ground truth in the challenging unilateral and bilateral CLP CBCTs. The results suggest that the proposed adaptable deep registration model enables patient-specific maxilla completion and automatic annotation of cleft defects, relieving tedious voxel-wise annotation and image acquisitionburdens.

Deformable registration of preoperative MR and intraoperative long-length tomosynthesis images for guidance of spine surgery via image synthesis

PurposeImproved integration and use of preoperative imaging during surgery hold significant potential for enhancing treatment planning and instrument guidance through surgical navigation. Despite its prevalent use in diagnostic settings, MR imaging is rarely used for navigation in spine surgery. This study aims to leverage MR imaging for intraoperative visualization of spine anatomy, particularly in cases where CT imaging is unavailable or when minimizing radiation exposure is essential, such as in pediatric surgery. MethodsThis work presents a method for deformable 3D-2D registration of preoperative MR images with a novel intraoperative long-length tomosynthesis imaging modality (viz., Long-Film [LF]). A conditional generative adversarial network is used to translate MR images to an intermediate bone image suitable for registration, followed by a model-based 3D-2D registration algorithm to deformably map the synthesized images to LF images. The algorithm’s performance was evaluated on cadaveric specimens with implanted markers and controlled deformation, and in clinical images of patients undergoing spine surgery as part of a large-scale clinical study on LF imaging. ResultsThe proposed method yielded a median 2D projection distance error of 2.0 mm (interquartile range [IQR]: 1.1–3.3 mm) and a 3D target registration error of 1.5 mm (IQR: 0.8–2.1 mm) in cadaver studies. Notably, the multi-scale approach exhibited significantly higher accuracy compared to rigid solutions and effectively managed the challenges posed by piecewise rigid spine deformation. The robustness and consistency of the method were evaluated on clinical images, yielding no outliers on vertebrae without surgical instrumentation and 3% outliers on vertebrae with instrumentation. ConclusionsThis work constitutes the first reported approach for deformable MR to LF registration based on deep image synthesis. The proposed framework provides access to the preoperative annotations and planning information during surgery and enables surgical navigation within the context of MR images and/or dual-plane LF images.

Incidental Ring-hyperenhancing Liver Micronodules at CT Hepatic Arteriography-guided Percutaneous Thermal Ablation of Colorectal Liver Metastases.

CT during hepatic arteriography (CTHA) is a highly sensitive imaging method for detecting colorectal liver metastases (CLMs), which supports its use during percutaneous thermal liver ablation. In contrast to its high sensitivity, its specificity for incidental small CLMs not detected at preablation cross-sectional imaging is believed to be low given the absence of specific imaging signatures and the common presence of pseudolesions. In this retrospective study of 22 patients (mean age, 55 years ± 10.6 [SD]; 63.6% male, 36.4% female) with CLMs undergoing CTHA-guided microwave percutaneous thermal ablation between November 2017 and October 2022, the authors provided a definition of incidental ring-hyperenhancing liver micronodules (RHLMs) and investigated whether there is a correlation of RHLMs with histologic analysis or intrahepatic tumor progression at imaging follow-up after applying a biomechanical deformable image registration method. The analysis revealed 25 incidental RHLMs in 41.7% (10 of 24) of the CTHA images from the respective guided ablation sessions. Of those, four RHLMs were ablated. Among the remaining 21 RHLMs, 71.4% (15 of 21) were confirmed to be CLM with either histology (n = 3) or imaging follow-up (n = 12). The remaining 28.6% (six of 21) of RHLMs were not observed at follow-up imaging. This suggests that RHLMs at CTHA may be an early indicator of incidental small CLMs. Keywords: Colorectal Neoplasms, Liver, Angiography, CT, Incidental Findings, Ablation Supplemental material is available for this article. © RSNA, 2024.

Robust Deformable Image Registration Using Cycle-Consistent Implicit Representations.

Recent works in medical image registration have proposed the use of Implicit Neural Representations, demonstrating performance that rivals state-of-the-art learning-based methods. However, these implicit representations need to be optimized for each new image pair, which is a stochastic process that may fail to converge to a global minimum. To improve robustness, we propose a deformable registration method using pairs of cycle-consistent Implicit Neural Representations: each implicit representation is linked to a second implicit representation that estimates the opposite transformation, causing each network to act as a regularizer for its paired opposite. During inference, we generate multiple deformation estimates by numerically inverting the paired backward transformation and evaluating the consensus of the optimized pair. This consensus improves registration accuracy over using a single representation and results in a robust uncertainty metric that can be used for automatic quality control. We evaluate our method with a 4D lung CT dataset. The proposed cycle-consistent optimization method reduces the optimization failure rate from 2.4% to 0.0% compared to the current state-of-the-art. The proposed inference method improves landmark accuracy by 4.5% and the proposed uncertainty metric detects all instances where the registration method fails to converge to a correct solution. We verify the generalizability of these results to other data using a centerline propagation task in abdominal 4D MRI, where our method achieves a 46% improvement in propagation consistency compared with single-INR registration and demonstrates a strong correlation between the proposed uncertainty metric and registration accuracy.

AR-UNet: A Deformable Image Registration Network with Cyclic Training.

Deformable image registration is a process to determine the non-linear spatial correspondence among deformed image pairs. Generative registration network is a novel structure involving a generative registration network and a discriminative network that encourages the former to generate better results. We propose an Attention Residual UNet (AR-UNet) to estimate the complicated deformation field. The model is trained using perceptual cyclic constraints. As an unsupervised method, we require labelling for training and use virtual data augmentation to improve the robustness of the proposed model. We also introduce comprehensive metrics for image registration comparison. Experimental results show quantitative evidence that the proposed method can predict reliable deformation field at a reasonable speed and outperform conventional learning based and non-learning based deformable image registration methods.

In vivo strain measurements in the human buttock during sitting using MR-based digital volume correlation

Advancements in systems for prevention and management of pressure ulcers require a more detailed understanding of the complex response of soft tissues to compressive loads. This study aimed at quantifying the progressive deformation of the buttock based on 3D measurements of soft tissue displacements from MR scans of 10 healthy subjects in a semi-recumbent position. Measurements were obtained using digital volume correlation (DVC) and released as a public dataset. A first parametric optimisation of the global registration step aimed at aligning skeletal elements showed acceptable values of Dice coefficient (around 80%). A second parametric optimisation on the deformable registration method showed errors of 0.99mm and 1.78mm against two simulated fields with magnitude 7.30±3.15mm and 19.37±9.58mm, respectively, generated with a finite element model of the buttock under sitting loads. Measurements allowed the quantification of the slide of the gluteus maximus away from the ischial tuberosity (IT, average 13.74 mm) that was only qualitatively identified in the literature, highlighting the importance of the ischial bursa in allowing sliding. Spatial evolution of the maximus shear strain on a path from the IT to the seating interface showed a peak of compression in the fat, close to the interface with the muscle. Obtained peak values were above the proposed damage threshold in the literature. Results in the study showed the complexity of the deformation of the soft tissues in the buttock and the need for further investigations aimed at isolating factors such as tissue geometry, duration and extent of load, sitting posture and tissue properties.

Comparison of physics-based deformable registration methods for image-guided neurosurgery.

This paper compares three finite element-based methods used in a physics-based non-rigid registration approach and reports on the progress made over the last 15 years. Large brain shifts caused by brain tumor removal affect registration accuracy by creating point and element outliers. A combination of approximation- and geometry-based point and element outlier rejection improves the rigid registration error by 2.5 mm and meets the real-time constraints (4 min). In addition, the paper raises several questions and presents two open problems for the robust estimation and improvement of registration error in the presence of outliers due to sparse, noisy, and incomplete data. It concludes with preliminary results on leveraging Quantum Computing, a promising new technology for computationally intensive problems like Feature Detection and Block Matching in addition to finite element solver; all three account for 75% of computing time in deformable registration.

Hierarchical cumulative network for unsupervised medical image registration

Unsupervised deep learning techniques have gained increasing popularity in deformable medical image registration However, existing methods usually overlook the optimal similarity position between moving and fixed images To tackle this issue, we propose a novel hierarchical cumulative network (HCN), which explicitly considers the optimal similarity position with an effective Bidirectional Asymmetric Registration Module (BARM). The BARM simultaneously learns two asymmetric displacement vector fields (DVFs) to optimally warp both moving images and fixed images to their optimal similar shape along the geodesic path. Furthermore, we incorporate the BARM into a Laplacian pyramid network with hierarchical recursion, in which the moving image at the lowest level of the pyramid is warped successively for aligning to the fixed image at the lowest level of the pyramid to capture multiple DVFs. We then accumulate these DVFs and up-sample them to warp the moving images at higher levels of the pyramid to align to the fixed image of the top level. The entire system is end-to-end and jointly trained in an unsupervised manner. Extensive experiments were conducted on two public 3D Brain MRI datasets to demonstrate that our HCN outperforms both the traditional and state-of-the-art registration methods. To further evaluate the performance of our HCN, we tested it on the validation set of the MICCAI Learn2Reg 2021 challenge. Additionally, a cross-dataset evaluation was conducted to assess the generalization of our HCN. Experimental results showed that our HCN is an effective deformable registration method and achieves excellent generalization performance.

Intraprocedural Versus Initial Follow-up Minimal Ablative Margin Assessment After Colorectal Liver Metastasis Thermal Ablation: Which One Better Predicts Local Outcomes?

The aim of this study was to investigate the prognostic value of 3-dimensional minimal ablative margin (MAM) quantified by intraprocedural versus initial follow-up computed tomography (CT) in predicting local tumor progression (LTP) after colorectal liver metastasis (CLM) thermal ablation. This single-institution, patient-clustered, tumor-based retrospective study included patients undergoing microwave and radiofrequency ablation between 2016 and 2021. Patients without intraprocedural and initial follow-up contrast-enhanced CT, residual tumors, or with follow-up less than 1 year without LTP were excluded. Minimal ablative margin was quantified by a biomechanical deformable image registration method with segmentations of CLMs on intraprocedural preablation CT and ablation zones on intraprocedural postablation and initial follow-up CT. Prognostic value of MAM to predict LTP was tested using area under the curve and competing-risk regression model. A total of 68 patients (mean age ± standard deviation, 57 ± 12 years; 43 men) with 133 CLMs were included. During a median follow-up of 30.3 months, LTP rate was 17% (22/133). The median volume of ablation zone was 27 mL and 16 mL segmented on intraprocedural and initial follow-up CT, respectively ( P < 0.001), with corresponding median MAM of 4.7 mm and 0 mm, respectively ( P < 0.001). The area under the curve was higher for MAM quantified on intraprocedural CT (0.89; 95% confidence interval [CI], 0.83-0.94) compared with initial follow-up CT (0.66; 95% CI, 0.54-0.76) in predicting 1-year LTP ( P < 0.001). An MAM of 0 mm on intraprocedural CT was an independent predictor of LTP with a subdistribution hazards ratio of 11.9 (95% CI, 4.9-28.9; P < 0.001), compared with 2.4 (95% CI, 0.9-6.0; P = 0.07) on initial follow-up CT. Ablative margin quantified on intraprocedural CT significantly outperformed initial follow-up CT in predicting LTP and should be used for ablation endpoint assessment.

PC-Reg: A pyramidal prediction–correction approach for large deformation image registration

Deformable image registration plays an important role in medical image analysis. Deep neural networks such as VoxelMorph and TransMorph are fast, but limited to small deformations and face challenges in the presence of large deformations. To tackle large deformations in medical image registration, we propose PC-Reg, a pyramidal Prediction and Correction method for deformable registration, which treats multi-scale registration akin to solving an ordinary differential equation (ODE) across scales. Starting with a zero-initialized deformation at the coarse level, PC-Reg follows the predictor–corrector regime and progressively predicts a residual flow and a correction flow to update the deformation vector field through different scales. The prediction in each scale can be regarded as a single step of ODE integration. PC-Reg can be easily extended to diffeomorphic registration and is able to alleviate the multiscale accumulated upsampling and diffeomorphic integration error. Further, to transfer details from full resolution to low scale, we introduce a distillation loss, where the output is used as the target label for intermediate outputs. Experiments on inter-patient deformable registration show that the proposed method significantly improves registration not only for large but also for small deformations.

MacJNet: weakly-supervised multimodal image deformable registration using joint learning framework and multi-sampling cascaded MIND

Deformable multimodal image registration plays a key role in medical image analysis. It remains a challenge to find accurate dense correspondences between multimodal images due to the significant intensity distortion and the large deformation. macJNet is proposed to align the multimodal medical images, which is a weakly-supervised multimodal image deformable registration method using a joint learning framework and multi-sampling cascaded modality independent neighborhood descriptor (macMIND). The joint learning framework consists of a multimodal image registration network and two segmentation networks. The proposed macMIND is a modality-independent image structure descriptor to provide dense correspondence for registration, which incorporates multi-orientation and multi-scale sampling patterns to build self-similarity context. It greatly enhances the representation ability of cross-modal features in the registration network. The semi-supervised segmentation networks generate anatomical labels to provide semantics correspondence for registration, and the registration network helps to improve the performance of multimodal image segmentation by providing the consistency of anatomical labels. 3D CT-MR liver image dataset with 118 samples is built for evaluation, and comprehensive experiments have been conducted to demonstrate that macJNet achieves superior performance over state-of-the-art multi-modality medical image registration methods.

Deformable lung 4DCT image registration via landmark-driven cycle network.

An automated, accurate, and efficient lung four-dimensional computed tomography (4DCT) image registration method is clinically important to quantify respiratory motion for optimal motion management. The purpose of this work is to develop a weakly supervised deep learning method for 4DCT lung deformable image registration (DIR). The landmark-driven cycle network is proposed as a deep learning platform that performs DIR of individual phase datasets in a simulation 4DCT. This proposed network comprises a generator and a discriminator. The generator accepts moving and target CTs as input and outputs the deformation vector fields (DVFs) to match the two CTs. It is optimized during both forward and backward paths to enhance the bi-directionality of DVF generation. Further, the landmarks are used to weakly supervise the generator network. Landmark-driven loss is used to guide the generator's training. The discriminator then judges the realism of the deformed CT to provide extra DVF regularization. We performed four-fold cross-validation on 10 4DCT datasets from the public DIR-Lab dataset and a hold-out test on our clinic dataset, which included 50 4DCT datasets. The DIR-Lab dataset was used to evaluate the performance of the proposed method against other methods in the literature by calculating the DIR-Lab Target Registration Error (TRE). The proposed method outperformed other deep learning-based methods on the DIR-Lab datasets in terms of TRE. Bi-directional and landmark-driven loss were shown to be effective for obtaining high registration accuracy. The mean and standard deviation of TRE for the DIR-Lab datasets was 1.20±0.72mm and the mean absolute error (MAE) and structural similarity index (SSIM) for our datasets were 32.1±11.6 HU and 0.979±0.011, respectively. The landmark-driven cycle network has been validated and tested for automatic deformable image registration of patients' lung 4DCTs with results comparable to or better than competing methods.

CT-CBCT deformable registration using weakly-supervised artifact-suppression transfer learning network

Objective. Computed tomography-cone-beam computed tomography (CT-CBCT) deformable registration has great potential in adaptive radiotherapy. It plays an important role in tumor tracking, secondary planning, accurate irradiation, and the protection of at-risk organs. Neural networks have been improving CT-CBCT deformable registration, and almost all registration algorithms based on neural networks rely on the gray values of both CT and CBCT. The gray value is a key factor in the loss function, parameter training, and final efficacy of the registration. Unfortunately, the scattering artifacts in CBCT affect the gray values of different pixels inconsistently. Therefore, the direct registration of the original CT-CBCT introduces artifact superposition loss. Approach. In this study, a histogram analysis method for the gray values was used. Based on an analysis of the gray value distribution characteristics of different regions in CT and CBCT, the degree of superposition of the artifact in the region of disinterest was found to be much higher than that in the region of interest. Moreover, the former was the main reason for artifact superposition loss. Consequently, a new weakly supervised two-stage transfer-learning network based on artifact suppression was proposed. The first stage was a pre-training network designed to suppress artifacts contained in the region of disinterest. The second stage was a convolutional neural network that registered the suppressed CBCT and CT. Main Results. Through a comparative test of the thoracic CT-CBCT deformable registration, whose data were collected from the Elekta XVI system, the rationality and accuracy after artifact suppression were confirmed to be significantly improved compared with the other algorithms without artifact suppression. Significance. This study proposed and verified a new deformable registration method with multi-stage neural networks, which can effectively suppress artifacts and further improve registration by incorporating a pre-training technique and an attention mechanism.

Motion-compensated scheme for sequential scanned statistical iterative dual-energy CT reconstruction

Objective. Dual-energy computed tomography (DECT) has been widely used to reconstruct numerous types of images due its ability to better discriminate tissue properties. Sequential scanning is a popular dual-energy data acquisition method as it requires no specialized hardware. However, patient motion between two sequential scans may lead to severe motion artifacts in DECT statistical iterative reconstructions (SIR) images. The objective is to reduce the motion artifacts in such reconstructions. Approach. We propose a motion-compensation scheme that incorporates a deformation vector field into any DECT SIR. The deformation vector field is estimated via the multi-modality symmetric deformable registration method. The precalculated registration mapping and its inverse or adjoint are then embedded into each iteration of the iterative DECT algorithm. Main results. Results from a simulated and clinical case show that the proposed framework is capable of reducing motion artifacts in DECT SIRs. Percentage mean square errors in regions of interest in the simulated and clinical cases were reduced from 4.6% to 0.5% and 6.8% to 0.8%, respectively. A perturbation analysis was then performed to determine errors in approximating the continuous deformation by using the deformation field and interpolation. Our findings show that errors in our method are mostly propagated through the target image and amplified by the inverse matrix of the combination of the Fisher information and Hessian of the penalty term. Significance. We have proposed a novel motion-compensation scheme to incorporate a 3D registration method into the joint statistical iterative DECT algorithm in order to reduce motion artifacts caused by inter-scan motion, and successfully demonstrate that interscan motion corrections can be integrated into the DECT SIR process, enabling accurate imaging of radiological quantities on conventional SECT scanners, without significant loss of either computational efficiency or accuracy.

The geometric and dosimetric accuracy of kilovoltage cone beam computed tomography images for adaptive treatment: a systematic review.

To provide an overview and meta-analysis of different techniques adopted to accomplish kVCBCT for dose calculation and automated segmentation. A systematic review and meta-analysis were performed on eligible studies demonstrating kVCBCT-based dose calculation and automated contouring of different tumor features. Meta-analysis of the performance was accomplished on the reported γ analysis and dice similarity coefficient (DSC) score of both collected results as three subgroups (head and neck, chest, and abdomen). After the literature scrutinization (n = 1008), 52 papers were recognized for the systematic review. Nine studies of dosimtric studies and eleven studies of geometric analysis were suitable for inclusion in meta-analysis. Using kVCBCT for treatment replanning depends on a method used. Deformable Image Registration (DIR) methods yielded small dosimetric error (≤2%), γ pass rate (≥90%) and DSC (≥0.8). Hounsfield Unit (HU) override and calibration curve-based methods also achieved satisfactory yielded small dosimetric error (≤2%) and γ pass rate ((≥90%), but they are prone to error due to their sensitivity to a vendor-specific variation in kVCBCT image quality. Large cohorts of patients ought to be undertaken to validate methods achieving low levels of dosimetric and geometric errors. Quality guidelines should be established when reporting on kVCBCT, which include agreed metrics for reporting on the quality of corrected kVCBCT and defines protocols of new site-specific standardized imaging used when obtaining kVCBCT images for adaptive radiotherapy. This review gives useful knowledge about methods making kVCBCT feasible for kVCBCT-based adaptive radiotherapy, simplifying patient pathway and reducing concomitant imaging dose to the patient.

Four-Dimensional Cone Beam CT Imaging Using a Single Routine Scan via Deep Learning.

A novel method is proposed to obtain four-dimensional (4D) cone-beam computed tomography (CBCT) images from a routine scan in patients with upper abdominal cancer. The projections are sorted according to the location of the lung diaphragm before being reconstructed to phase-sorted data. A multiscale-discriminator generative adversarial network (MSD-GAN) is proposed to alleviate the severe streaking artifacts in the original images. The MSD-GAN is trained using simulated CBCT datasets from patient planning CT images. The enhanced images are further used to estimate the deformable vector field (DVF) among breathing phases using a deformable image registration method. The estimated DVF is then applied in the motion-compensated ordered-subset simultaneous algebraic reconstruction approach to generate 4D CBCT images. The proposed MSD-GAN is compared with U-Net on the performance of image enhancement. Results show that the proposed method significantly outperforms the total variation regularization-based iterative reconstruction approach and the method using only MSD-GAN to enhance original phase-sorted images in simulation and patient studies on 4D reconstruction quality. The MSD-GAN also shows higher accuracy than the U-Net. The proposed method enables a practical way for 4D-CBCT imaging from a single routine scan in upper abdominal cancer treatment including liver and pancreatic tumors.

Few-shot multi-modal registration with mono-modal knowledge transfer

Multi-modal registration is a key problem in many medical image analysis applications. Recent learning-based deformable image registration methods become attractive alternatives to traditional methods because of their great performance and fast run time. However, their success relies on large training datasets that are rarely available in multi-modal registration scenarios. To address this, we propose a novel knowledge transfer-based network (KT-Net) for few-shot multi-modal registration, which focuses on transferring knowledge of the mono-modal registration model to multi-modal registration. The contributions can be two-fold: (1) we propose model decoupling to disentangle the registration model into a feature learning network and an alignment learning network. The two networks are trained on large mono-modal datasets, making preparations for knowledge transfer. (2) a reverse teaching strategy is further designed to align the features of multi-modal images with a few samples, enabling the knowledge from mono-modal registration to transfer to multi-modal registration. Experimental results on multi-contrast brain MRI datasets demonstrate that our proposed method yields accurate and robust registration performance under the constraint of a few multi-modal samples. Compared with state-of-the-art registration methods, our proposed method achieves better registration performance with a high average Dice score of up to 83.5% and an average 95% percentile of Hausdorff distance as low as 1.26mm in various anatomical structures, showing great potential for mono-modal knowledge transfer to be applied in few-shot multi-modal registration.