Deformable Image Registration Research Articles

Identification of A0 minimum ablative margins for colorectal liver metastases: multicentre, retrospective study using deformable CT registration and artificial intelligence-based autosegmentation.

Several ablation confirmation software methods for minimum ablative margin assessment have recently been developed to improve local outcomes for patients undergoing thermal ablation of colorectal liver metastases. Previous assessments were limited to single institutions mostly at the place of development. The aim of this study was to validate the previously identified 5 mm minimum ablative margin (A0) using autosegmentation and biomechanical deformable image registration in a multi-institutional setting. This was a multicentre, retrospective study including patients with colorectal liver metastases undergoing CT- or ultrasound-guided microwave or radiofrequency ablation during 2009-2022, reporting 3-year local disease progression (residual unablated tumour or local tumour progression) rates by minimum ablative margin across all institutions and identifying an intraprocedural contrast-enhanced CT-based minimum ablative margin associated with a 3-year local disease progression rate of less than 1%. A total of 400 ablated colorectal liver metastases (median diameter of 1.5 cm) in 243 patients (145 men; median age of 62 [interquartile range 54-70] years) were evaluated, with a median follow-up of 26 (interquartile range 17-40) months. A total of 119 (48.9%) patients with 186 (46.5%) colorectal liver metastases were from international institutions B, C, and D that were not involved in the software development. Three-year local disease progression rates for 0 mm, >0 and <5 mm, and 5 mm or larger minimum ablative margins were 79%, 15%, and 0% respectively for institution A (where the software was developed) and 34%, 19%, and 2% respectively for institutions B, C, and D combined. Local disease progression risk decreased to less than 1% with an intraprocedurally confirmed minimum ablative margin greater than 4.6 mm. A minimum ablative margin of 5 mm or larger demonstrates optimal local oncological outcomes. It is proposed that an intraprocedural minimum ablative margin of 5 mm or larger, confirmed using biomechanical deformable image registration, serves as the A0 for colorectal liver metastasis thermal ablation.

Quantitative Analysis of Small Intestinal Motility in 3D Cine-MRI Using Centerline-Aware Motion Estimation.

Currently available tools for noninvasive motility quantification of the small intestine are limited to dynamic 2D MRI scans, which are limited in their ability to differentiate between types of intestinal motility. To develop a method for quantification and characterization of small intestinal motility in 3D, capable of differentiating motile, non-motile and peristaltic motion patterns. Prospective. Fourteen healthy volunteers (127 small intestinal segments) and 10 patients with Crohn's disease (87 small intestinal segments). 3.0 T, 3D balanced fast field echo sequence, 1 volume per second. Using deformable image registration between subsequent volumes, the local velocity within the intestinal lumen was quantified. Average velocity and average absolute velocity along intestinal segments were used with linear classifiers to differentiate motile from non-motile intestines, as well as erratic motility from peristalsis. The mean absolute velocity of small intestinal content was compared between healthy volunteers and Crohn's disease patients, and the discriminative power of the proposed motility metrics for detecting motility and peristalsis was determined. The consensus of two observers was used as referenced standard. Student's t-test to assess differences between groups; area under the receiver operating characteristic curve (AUC) to assess discriminative ability. P < 0.001 was considered significant. A significant difference in the absolute velocity of intestinal content between Crohn's patients and healthy volunteers was observed (median [IQR] 1.06 [0.61, 1.56] mm/s vs. 1.84 [1.37, 2.43] mm/s), which was consistent with manual reference annotations of motile activity. The proposed method had a strong discriminative performance for detecting non-motile intestines (AUC 0.97) and discernible peristalsis (AUC 0.81). Analysis of 3D cine-MRI using centerline-aware motion estimation has the potential to allow noninvasive characterization of small intestinal motility and peristaltic motion in 3D. 3 TECHNICAL EFFICACY: Stage 2.

An operator-independent quality assurance system for automatically generated structure sets

Objective. This study describes geometry-based and intensity-based tools for quality assurance (QA) of automatically generated structures for online adaptive radiotherapy, and designs an operator-independent traffic light system that identifies erroneous structure sets. Approach. A cohort of eight head and neck (HN) patients with daily CBCTs was selected for test development. Radiotherapy contours were propagated from planning computed tomography (CT) to daily cone beam CT (CBCT) using deformable image registration. These propagated structures were visually verified for acceptability. For each CBCT, several error scenarios were used to generate what were judged unacceptable structures. Ten additional HN patients with daily CBCTs and different error scenarios were selected for validation. A suite of tests based on image intensity, intensity gradient, and structure geometry was developed using acceptable and unacceptable HN planning structures. Combinations of one test applied to one structure, referred to as structure-test combinations, were selected for inclusion in the QA system based on their discriminatory power. A traffic light system was used to aggregate the structure-test combinations, and the system was evaluated on all fractions of the ten validation HN patients. Results. The QA system distinguished between acceptable and unacceptable fractions with high accuracy, labeling 294/324 acceptable fractions as green or yellow and 19/20 unacceptable fractions as yellow or red. Significance. This study demonstrates a system to supplement manual review of radiotherapy planning structures. Automated QA is performed by aggregating results from multiple intensity- and geometry-based tests.

Hierarchical multi-level dynamic hyperparameter deformable image registration with convolutional neural network

Objective. To enable the registration network to be trained only once, achieving fast regularization hyperparameter selection during the inference phase, and to improve registration accuracy and deformation field regularity. Approach. Hyperparameter tuning is an essential process for deep learning deformable image registration (DLDIR). Most DLDIR methods usually perform a large number of independent experiments to select the appropriate regularization hyperparameters, which are time-consuming and resource-consuming. To address this issue, we propose a novel dynamic hyperparameter block, which comprises a distributed mapping network, dynamic convolution, attention feature extraction layer, and instance normalization layer. The dynamic hyperparameter block encodes the input feature vectors and regularization hyperparameters into learnable feature variables and dynamic convolution parameters which changes the feature statistics of the high-dimensional features layer feature variables, respectively. In addition, the proposed method replaced the single-level structure residual blocks in LapIRN with a hierarchical multi-level architecture for the dynamic hyperparameter block in order to improve registration performance. Main results. On the OASIS dataset, the proposed method reduced the percentage of |Jϕ|⩽0 by 28.01 % , 9.78 % and improved Dice similarity coefficient by 1.17 % , 1.17 % , compared with LapIRN and CIR, respectively. On the DIR-Lab dataset, the proposed method reduced the percentage of |Jϕ|⩽0 by 10.00 % , 5.70 % and reduced target registration error by 10.84 % , 10.05 % , compared with LapIRN and CIR, respectively. Significance. The proposed method can fast achieve the corresponding registration deformation field for arbitrary hyperparameter value during the inference phase. Extensive experiments demonstrate that the proposed method reduces training time compared to DLDIR with fixed regularization hyperparameters while outperforming the state-of-the-art registration methods concerning registration accuracy and deformation smoothness on brain dataset OASIS and lung dataset DIR-Lab.

NCNet: Deformable medical image registration network based on neighborhood cross-attention combined with multi-resolution constraints

Objective. Existing registration networks based on cross-attention design usually divide the image pairs to be registered into patches for input. The division and merging operations of a series of patches are difficult to maintain the topology of the deformation field and reduce the interpretability of the network. Therefore, our goal is to develop a new network architecture based on a cross-attention mechanism combined with a multi-resolution strategy to improve the accuracy and interpretability of medical image registration. Approach. We propose a new deformable image registration network NCNet based on neighborhood cross-attention combined with multi-resolution strategy. The network structure mainly consists of a multi-resolution feature encoder, a multi-head neighborhood cross-attention module and a registration decoder. The hierarchical feature extraction capability of our encoder is improved by introducing large kernel parallel convolution blocks; the cross-attention module based on neighborhood calculation is used to reduce the impact on the topology of the deformation field and double normalization is used to reduce its computational complexity. Main result. We performed atlas-based registration and inter-subject registration tasks on the public 3D brain magnetic resonance imaging datasets LPBA40 and IXI respectively. Compared with the popular VoxelMorph method, our method improves the average DSC value by 7.9% and 3.6% on LPBA40 and IXI. Compared with the popular TransMorph method, our method improves the average DSC value by 4.9% and 1.3% on LPBA40 and IXI. Significance. We proved the advantages of the neighborhood attention calculation method compared to the window attention calculation method based on partitioning patches, and analyzed the impact of the pyramid feature encoder and double normalization on network performance. This has made a valuable contribution to promoting the further development of medical image registration methods.

Technical note: Phantom-based evaluation of CBCT dose calculation accuracy for use in adaptive radiotherapy.

High-quality 3D-anatomy of the day is needed for treatment plan adaptation in radiotherapy. For online x-ray-based CBCT workflows, one approach is to create a synthetic CT or to utilize a fan-beam CT with corresponding registrations. The former potentially introduces uncertainties in the dose calculation if deformable image registration is used. The latter can introduce burden and complexity to the process, the facility, and thepatient. Using the CBCT of the day, acquired on the treatment device, for direct dose calculation and plan adaptation can overcome these limitations. This study aims to assess the accuracy of the calculated dose on the CBCT scans acquired on a Halcyon linear accelerator equipped withHyperSight. HyperSight's new CBCT reconstruction algorithm includes improvements in scatter correction, HU calibration of the imager, and beam shape adaptation. Furthermore, HyperSight introduced a new x-ray detector. To show the effect of the implemented improvements, gamma comparisons of 2%/2mm, 2%/1mm, and 1%/1mm were made between the dose distribution in phantoms calculated on the CBCT reconstructions and the simulation CT scans, considering this the standard of care. The resulting gamma passing rates were compared to those obtained with the Halcyon 3.0 reconstruction and hardware without HyperSight's technologies. Various anatomical phantoms for dosimetric evaluations on brain, head and neck, lung, breast, and prostate cases have been used in thisstudy. The overall results demonstrated that HyperSight outperformed the Halcyon 3.0 version. Based on the gamma analysis, the calculated dose using HyperSight was closer to the CT scan-based doses than the calculated dose using iCBCT Halcyon 3.0 for most cases. Over all plans and gamma criteria, Halcyon 3.0 achieved an average passing rate of 92.9%, whereas HyperSight achieved 98.1%. Using HyperSight CBCT images for direct dose calculation, for example, in (online) plan adaptation, seems feasible for the investigatedcases.

OTMorph: Unsupervised Multi-domain Abdominal Medical Image Registration Using Neural Optimal Transport.

Deformable image registration is one of the essential processes in analyzing medical images. In particular, when diagnosing abdominal diseases such as hepatic cancer and lymphoma, multi-domain images scanned from different modalities or different imaging protocols are often used. However, they are not aligned due to scanning times, patient breathing, movement, etc. Although recent learning-based approaches can provide deformations in real-time with high performance, multi-domain abdominal image registration using deep learning is still challenging since the images in different domains have different characteristics such as image contrast and intensity ranges. To address this, this paper proposes a novel unsupervised multi-domain image registration framework using neural optimal transport, dubbed OTMorph. When moving and fixed volumes are given as input, a transport module of our proposed model learns the optimal transport plan to map data distributions from the moving to the fixed volumes and estimates a domain-transported volume. Subsequently, a registration module taking the transported volume can effectively estimate the deformation field, leading to deformation performance improvement. Experimental results on multi-domain image registration using multi-modality and multi-parametric abdominal medical images demonstrate that the proposed method provides superior deformable registration via the domain-transported image that alleviates the domain gap between the input images. Also, we attain the improvement even on out-of-distribution data, which indicates the superior generalizability of our model for the registration of various medical images. Our source code is available at https://github.com/boahK/OTMorph.

Development and validation of an MR-driven dose-of-the-day procedure for online adaptive radiotherapy in upper gastrointestinal cancer patients

Objective. To develop and validate a dose-of-the-day (DOTD) treatment plan verification procedure for liver and pancreas cancer patients treated with an magnetic resonance (MR)-Linac system. Approach. DOTD was implemented as an automated process that uses 3D datasets collected during treatment delivery. Particularly, the DOTD pipeline’s input included the adapt-to-shape (ATS) plan—i.e. 3D-MR dataset acquired at beginning of online session, anatomical contours, dose distribution—and 3D-MR dataset acquired during beam-on (BON). The DOTD automated analysis included (a) ATS-to-BON image intensity-based deformable image registration (DIR), (b) ATS-to-BON contours mapping via DIR, (c) BON-to-ATS contours copying through rigid registration, (d) determining ATS-to-BON dosimetric differences, and (e) PDF report generation. The DIR process was validated by two expert reviewers. ATS-plans were recomputed on BON datasets to assess dose differences. DOTD analysis was performed retrospectively for 75 treatment fractions (12-liver and 5-pancreas patients). Main results. The accuracy of DOTD process relied on DIR and mapped contours quality. Most DIR-generated contours (99.6%) were clinically acceptable. DICE correlated with depreciation of DIR-based region of interest mapping process. The ATS-BON plan difference was found negligible (<1%). The duodenum and large bowel exhibited highest variations, 24% and 39% from fractional values, for 5-fraction liver and pancreas. For liver 1-fraction, a 62% variation was observed for duodenum. Significance. The DOTD methodology provides an automated approach to quantify 3D dosimetric differences between online plans and their delivery. This analysis offers promise as a valuable tool for plan quality assessment and decision-making in the verification stage of the online workflow.

Robust optimization strategies for contour uncertainties in online adaptive radiation therapy

Objective. Online adaptive radiation therapy requires fast and automated contouring of daily scans for treatment plan re-optimization. However, automated contouring is imperfect and introduces contour uncertainties. This work aims at developing and comparing robust optimization strategies accounting for such uncertainties. Approach. A deep-learning method was used to predict the uncertainty of deformable image registration, and to generate a finite set of daily contour samples. Ten optimization strategies were compared: two baseline methods, five methods that convert contour samples into voxel-wise probabilities, and three methods accounting explicitly for contour samples as scenarios in robust optimization. Target coverage and organ-at-risk (OAR) sparing were evaluated robustly for simplified proton therapy plans for five head-and-neck cancer patients. Results. We found that explicitly including target contour uncertainty in robust optimization provides robust target coverage with better OAR sparing than the baseline methods, without increasing the optimization time. Although OAR doses first increased when increasing target robustness, this effect could be prevented by additionally including robustness to OAR contour uncertainty. Compared to the probability-based methods, the scenario-based methods spared the OARs more, but increased integral dose and required more computation time. Significance. This work proposed efficient and beneficial strategies to mitigate contour uncertainty in treatment plan optimization. This facilitates the adoption of automatic contouring in online adaptive radiation therapy and, more generally, enables mitigation also of other sources of contour uncertainty in treatment planning.

Adaptive assessment based on fractional CBCT images for cervical cancer.

Anatomical and other changes during radiotherapy will cause inaccuracy of dose distributions, therefore the expectation for online adaptive radiation therapy (ART) is high in effectively reducing uncertainties due to intra-variation. However, ART requires extensive time and effort. This study investigated an adaptive assessment workflow based on fractional cone-beam computed tomography (CBCT) images. Image registration, synthetic CT (sCT) generation, auto-segmentation, and dose calculation were implemented and integrated into ArcherQA Adaptive Check. The rigid registration was based on ITK open source. The deformable image registration (DIR) method was based on a 3D multistage registration network, and the sCT generation method was performed based on a 2D cycle-consistent adversarial network (CycleGAN). The auto-segmentation of organs at risk (OARs) on sCT images was finished by a deep learning-based auto-segmentation software, DeepViewer. The contours of targets were obtained by the structure-guided registration. Finally, the dose calculation was based on a GPU-based Monte Carlo (MC) dose code, ArcherQA. The dice similarity coefficient (DSCs) were over 0.86 for target volumes and over 0.79 for OARs. The gamma pass rate of ArcherQA versus Eclipse treatment planning system was more than 99% at the 2%/2mm criterion with a low-dose threshold of 10%. The time for the whole process was less than 3 min. The dosimetric results of ArcherQA Adaptive Check were consistent with the Ethos scheduled plan, which can effectively identify the fractions that need the implementation of the Ethos adaptive plan. This study integrated AI-based technologies and GPU-based MC technology to evaluate the dose distributions using fractional CBCT images, demonstrating remarkably high efficiency and precision to support future ART processes.

Modelling systematic anatomical uncertainties of head and neck cancer patients during fractionated radiotherapy treatment

Objective. Head and neck cancer patients experience systematic as well as random day to day anatomical changes during fractionated radiotherapy treatment. Modelling the expected systematic anatomical changes could aid in creating treatment plans which are more robust against such changes. Approach. Inter- patient correspondence aligned all patients to a model space. Intra- patient correspondence between each planning CT scan and on treatment cone beam CT scans was obtained using diffeomorphic deformable image registration. The stationary velocity fields were then used to develop B-Spline based patient specific (SM) and population average (AM) models. The models were evaluated geometrically and dosimetrically. A leave-one-out method was used to compare the training and testing accuracy of the models. Main results. Both SMs and AMs were able to capture systematic changes. The average surface distance between the registration propagated contours and the contours generated by the SM was less than 2 mm, showing that the SM are able to capture the anatomical changes which a patient experiences during the course of radiotherapy. The testing accuracy was lower than the training accuracy of the SM, suggesting that the model overfits to the limited data available and therefore, also captures some of the random day to day changes. For most patients the AMs were a better estimate of the anatomical changes than assuming there were no changes, but the AMs could not capture the variability in the anatomical changes seen in all patients. No difference was seen in the training and testing accuracy of the AMs. These observations were highlighted in both the geometric and dosimetric evaluations and comparisons. Significance. In this work, a SM and AM are presented which are able to capture the systematic anatomical changes of some head and neck cancer patients over the course of radiotherapy treatment. The AM is able to capture the overall trend of the population, but there is large patient variability which highlights the need for more complex, capable population models.

Stability analysis of patient-specific 4DCT- and 4DCBCT-based correspondence models.

Surrogate-based motion compensation in stereotactic body radiation therapy (SBRT) strongly relies on a constant relationship between an external breathing signal and the internal tumor motion over the course of treatment, that is, a stable patient-specific correspondencemodel. This study aims to develop methods for analyzing the stability of correspondence models by integrating planning 4DCT and pretreatment 4D cone-beam computed tomography (4DCBCT) data and assessing the relation to patient-specific clinicalparameters. For correspondence modeling, a regression-based approach is applied, correlating patient-specific internal motion (vector fields computed by deformable image registration) and external breathing signals (recorded by Varian's RPM and RGSC system). To analyze correspondence model stability, two complementary methods are proposed. (1) Target volume-based analysis: 4DCBCT-based correspondence models predict clinical target volumes (GTV and internal target volume [ITV]) within the planning 4DCT, which are evaluated by overlap and distance measures (Dice similarity coefficient [DSC]/average symmetric surface distance [ASSD]). (2) System matrix-based analysis: 4DCBCT-based regression models are compared to 4DCT-based models using mean squared difference (MSD) and principal component analysis of the system matrices. Stability analysis results are correlated with clinical parameters. Both methods are applied to a dataset of 214 pretreatment 4DCBCT scans (Varian TrueBeam) from a cohort of 46 lung tumor patients treated with ITV-based SBRT (planning 4DCTs acquired with Siemens AS Open and SOMATOM go.OPEN Pro CT scanners). Consistent results across the two complementary analysis approaches (Spearman correlation coefficient of between system matrix-based MSD and GTV-based DSC/ASSD) were observed. Analysis showed that stability was not predominant, with 114/214 fraction-wise models not surpassing a threshold of for the GTV, and only 14/46 patients demonstrating a in all fractions. Model stability did not degrade over the course of treatment. The mean GTV-based DSC is (mean ASSD of ) and the respective ITV-based DSC is (mean ASSD of ). The clinical parameters showed a strong correlation between smaller tumor motion ranges and increasedstability. The proposed methods identify patients with unstable correspondence models prior to each treatment fraction, serving as direct indicators for the necessity of replanning and adaptive treatment approaches to account for internal-external motion variations throughout the course oftreatment.

Improvement of accumulated dose distribution in combined cervical cancer radiotherapy with deep learning-based dose prediction.

Difficulties remain in dose optimization and evaluation of cervical cancer radiotherapy that combines external beam radiotherapy (EBRT) and brachytherapy (BT). This study estimates and improves the accumulated dose distribution of EBRT and BT with deep learning-based dose prediction. A total of 30 patients treated with combined cervical cancer radiotherapy were enrolled in this study. The dose distributions of EBRT and BT plans were accumulated using commercial deformable image registration. A ResNet-101-based deep learning model was trained to predict pixel-wise dose distributions. To test the role of the predicted accumulated dose in clinic, each EBRT plan was designed using conventional method and then redesigned referencing the predicted accumulated dose distribution. Bladder and rectum dosimetric parameters and normal tissue complication probability (NTCP) values were calculated and compared between the conventional and redesigned accumulated doses. The redesigned accumulated doses showed a decrease in mean values of V50, V60, and D2cc for the bladder (-3.02%, -1.71%, and -1.19 Gy, respectively) and rectum (-4.82%, -1.97%, and -4.13 Gy, respectively). The mean NTCP values for the bladder and rectum were also decreased by 0.02‰ and 0.98%, respectively. All values had statistically significant differences (p < 0.01), except for the bladder D2cc (p = 0.112). This study realized accumulated dose prediction for combined cervical cancer radiotherapy without knowing the BT dose. The predicted dose served as a reference for EBRT treatment planning, leading to a superior accumulated dose distribution and lower NTCP values.

Deep learning-based deformable image registration with bilateral pyramid to align pre-operative and follow-up magnetic resonance imaging (MRI) scans.

The evaluation of brain tumor recurrence after surgery is based on the comparison between tumor regions on pre-operative and follow-up magnetic resonance imaging (MRI) scans in clinical practice. Accurate alignment of MRI scans is important in this evaluation process. However, existing methods often fail to yield accurate alignment due to substantial appearance and shape changes of tumor regions. The study aimed to improve this misalignment situation through multimodal information and compensation for shape changes. In this work, a deep learning-based deformation registration method using bilateral pyramid to create multi-scale image features was developed. Moreover, morphology operations were employed to build correspondence between the surgical resection on the follow-up and pre-operative MRI scans. Compared with baseline methods, the proposed method achieved the lowest mean absolute error of 1.82 mm on the public BraTS-Reg 2022 dataset. The results suggest that the proposed method is potentially useful for evaluating tumor recurrence after surgery. We effectively verified its ability to extract and integrate the information of the second modality, and also revealed the micro representation of tumor recurrence. This study can assist doctors in registering multiple sequence images of patients, observing lesions and surrounding areas, analyzing and processing them, and guiding doctors in their treatment plans.

Generalized div-curl based regularization for physically constrained deformable image registration

Variational image registration methods commonly employ a similarity metric and a regularization term that renders the minimization problem well-posed. However, many frequently used regularizations such as smoothness or curvature do not necessarily reflect the underlying physics that apply to anatomical deformations. This, in turn, can make the accurate estimation of complex deformations particularly challenging. Here, we present a new highly flexible regularization inspired from the physics of fluid dynamics which allows applying independent penalties on the divergence and curl of the deformations and/or their nth order derivative. The complexity of the proposed generalized div-curl regularization renders the problem particularly challenging using conventional optimization techniques. To this end, we develop a transformation model and an optimization scheme that uses the divergence and curl components of the deformation as control parameters for the registration. We demonstrate that the original unconstrained minimization problem reduces to a constrained problem for which we propose the use of the augmented Lagrangian method. Doing this, the equations of motion greatly simplify and become managable. Our experiments indicate that the proposed framework can be applied on a variety of different registration problems and produce highly accurate deformations with the desired physical properties.

Indirect deformable image registration using synthetic image generated by unsupervised deep learning

Background and purpose3D image registration is now common in many medical domains. Multimodal registration implies the use of different imaging modalities, which results in lower accuracy compared to monomodal registration. The aim of this study was to propose a novel approach for deformable image registration (DIR) that incorporates an unsupervised deep learning (DL)-based generation step. The objective was to reduce the challenge of multimodal registration to monomodal registration. Material and methodsTwo datasets from prostate radiotherapy patients were used to evaluate the proposed method. The first dataset consisted of Computed Tomography (CT)/ Cone Beam Computed Tomography (CBCT) pairs from 23 patients using different CBCT devices. The second dataset included Magnetic Resonance Imaging (MRI)/CT pairs from two different care centers, utilizing different MRI devices (0.35 T MRIdian MR-Linac, 1.5 T GE lightspeed MRI). Following a preprocessing step essential for ensuring DL synthesis accuracy and standardizing the database, synthetic CTs (sCTreg) were generated using an unsupervised conditional Generative Adversarial Network (cGAN). The generated sCTs from CBCT or MRI were then utilized for deformable registration with CT scans. This registration method was compared to three standard methods: rigid registration, Elastix registration based on BSplines, and VoxelMorph-based registration (applied exclusively to CBCT/CT). The endpoints of comparison were the dice coefficients calculated between delineated structures for both datasets. ResultsFor both datasets, intermediary sCT generation provided the highest dice coefficients. Dices reached 0.85, 0.85 and 0.75 for the prostate, bladder and rectum for the dataset 1 and 0.90, 0.95 and 0.87 respectively for the dataset 2. When the sCT were not used, dices reached 0.66, 0.78, 0.66 for the dataset 1 and 0.93, 0.87 and 0.84 for the dataset 2. Furthermore, the evaluation of the impact of registration on sCT generation showed that lower Mean Absolute Errors were obtained when the registration was conducted with a sCT. ConclusionsUsing unsupervised deep learning to synthesize intermediate sCT has led to improved registration accuracy in radiotherapy applications employing two distinct imaging modalities.

Deformable anthropomorphic pelvis phantom for dose accumulation verification

Objective. The validation of deformable image registration (DIR) for contour propagation is often done using contour-based metrics. Meanwhile, dose accumulation requires evaluation of voxel mapping accuracy, which might not be accurately represented by contour-based metrics. By fabricating a deformable anthropomorphic pelvis phantom, we aim to (1) quantify the voxel mapping accuracy for various deformation scenarios, in high- and low-contrast regions, and (2) identify any correlation between dice similarity coefficient (DSC), a commonly used contour-based metric, and the voxel mapping accuracy for each organ. Approach. Four organs, i.e. pelvic bone, prostate, bladder and rectum (PBR), were 3D printed using PLA and a Polyjet digital material, and assembled. The latter three were implanted with glass bead and CT markers within or on their surfaces. Four deformation scenarios were simulated by varying the bladder and rectum volumes. For each scenario, nine DIRs with different parameters were performed on RayStation v10B. The voxel mapping accuracy was quantified by finding the discrepancy between true and mapped marker positions, termed the target registration error (TRE). Pearson correlation test was done between the DSC and mean TRE for each organ. Main results. For the first time, we fabricated a deformable phantom purely from 3D printing, which successfully reproduced realistic anatomical deformations. Overall, the voxel mapping accuracy dropped with increasing deformation magnitude, but improved when more organs were used to guide the DIR or limit the registration region. DSC was found to be a good indicator of voxel mapping accuracy for prostate and rectum, but a comparatively poorer one for bladder. DSC > 0.85/0.90 was established as the threshold of mean TRE ⩽ 0.3 cm for rectum/prostate. For bladder, extra metrics in addition to DSC should be considered. Significance. This work presented a 3D printed phantom, which enabled quantification of voxel mapping accuracy and evaluation of correlation between DSC and voxel mapping accuracy.

Diffeomorphic transformer-based abdomen MRI-CT deformable image registration.

Stereotactic body radiotherapy (SBRT) is a well-established treatment modality for liver metastases in patients unsuitable for surgery. Both CT and MRI are useful during treatment planning for accurate target delineation and to reduce potential organs-at-risk (OAR) toxicity from radiation. MRI-CT deformable image registration (DIR) is required to propagate the contours defined on high-contrast MRI to CT images. An accurate DIR method could lead to more precisely defined treatment volumes and superior OAR sparing on the treatment plan. Therefore, it is beneficial to develop an accurate MRI-CT DIR for liver SBRT. To create a new deep learning model that can estimate the deformation vector field (DVF) for directly registering abdominal MRI-CT images. The proposed method assumed a diffeomorphic deformation. By using topology-preserved deformation features extracted from the probabilistic diffeomorphic registration model, abdominal motion can be accurately obtained and utilized for DVF estimation. The model integrated Swin transformers, which have demonstrated superior performance in motion tracking, into the convolutional neural network (CNN) for deformation feature extraction. The model was optimized using a cross-modality image similarity loss and a surface matching loss. To compute the image loss, a modality-independent neighborhood descriptor (MIND) was used between the deformed MRI and CT images. The surface matching loss was determined by measuring the distance between the warped coordinates of the surfaces of contoured structures on the MRI and CT images. To evaluate the performance of the model, a retrospective study was carried out on a group of 50 liver cases that underwent rigid registration of MRI and CT scans. The deformed MRI image was assessed against the CT image using the target registration error (TRE), Dice similarity coefficient (DSC), and mean surface distance (MSD) between the deformed contours of the MRI image and manual contours of the CT image. When compared to only rigid registration, DIR with the proposed method resulted in an increase of the mean DSC values of the liver and portal vein from 0.850±0.102 and 0.628±0.129 to 0.903±0.044 and 0.763±0.073, a decrease of the mean MSD of the liver from 7.216±4.513mm to 3.232±1.483mm, and a decrease of the TRE from 26.238±2.769mm to 8.492±1.058mm. The proposed DIR method based on a diffeomorphic transformer provides an effective and efficient way to generate an accurate DVF from an MRI-CT image pair of the abdomen. It could be utilized in the current treatment planning workflow for liver SBRT.

Uncertainty estimation and evaluation of deformation image registration based convolutional neural networks

Objective. Fast and accurate deformable image registration (DIR), including DIR uncertainty estimation, is essential for safe and reliable clinical deployment. While recent deep learning models have shown promise in predicting DIR with its uncertainty, challenges persist in proper uncertainty evaluation and hyperparameter optimization for these methods. This work aims to develop and evaluate a model that can perform fast DIR and predict its uncertainty in seconds. Approach. This study introduces a novel probabilistic multi-resolution image registration model utilizing convolutional neural networks to estimate a multivariate normal distributed dense displacement field (DDF) in a multimodal image registration problem. To assess the quality of the DDF distribution predicted by the model, we propose a new metric based on the Kullback–Leibler divergence. The performance of our approach was evaluated against three other DIR algorithms (VoxelMorph, Monte Carlo dropout, and Monte Carlo B-spline) capable of predicting uncertainty. The evaluation of the models included not only the quality of the deformation but also the reliability of the estimated uncertainty. Our application investigated the registration of a treatment planning computed tomography (CT) to follow-up cone beam CT for daily adaptive radiotherapy. Main results. The hyperparameter tuning of the models showed a trade-off between the estimated uncertainty’s reliability and the deformation’s accuracy. In the optimal trade-off, our model excelled in contour propagation and uncertainty estimation (p <0.05) compared to existing uncertainty estimation models. We obtained an average dice similarity coefficient of 0.89 and a KL-divergence of 0.15. Significance. By addressing challenges in DIR uncertainty estimation and evaluation, our work showed that both the DIR and its uncertainty can be reliably predicted, paving the way for safe deployment in a clinical environment.

Pure large kernel convolutional neural network transformer for medical image registration

Deformable medical image registration is a fundamental and critical task in medical image analysis. Recently, deep learning-based methods have rapidly developed and have shown impressive results in deformable image registration. However, existing approaches still suffer from limitations in registration accuracy or generalization performance. To address these challenges, in this paper, we propose a pure convolutional neural network module (CVTF) to implement hierarchical transformers and enhance the registration performance of medical images. CVTF has a larger convolutional kernel, providing a larger global effective receptive field, which can improve the network’s ability to capture long-range dependencies. In addition, we introduce the spatial interaction attention (SIA) module to compute the interrelationship between the target feature pixel points and all other points in the feature map. This helps to improve the semantic understanding of the model by emphasizing important features and suppressing irrelevant ones. Based on the proposed CVTF and SIA, we construct a novel registration framework named PCTNet. We applied PCTNet to generate displacement fields and register medical images, and we conducted extensive experiments and validation on two public datasets, OASIS and LPBA40. The experimental results demonstrate the effectiveness and generality of our method, showing significant improvements in registration accuracy and generalization performance compared to existing methods. Our code has been available at https://github.com/fz852/PCTNet.