Deformation Vector Fields Research Articles

Technical Note: Four-dimensional deformable digital phantom for MRI sequence development.

MR-guided radiotherapy has different requirements for the images than diagnostic radiology, thus requiring development of novel imaging sequences. MRI simulation is an excellent tool for optimizing these new sequences; however, currently available software does not provide all the necessary features. In this paper, we present a digital framework for testing MRI sequences that incorporates anatomical structure, respiratory motion, and realistic presentation of MR physics. The extended Cardiac-Torso (XCAT) software was used to create T1 , T2 , and proton density maps that formed the anatomical structure of the phantom. Respiratory motion model was based on the XCAT deformation vector fields, modified to create a motion model driven by a respiration signal. MRI simulation was carried out with JEMRIS, an open source Bloch simulator. We developed an extension for JEMRIS, which calculates the motion of each spin independently, allowing for deformable motion. The performance of the framework was demonstrated through simulating the acquisition of a two-dimensional (2D) cine and demonstrating expected motion ghosts from T2 weighted spin echo acquisitions with different respiratory patterns. All simulations were consistent with behavior previously described in literature. Simulations with deformable motion were not more time consuming than with rigid motion. We present a deformable four-dimensional (4D) digital phantom framework for MR sequence development. The framework incorporates anatomical structure, realistic breathing patterns, deformable motion, and Bloch simulation to achieve accurate simulation of MRI. This method is particularly relevant for testing novel imaging sequences for the purpose of MR-guided radiotherapy in lungs and abdomen.

Real-time CT image generation based on voxel-by-voxel modeling of internal deformation by utilizing the displacement of fiducial markers.

To show the feasibility of real-time CT image generation technique utilizing internal fiducial markers that facilitate the evaluation of internal deformation. In the proposed method, a linear regression model that can derive internal deformation from the displacement of fiducial markers is built for each voxel in the training process before the treatment session. Marker displacement and internal deformation are derived from the four-dimensional computed tomography (4DCT) dataset. In the treatment session, the three-dimensional deformation vector field is derived according to the marker displacement, which is monitored by the real-time imaging system. The whole CT image can be synthesized by deforming the reference CT image with a deformation vector field in real-time. To show the feasibility of the technique, image synthesis accuracy and tumor localization accuracy were evaluated using the dataset generated by extended NURBS-Based Cardiac-Torso (XCAT) phantom and clinical 4DCT datasets from six patients, containing 10 CT datasets each. In the validation with XCAT phantom, motion range of the tumor in training data and validation data were about 10 and 15mm, respectively, so as to simulate motion variation between 4DCT acquisition and treatment session. In the validation with patient 4DCT dataset, eight CT datasets from the 4DCT dataset were used in the training process. Two excluded inhale CT datasets can be regarded as the datasets with large deformations more than training dataset. CT images were generated for each respiratory phase using the corresponding marker displacement. Root mean squared error (RMSE), normalized RMSE (NRMSE), and structural similarity index measure (SSIM) between the original CT images and the synthesized CT images were evaluated as the quantitative indices of the accuracy of image synthesis. The accuracy of tumor localization was also evaluated. In the validation with XCAT phantom, the mean NRMSE, SSIM, and three-dimensional tumor localization error were 7.5±1.1%, 0.95±0.02, and 0.4±0.3mm, respectively. In the validation with patient 4DCT dataset, the mean RMSE, NRMSE, SSIM, and three-dimensional tumor localization error in six patients were 73.7±19.6 HU, 9.2±2.6%, 0.88±0.04, and 0.8±0.6mm, respectively. These results suggest that the accuracy of the proposed technique is adequate when the respiratory motion is within the range of the training dataset. In the evaluation with a marker displacement larger than that of the training dataset, the mean RMSE, NRMSE, and tumor localization error were about 100 HU, 13%, and <2.0mm, respectively, except for one case having large motion variation. The performance of the proposed method was similar to those of previous studies. Processing time to generate the volumetric image was <100ms. We have shown the feasibility of the real-time CT image generation technique for volumetric imaging.

Bilateral filtering based sliding motion compensated 4D-CBCT: a simulation study

Objective This study reconstructed 4D-CBCT for fully automatic compensated sliding motion by incorporating the bilateral filtering into the Deformable Vector Field (DVF). Methods First, a motion compensated simultaneous algebraic reconstruction technique (Modified Simultaneous Algebra Reconstruction Technique, mSART) was used to generate a high quality reference phase by using all phase projection stogether with the initial 4D-DVFs, which were generated via Demons registration between 0% phase and each other phaseimage. The 4D-DVF was optimized by matching the forward projection of the deformed 0% phase with the measured projection of the target phase. The loss function’s DVF smoothing constrain term contained bilateral filtering kernel that contained: 1) an spatial domain Guassian kernel; 2) animage intensity domain Guassian kernel; and 3) a DVF domain Guassian kernel. By choosing suitable kernel variances, the sliding motion can be extracted. A non-linear conjugate gradient optimizer wasused. We validated the algorithm on a Non-Uniform Rotational B-spline based Cardiac-Torso (NCAT) phantom. Quantification was evaluated by: 1) the Root-Mean-Square-Error (RMSE) together with the Maximum-Error (MaxE); 2) the Dice coefficient of the extracted lung contour from the final reconstructed images and 3) the relative reconstruction error (RE) to evaluate the algorithm’s performance. Results The motion trajectory’s RMSE/MaxEare 0.796/1.02 mm for bilateral filtering reconstruction; and 2.704/4.08 mm for original reconstruction. Image content such a stherib position, the hearted gedefinition, the fibrous structures all had been better corrected with bilateral filtering. Conclusion We developed a bilateral filtering based fully automatic sliding motion compensated 4D-CBCT scheme. Digital phantom study confirmed the improved motion estimation and image reconstruction ability. It can be used as a 4D-CBCT image guidance tool for lung SBRT treatment. 摘要： 目的 将双边滤波引入基于可变形矢量场 (DVF) 的 4D-CBCT 重建, 实现全自动滑动运动补偿 4D-CBCT 重 建。 方法 首先利用所有相位投影, 用改良的运动补偿瞬时代数重建技术 (Modified Simultaneous Algebra Reconstruction Technique, mSART) 生成高质 量参考相位。初始 4D-DVF 通过 0% 相位和其他相位图像依次配准生成。之后通过 配准目标相位测量投影和参考相位变形到目标相位后的正投来优化求解 4D-DVF。优化过程中的损失函数平滑约束 项中引入双边滤波。其包含 3 个子核:空间域 Guassian 核; 图像强度域 Guassian 核; 和 DVF 域 Guassian 核。选择合适 的子核方差提取滑动运动, 采用非线性共轭梯度算子优化, 用 B 样条心脏躯干体模 (NURBS-based Cardiac-Torso phantom, NCAT phantom) 验证算法。采用量化评价指标: Root-Mean-Square-Error (RMSE) 和最大误差 (MaxE); 重建图 像提取的肺轮廓 Dice 系数和相对重建误差 (RE) 评价算法性能。 结果 NCAT 模体的双边滤波重建运动轨迹的 RMSE/MaxE 为 0.796/1.02 mm; 原始重建方法的相应结果为 2.704/4.08 mm。图像中的特定结构如肋骨位置, 心脏边缘 的定义, 纤维结构通过双边过滤都得到了更好的纠正。 结论 开发了一种基于双边滤波的全自动滑动运动补偿 4D-CBCT 方案, 数字模体研宄证实了改进的运动估计和图像重建能力, 其可被用作肺 SBRT 治疗的 4D-CBCT 图像引 导工具。

Deep Learning–Based CT-to-CBCT Deformable Image Registration for Autosegmentation in Head and Neck Adaptive Radiation Therapy

The purpose of this study is to develop a deep learning–based method that can automatically generate segmentations on conebeam computed tomography (CBCT) for head and neck online adaptive radiation therapy (ART), where expert-drawn contours in planning CT (pCT) images serve as prior knowledge. Because of the many artifacts and truncations that characterize CBCT, we propose to utilize a learning-based deformable image registration method and contour propagation to get updated contours on CBCT. Our method takes CBCT and pCT as inputs, and it outputs a deformation vector field and synthetic CT (sCT) simultaneously by jointly training a CycleGAN model and 5-cascaded Voxelmorph model. The CycleGAN generates the sCT from CBCT, while the 5-cascaded Voxelmorph warps the pCT to the sCT’s anatomy. We compared the segmentation results to Elastix, Voxelmorph and 5-cascaded Voxelmorph models on 18 structures including target and organs-at-risk. Our proposed method achieved an average Dice similarity coefficient of 0.83 ± 0.09 and an average 95% Hausdorff distance of 2.01 ± 1.81 mm. Our method showed better accuracy than Voxelmorph and 5-cascaded Voxelmorph and comparable accuracy to Elastix, but with much higher efficiency. The proposed method can rapidly and simultaneously generate sCT with correct CT numbers and propagate contours from pCT to CBCT for online ART replanning.

A generative adversarial network (GAN)-based technique for synthesizing realistic respiratory motion in the extended cardiac-torso (XCAT) phantoms

Objective. Synthesize realistic and controllable respiratory motions in the extended cardiac-torso (XCAT) phantoms by developing a generative adversarial network (GAN)-based deep learning technique. Methods. A motion generation model was developed using bicycle-GAN with a novel 4D generator. Input with the end-of-inhale (EOI) phase images and a Gaussian perturbation, the model generates inter-phase deformable-vector-fields (DVFs), which were composed and applied to the input to generate 4D images. The model was trained and validated using 71 4D-CT images from lung cancer patients and then applied to the XCAT EOI images to generate 4D-XCAT with realistic respiratory motions. A separate respiratory motion amplitude control model was built using decision tree regression to predict the input perturbation needed for a specific motion amplitude, and this model was developed using 300 4D-XCAT generated from 6 XCAT phantom sizes with 50 different perturbations for each size. In both patient and phantom studies, Dice coefficients for lungs and lung volume variation during respiration were compared between the simulated images and reference images. The generated DVF was evaluated by deformation energy. DVFs and ventilation maps of the simulated 4D-CT were compared with the reference 4D-CTs using cross correlation and Spearman’s correlation. Comparison of DVFs and ventilation maps among the original 4D-XCAT, the generated 4D-XCAT, and reference patient 4D-CTs were made to show the improvement of motion realism by the model. The amplitude control error was calculated. Results. Comparing the simulated and reference 4D-CTs, the maximum deviation of lung volume during respiration was 5.8%, and the Dice coefficient reached at least 0.95 for lungs. The generated DVFs presented comparable deformation energy levels. The cross correlation of DVFs achieved 0.89 ± 0.10/0.86 ± 0.12/0.95 ± 0.04 along the x/y/z direction in the testing group. The cross correlation of ventilation maps derived achieved 0.80 ± 0.05/0.67 ± 0.09/0.68 ± 0.13, and the Spearman’s correlation achieved 0.70 ± 0.05/0, 60 ± 0.09/0.53 ± 0.01, respectively, in the training/validation/testing groups. The generated 4D-XCAT phantoms presented similar deformation energy as patient data while maintained the lung volumes of the original XCAT phantom (Dice = 0.95, maximum lung volume variation = 4%). The motion amplitude control models controlled the motion amplitude control error to be less than 0.5 mm. Conclusions. The results demonstrated the feasibility of synthesizing realistic controllable respiratory motion in the XCAT phantom using the proposed method. This crucial development enhances the value of XCAT phantoms for various 4D imaging and therapy studies.

An approach for estimating dosimetric uncertainties in deformable dose accumulation in pencil beam scanning proton therapy for lung cancer

Deformable image registration (DIR) is an important component for dose accumulation and associated clinical outcome evaluation in radiotherapy. However, the resulting deformation vector field (DVF) is subject to unavoidable discrepancies when different algorithms are applied, leading to dosimetric uncertainties of the accumulated dose. We propose here an approach for proton therapy to estimate dosimetric uncertainties as a consequence of modeled or estimated DVF uncertainties. A patient-specific DVF uncertainty model was built on the first treatment fraction, by correlating the magnitude differences of five DIR results at each voxel to the magnitude of any single reference DIR. In the following fractions, only the reference DIR needs to be applied, and DVF geometric uncertainties were estimated by this model. The associated dosimetric uncertainties were then derived by considering the estimated geometric DVF uncertainty, the dose gradient of fractional recalculated dose distribution and the direction factor from the applied reference DIR of this fraction. This estimated dose uncertainty was respectively compared to the reference dose uncertainty when different DIRs were applied individually for each dose warping. This approach was validated on seven NSCLC patients, each with nine repeated CTs. The proposed model-based method is able to achieve dose uncertainty distribution on a conservative voxel-to-voxel comparison within ±5% of the prescribed dose to the ‘reference’ dosimetric uncertainty, for 77% of the voxels in the body and 66%–98% of voxels in investigated structures. We propose a method to estimate DIR induced uncertainties in dose accumulation for proton therapy of lung tumor treatments.

Validation of accuracy deformable image registration contour propagation using a benchmark virtual HN phantom dataset.

Virtual anatomic phantoms offer precise voxel mapping of the variation of anatomy with ground truth deformation vector fields (DVFs). Dice similarity coefficient (DSC) and mean distance to agreement (MDA) are the standard metrics for evaluating geometric contour congruence when testing deformable registration (DIR) algorithms. A HN virtual patient phantom data set was used for a kVCT‐kVCT automatic propagation contour validation study employing the Accuray DIR algorithm. Furthermore, since TomoTherapy uses MVCT images of the relevant anatomy for adaptive monitoring, the kVCT image data set quality was transformed to an MVCT image data set quality to study intermodal kVCT‐MVCT DIR accuracy. The results of the study indicate that the Accuray DIR algorithm can be expected to autopropagate HN contours adequately, on average, within tolerances recommended by TG‐132 (DSC 0.8‐0.9, MDA within voxel width). However, contours critical to dosimetric planning should always be visually proofed for accuracy. Using standard reconstruction MVCT image quality causes slightly less, but acceptable, agreement with ground truth contours.

Simultaneous multi-slice accelerated 4D-MRI for radiotherapy guidance

4D-MRI is becoming increasingly important for daily guidance of thoracic and abdominal radiotherapy. This study exploits the simultaneous multi-slice (SMS) technique to accelerate the acquisition of a balanced turbo field echo (bTFE) and a turbo spin echo (TSE) coronal 4D-MRI sequence performed on 1.5 T MRI scanners. SMS single-shot bTFE and TSE sequences were developed to acquire a stack of 52 coronal 2D images over 30 dynamics. Simultaneously excited slices were separated by half the field of view. Slices intersecting with the liver-lung interface were used as navigator slices. For each navigator slice location, an end-exhale dynamic was automatically identified, and used to derive the self-sorting signal by rigidly registering the remaining dynamics. Navigator slices were sorted into 10 amplitude bins, and the temporal relationship of simultaneously excited slices was used to generate sorted 4D-MRIs for 12 healthy volunteers. The self-sorting signal was validated using an in vivo peak-to-peak motion analysis. The smoothness of the liver-lung interface was quantified by comparing to sagittal cine images acquired directly after the SMS-4D-MRI sequence. To ensure compatibility with the MR-linac radiotherapy workflow, the 4D-MRIs were transformed into 3D mid-position (MidP) images using deformable image registration. Consistency of the deformable vector fields was quantified in terms of the distance discordance metric (DDM) in the body. The SMS-4D-TSE sequence was additionally acquired for 3 lung cancer patients to investigate tumor visibility. SMS-4D-MRI acquisition and processing took approximately 7 min. 4D-MRI reconstruction was possible for 26 out of 27 acquired datasets. Missing data in the sorted 4D-MRIs varied from 4%–26% for the volunteers and varied from 8%–24% for the patients. Peak-to-peak (SD) amplitudes analysis agreed within 1.8 (1.1) mm and 0.9 (0.4) mm between the sorted 4D-MRIs and the self-sorting signals of the volunteers and patients, respectively. Liver-lung interface smoothness was found to be in the range of 0.6–3.1 mm for volunteers. The percentage of DDM values smaller than 2 mm was in the range of 85%–89% and 86%–92% for the volunteers and patients, respectively. Lung tumors were clearly visibility in the SMS-4D-TSE images and MidP images. Two fast SMS-accelerated 4D-MRI sequences were developed resulting in T2/T1 or T2 weighted contrast. The SMS-4D-MRIs and derived 3D MidP-MRIs yielded anatomically plausible images and good tumor visibility. SMS-4D-MRI is therefore a strong candidate to be used for treatment simulation and daily guidance of thoracic and abdominal MR-guided radiotherapy.

A 3D cine-MRI acquisition technique and image analysis framework to quantify bowel motion demonstrated in gynecological cancer patients.

PurposeMagnetic resonance imaging (MRI) is increasingly used in radiation oncology for target delineation and radiotherapy treatment planning, for example, in patients with gynecological cancers. As a consequence of pelvic radiotherapy, a part of the bowel is irradiated, yielding risk of bowel toxicity. Existing dose‐effect models predicting bowel toxicity are inconclusive and bowel motion might be an important confounding factor. The exact motion of the bowel and dosimetric effects of its motion are yet uncharted territories in radiotherapy. In diagnostic radiology methods on the acquisition of dynamic MRI sequences were developed for bowel motility visualization and quantification. Our study aim was to develop an imaging technique based on three‐dimensional (3D) cine‐MRI to visualize and quantify bowel motion and demonstrate it in a cohort of gynecological cancer patients.MethodsWe developed an MRI acquisition suitable for 3D bowel motion quantification, namely a balanced turbo field echo sequence (TE = 1.39 ms, TR = 2.8 ms), acquiring images in 3.7 s (dynamic) with a 1.25 × 1.25 × 2.5 mm3 resolution, yielding a field of view of 200 × 200 × 125 mm3. These MRI bowel motion sequences were acquired in 22 gynecological patients. During a 10‐min scan, 160 dynamics were acquired. Subsequent dynamics were deformably registered using a B‐spline transformation model, resulting in 159 3D deformation vector fields (DVFs) per MRI set. From the 159 DVFs, the average vector length was calculated per voxel to generate bowel motion maps. Quality assurance was performed on all 159 DVFs per MRI, using the Jacobian Determinant and the Harmonic Energy as deformable image registration error metrics. In order to quantify bowel motion, we introduced the concept of cumulative motion–volume histogram (MVH) of the bowel bag volume. Finally, interpatient variation of bowel motion was analyzed using the MVH parameters M10%, M50%, and M90%. The M10%/M50%/M90% represents the minimum bowel motion per frame of 10%/50%/90% of the bowel bag volume.ResultsThe motion maps resulted in a visualization of areas with small and large movements within the bowel bag. After applying quality assurance, the M10%, M50%, and M90% were 4.4 (range 2.2–7.6) mm, 2.2 (range 0.9–4.1) mm, and 0.5 (range 0.2–1.4) mm per frame, on average over all patients, respectively.ConclusionWe have developed a method to visualize and quantify 3D bowel motion with the use of bowel motion specific MRI sequences in 22 gynecological cancer patients. This 3D cine‐MRI‐based quantification tool and the concept of MVHs can be used in further studies to determine the effect of radiotherapy on bowel motion and to find the relation with dose effects to the small bowel. In addition, the developed technique can be a very interesting application for bowel motility assessment in diagnostic radiology.

Variability in commercially available deformable image registration: A multi‐institution analysis using virtual head and neck phantoms

PurposeThe purpose of this study was to evaluate the performance of three common deformable image registration (DIR) packages across algorithms and institutions.Methods and MaterialsThe Deformable Image Registration Evaluation Project (DIREP) provides ten virtual phantoms derived from computed tomography (CT) datasets of head‐and‐neck cancer patients over a single treatment course. Using the DIREP phantoms, DIR results from 35 institutions were submitted using either Velocity, MIM, or Eclipse. Submitted deformation vector fields (DVFs) were compared to ground‐truth DVFs to calculate target registration error (TRE) for six regions of interest (ROIs). Statistical analysis was performed to determine the variability between each DIR software package and the variability of users within each algorithm.ResultsOverall mean TRE was 2.04 ± 0.35 mm for Velocity, 1.10 ± 0.29 mm for MIM, and 2.35 ± 0.15 mm for Eclipse. The MIM mean TRE was significantly different than both Velocity and Eclipse for all ROIs. Velocity and Eclipse mean TREs were not significantly different except for when evaluating the registration of the cord or mandible. Significant differences between institutions were found for the MIM and Velocity platforms. However, these differences could be explained by variations in Velocity DIR parameters and MIM software versions.ConclusionsAverage TRE was shown to be <3 mm for all three software platforms. However, maximum errors could be larger than 2 cm indicating that care should be exercised when using DIR. While MIM performed statistically better than the other packages, all evaluated algorithms had an average TRE better than the largest voxel dimension. For the phantoms studied here, significant differences between algorithm users were minimal suggesting that the algorithm used may have more impact on DIR accuracy than the particular registration technique employed. A significant difference in TRE was discovered between MIM versions showing that DIR QA should be performed after software upgrades as recommended by TG‐132.

An unsupervised 2D–3D deformable registration network (2D3D-RegNet) for cone-beam CT estimation

Acquiring CBCTs from a limited scan angle can help to reduce the imaging time, save the imaging dose, and allow continuous target localizations through arc-based treatments with high temporal resolution. However, insufficient scan angle sampling leads to severe distortions and artifacts in the reconstructed CBCT images, limiting their clinical applicability. 2D–3D deformable registration can map a prior fully-sampled CT/CBCT volume to estimate a new CBCT, based on limited-angle on-board cone-beam projections. The resulting CBCT images estimated by 2D–3D deformable registration can successfully suppress the distortions and artifacts, and reflect up-to-date patient anatomy. However, traditional iterative 2D–3D deformable registration algorithm is very computationally expensive and time-consuming, which takes hours to generate a high quality deformation vector field (DVF) and the CBCT. In this work, we developed an unsupervised, end-to-end, 2D–3D deformable registration framework using convolutional neural networks (2D3D-RegNet) to address the speed bottleneck of the conventional iterative 2D–3D deformable registration algorithm. The 2D3D-RegNet was able to solve the DVFs within 5 seconds for 90 orthogonally-arranged projections covering a combined 90° scan angle, with DVF accuracy superior to 3D–3D deformable registration, and on par with the conventional 2D–3D deformable registration algorithm. We also performed a preliminary robustness analysis of 2D3D-RegNet towards projection angular sampling frequency variations, as well as scan angle offsets. The synergy of 2D3D-RegNet with biomechanical modeling was also evaluated, and demonstrated that 2D3D-RegNet can function as a fast DVF solution core for further DVF refinement.

Benchmarking of Deformable Image Registration for Multiple Anatomic Sites Using Digital Data Sets With Ground-Truth Deformation Vector Fields

PurposeThis study aimed to evaluate the accuracy of deformable image registration (DIR) algorithms using data sets with different levels of ground-truth deformation vector fields (DVFs) and to investigate the correlation between DVF errors and contour-based metrics. Methods and MaterialsNine pairs of digital data sets were generated through contour-controlled deformations based on 3 anonymized patients’ CTs (head and neck, thorax/abdomen, and pelvis) with low, medium, and high deformation intensity for each site using the ImSimQA software. Image pairs and their associated contours were imported to MIM-Maestro, Raystation, and Velocity systems, followed by DIR and contour propagation. The system-generated DVF and propagated contours were compared with the ground-truth data. The correlation between DVF errors and contour-based metrics was evaluated using the Pearson correlation coefficient (r), while their correlation with volumes were calculated using Spearman correlation coefficient (rho). ResultsThe DVF errors increased with increasing deformation intensity. All DIR algorithms performed well for esophagus, trachea, left femoral, right femoral, and urethral (mean and maximum DVF errors <2.50 mm and <4.27 mm, respectively; Dice similarity coefficient: 0.93-0.99). Brain, liver, left lung, and bladder showed large DVF errors for all 3 systems (dmax: 2.8-91.90 mm). The minimum and maximum DVF errors, conformity index, and Dice similarity coefficient were correlated with volumes (|rho|: 0.41-0.64), especially for very large or small structures (|rho|: 0.64-0.80). Only mean distance to agreement of Raystation and Velocity correlated with some indices of DVF errors (r: 0.70-0.78). ConclusionsMost contour-based metrics had no correlation with DVF errors. For adaptive radiation therapy, well-performed contour propagation does not directly indicate accurate dose deformation and summation/accumulation within each contour (determined by DVF accuracy). Tolerance values for DVF errors should vary as the acceptable accuracy for overall adaptive radiation therapy depends on anatomic site, deformation intensity, organ size, and so forth. This study provides benchmark tables for evaluating DIR accuracy in various clinical scenarios.

Shape-constrained Gaussian process regression for surface reconstruction and multimodal, non-rigid image registration

We present a new statistical framework for landmark ?>curve-based image registration and surface reconstruction. The proposed method first elastically aligns geometric features (continuous, parameterized curves) to compute local deformations, and then uses a Gaussian random field model to estimate the full deformation vector field as a spatial stochastic process on the entire surface or image domain. The statistical estimation is performed using two different methods: maximum likelihood and Bayesian inference via Markov Chain Monte Carlo sampling. The resulting deformations accurately match corresponding curve regions while also being sufficiently smooth over the entire domain. We present several qualitative and quantitative evaluations of the proposed method on both synthetic and real data. We apply our approach to two different tasks on real data: (1) multimodal medical image registration, and (2) anatomical and pottery surface reconstruction.

Training deep-learning segmentation models from severely limited data.

To enable generation of high-quality deep learning segmentation models from severely limited contoured cases (e.g., ~10 cases). Thirty head and neck computed tomography (CT) scans with well-defined contours were deformably registered to 200 CT scans of the same anatomic site without contours. Acquired deformation vector fields were used to train a principal component analysis (PCA) model for each of the 30 contoured CT scans by capturing the mean deformation and most prominent variations. Each PCA model can produce an infinite number of synthetic CT scans and corresponding contours by applying random deformations. We used 300, 600, 1000, and 2000 synthetic CT scans and contours generated from one PCA model to train V-Net, a 3D convolutional neural network architecture, to segment parotid and submandibular glands. We repeated the training using same numbers of training cases generated from 7, 10, 20, and 30 PCA models, with the data distributed evenly between each PCA model. Performance of the segmentation models was evaluated with Dice similarity coefficients between auto-generated contours and physician-drawn contours on 162 test CT scans for parotid glands and another 21 test CT scans for submandibular glands. Dice values varied with the number of synthetic CT scans and the number of PCA models used to train the network. By using 2000 synthetic CT scans generated from 10 PCA models, we achieved Dice values of 82.8%±6.8% for right parotid, 82.0%±6.9% for left parotid, and 74.2%±6.8% for submandibular glands. These results are comparable with those obtained from state-of-the-art auto-contouring approaches, including a deep learning network trained from more than 1000 contoured patients and a multi-atlas algorithm from 12 well-contoured atlases. Improvement was marginal when >10 PCA models or >2000 synthetic CT scans were used. We demonstrated an effective data augmentation approach to train high-quality deep learning segmentation models from a limited number of well-contoured patient cases.

Porcine lung phantom-based validation of estimated 4D-MRI using orthogonal cine imaging for low-field MR-Linacs

Real-time motion monitoring of lung tumors with low-field magnetic resonance imaging-guided linear accelerators (MR-Linacs) is currently limited to sagittal 2D cine magnetic resonance imaging (MRI). To provide input data for improved intrafractional and interfractional adaptive radiotherapy, the 4D anatomy has to be inferred from data with lower dimensionality. The purpose of this study was to experimentally validate a previously proposed propagation method that provides continuous time-resolved estimated 4D-MRI based on orthogonal cine MRI for a low-field MR-Linac. Ex vivo porcine lungs were injected with artificial nodules and mounted in a dedicated phantom that allows for the simulation of periodic and reproducible breathing motion. The phantom was scanned with a research version of a commercial 0.35 T MR-Linac. Respiratory-correlated 4D-MRI were reconstructed and served as ground truth images. Series of interleaved orthogonal slices in sagittal and coronal orientation, intersecting the injected targets, were acquired at 7.3 Hz. Estimated 4D-MRI at 3.65 Hz were created in post-processing using the propagation method and compared to the ground truth 4D-MRI. Eight datasets at different breathing frequencies and motion amplitudes were acquired for three porcine lungs. The overall median (95 percentile) deviation between ground truth and estimated deformation vector fields was 2.3 mm (5.7 mm), corresponding to 0.7 (1.6) times the in-plane imaging resolution (3.5 × 3.5 mm2). Median (95 percentile) estimated nodule position errors were 1.5 mm (3.8 mm) for nodules intersected by orthogonal slices and 2.1 mm (7.1 mm) for nodules located more than 2 cm away from either of the orthogonal slices. The estimation error depended on the breathing phase, the motion amplitude and the location of the estimated position with respect to the orthogonal slices. By using the propagation method, the 4D motion within the porcine lung phantom could be accurately and robustly estimated. The method could provide valuable information for treatment planning, real-time motion monitoring, treatment adaptation, and post-treatment evaluation of MR-guided radiotherapy treatments.

Respiratory deformation registration in 4D-CT/cone beam CT using deep learning.

To investigate the feasibility of using a supervised convolutional neural network (CNN) to register phase-to-phase deformable vector field of lung 4D-CT/4D-cone beam CT for 4D dose accumulation, contour propagation, motion modeling, or target verification. We built a CNN-based deep learning method to register the deformation field directly between phases of patients' 4D-CT or 4D-cone beam CT. The input consists of patch pairs of two phases, while the output is the corresponding deformation field that registers the patch pairs. The centers of the patch pairs were uniformly sampled across the lung, and the size of the patches was chosen to cover the range of the respiratory motion. The network was trained to generate deformation field that matches with the reference deformation field generated by VelocityAI (Varian). The network is structured with four convolutional layers, two average pooling layers, and two fully connected layers. Half mean squared error is applied to guide the study as loss function. Nine patients with eleven sets of 4D-CT/cone beam CT image volumes were used for training and testing. The performance of the network was validated with intra-patient and inter-patient setups. Registered images were generated with Velocity deformation field and the CNN deformation field, respectively. Main anatomic features such as the main vessels and the diaphragm matched well between two deformed images. In the diaphragm region, the coefficients of cross-correlation, root mean squared error, and structural similarity index measure (SSIM) between deformed images registered by CNN and VelocityAI was calculated. The cross-correlation was above 0.9 for all the intra-patient cases. Patch-based deep learning methods achieved comparable deformable registration accuracy as VelocityAI. Compared to VelocityAI, the deep learning method is fully automatic and faster without user dependency, which makes it more preferable in clinical applications.

Liver-ultrasound-guided lung tumour tracking for scanned proton therapy: a feasibility study

Pencil beam scanned (PBS) proton therapy of lung tumours is hampered by respiratory motion and the motion-induced density changes along the beam path. In this simulation study, we aim to investigate the effectiveness of proton beam tracking for lung tumours both under ideal conditions and in conjunction with a respiratory motion model guided by real-time ultrasound imaging of the liver. Multiple-breathing-cycle 4DMRIs of the thorax and abdominal 2D ultrasound images were acquired simultaneously for five volunteers. Deformation vector fields extracted from the 4DMRI, referred to as ground truth motion, were used to generate 4DCT(MRI) data sets of two lung cancer patients, resulting in 10 data sets with variable motion patterns. Given the 4DCT(MRI) and the corresponding ultrasound images as surrogate data, a patient-specific motion model was built. The model consists of an autoregressive model and Gaussian process regression for the temporal and spatial prediction, respectively. Two-field PBS plans were optimised on the reference CTs, and 4D dose calculations (4DDC) were used to simulate dose delivery for (a) unmitigated motion, (b) ideal 2D and 3D tracking (both beam adaption and 4DDC based on ground truth motion), and (c) realistic 2D and 3D tracking (beam adaption based on motion predictions, 4DDC on ground truth motion). Model-guided tracking retrieved clinically acceptable target dose homogeneity, as seen in a substantial reduction of the D5%–D95% compared to the non-mitigated simulation. Tracking in 2D and 3D resulted in a similar improvement of the dose homogeneity, as did ideal and realistic tracking simulations. In some cases, however, the tracked deliveries resulted in a shift towards higher or lower dose levels, leading to unacceptable target over- or under-coverage. The presented motion modelling framework was shown to be an accurate motion prediction tool for the use in proton beam tracking. Tracking alone, however, may not always effectively mitigate motion effects, making it necessary to combine it with other techniques such as rescanning.

Fully Automatic Sliding Motion Compensated and Simultaneous 4D-CBCT via Bilateral Filtering.

PurposeTo incorporate the bilateral filtering into the Deformable Vector Field (DVF) based 4D-CBCT reconstruction for realizing a fully automatic sliding motion compensated 4D-CBCT.Materials and MethodsInitially, a motion compensated simultaneous algebraic reconstruction technique (mSART) is used to generate a high quality reference phase (e.g. 0% phase) by using all phase projections together with the initial 4D-DVFs. The initial 4D-DVF were generated via Demons registration between 0% phase and each other phase image. The 4D-DVF will then kept updating by matching the forward projection of the deformed high quality 0% phase with the measured projection of the target phase. The loss function during this optimization contains an projection intensity difference matching criterion plus a DVF smoothing constrain term. We introduce a bilateral filtering kernel into the DVF constrain term to estimate the sliding motion automatically. The bilateral filtering kernel contains three sub-kernels: 1) an spatial domain Guassian kernel; 2) an image intensity domain Guassian kernel; and 3) a DVF domain Guassian kernel. By choosing suitable kernel variances, the sliding motion can be extracted. A non-linear conjugate gradient optimizer was used. We validated the algorithm on a non-uniform rotational B-spline based cardiac-torso (NCAT) phantom and four anonymous patient data. For quantification, we used: 1) the Root-Mean-Square-Error (RMSE) together with the Maximum-Error (MaxE); 2) the Dice coefficient of the extracted lung contour from the final reconstructed images and 3) the relative reconstruction error (RE) to evaluate the algorithm's performance.ResultsFor NCAT phantom, the motion trajectory's RMSE/MaxE are 0.796/1.02 mm for bilateral filtering reconstruction; and 2.704/4.08 mm for original reconstruction. For patient pilot study, the 4D-Dice coefficient obtained with bilateral filtering are consistently higher than that without bilateral filtering. Meantime several image content such as the rib position, the heart edge definition, the fibrous structures all has been better corrected with bilateral filtering.ConclusionWe developed a bilateral filtering based fully automatic sliding motion compensated 4D-CBCT scheme. Both digital phantom and initial patient pilot studies confirmed the improved motion estimation and image reconstruction ability. It can be used as a 4D-CBCT image guidance tool for lung SBRT treatment.

Early Prediction of Acute Esophagitis for Adaptive Radiation Therapy

Acute esophagitis (AE) is a common dose-limiting toxicity in radiation therapy of locally advanced non-small cell lung cancer (LA-NSCLC). We developed an early AE prediction model from weekly accumulated esophagus dose and its associated local volumetric change. Fifty-one patients with LA-NSCLC underwent treatment with intensity modulated radiation therapy to 60 Gy in 2-Gy fractions with concurrent chemotherapy and weekly cone beam computed tomography (CBCT). Twenty-eight patients (55%) developed grade ≥2 AE (≥AE2) at a median of 4 weeks after the start of radiation therapy. For early ≥AE2 prediction, the esophagus on CBCT of the first 2 weeks was deformably registered to the planning computed tomography images, and weekly esophagus dose was accumulated. Week 1-to-week 2 (w1→w2) esophagus volume changes including maximum esophagus expansion (MEex%) and volumes with ≥x% local expansions (VEx%; x = 5, 10, 15) were calculated from the Jacobian map of deformation vector field gradients. Logistic regression model with 5-fold cross-validation was built using combinations of the accumulated mean esophagus doses (MED) and the esophagus change parameters with the lowest P value in univariate analysis. The model was validated on an additional 18 and 11 patients with weekly CBCT and magnetic resonance imaging (MRI), respectively, and compared with models using only planned mean dose (MEDPlan). Performance was assessed using area under the curve (AUC) and Hosmer-Lemeshow test (PHL). Univariately, w1→w2 VE10% (P = .004), VE5% (P = .01) and MEex% (P = .02) significantly predicted ≥AE2. A model combining MEDW2 and w1→w2 VE10% had the best performance (AUC = 0.80; PHL = 0.43), whereas the MEDPlan model had a lower accuracy (AUC = 0.67; PHL = 0.26). The combined model also showed high accuracy in the CBCT (AUC = 0.78) and MRI validations (AUC = 0.75). A CBCT-based, cross-validated, and internally validated model on MRI with a combination of accumulated esophagus dose and local volume change from the first 2 weeks of chemotherapy significantly improved AE prediction compared with conventional models using only the planned dose. This model could inform plan adaptation early to lower the risk of esophagitis.

Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer

Medical image registration and segmentation are two of the most frequent tasks in medical image analysis. As these tasks are complementary and correlated, it would be beneficial to apply them simultaneously in a joint manner. In this paper, we formulate registration and segmentation as a joint problem via a Multi-Task Learning (MTL) setting, allowing these tasks to leverage their strengths and mitigate their weaknesses through the sharing of beneficial information. We propose to merge these tasks not only on the loss level, but on the architectural level as well. We studied this approach in the context of adaptive image-guided radiotherapy for prostate cancer, where planning and follow-up CT images as well as their corresponding contours are available for training. At testing time the contours of the follow-up scans are not available, which is a common scenario in adaptive radiotherapy. The study involves two datasets from different manufacturers and institutes. The first dataset was divided into training (12 patients) and validation (6 patients), and was used to optimize and validate the methodology, while the second dataset (14 patients) was used as an independent test set. We carried out an extensive quantitative comparison between the quality of the automatically generated contours from different network architectures as well as loss weighting methods. Moreover, we evaluated the quality of the generated deformation vector field (DVF). We show that MTL algorithms outperform their Single-Task Learning (STL) counterparts and achieve better generalization on the independent test set. The best algorithm achieved a mean surface distance of 1.06 ± 0.3 mm, 1.27 ± 0.4 mm, 0.91 ± 0.4 mm, and 1.76 ± 0.8 mm on the validation set for the prostate, seminal vesicles, bladder, and rectum, respectively. The high accuracy of the proposed method combined with the fast inference speed, makes it a promising method for automatic re-contouring of follow-up scans for adaptive radiotherapy, potentially reducing treatment related complications and therefore improving patients quality-of-life after treatment. The source code is available at https://github.com/moelmahdy/JRS-MTL .