Evaluation Of Image Registration Research Articles

Uncertainty estimation and evaluation of deformation image registration based convolutional neural networks

Objective. Fast and accurate deformable image registration (DIR), including DIR uncertainty estimation, is essential for safe and reliable clinical deployment. While recent deep learning models have shown promise in predicting DIR with its uncertainty, challenges persist in proper uncertainty evaluation and hyperparameter optimization for these methods. This work aims to develop and evaluate a model that can perform fast DIR and predict its uncertainty in seconds. Approach. This study introduces a novel probabilistic multi-resolution image registration model utilizing convolutional neural networks to estimate a multivariate normal distributed dense displacement field (DDF) in a multimodal image registration problem. To assess the quality of the DDF distribution predicted by the model, we propose a new metric based on the Kullback–Leibler divergence. The performance of our approach was evaluated against three other DIR algorithms (VoxelMorph, Monte Carlo dropout, and Monte Carlo B-spline) capable of predicting uncertainty. The evaluation of the models included not only the quality of the deformation but also the reliability of the estimated uncertainty. Our application investigated the registration of a treatment planning computed tomography (CT) to follow-up cone beam CT for daily adaptive radiotherapy. Main results. The hyperparameter tuning of the models showed a trade-off between the estimated uncertainty’s reliability and the deformation’s accuracy. In the optimal trade-off, our model excelled in contour propagation and uncertainty estimation (p <0.05) compared to existing uncertainty estimation models. We obtained an average dice similarity coefficient of 0.89 and a KL-divergence of 0.15. Significance. By addressing challenges in DIR uncertainty estimation and evaluation, our work showed that both the DIR and its uncertainty can be reliably predicted, paving the way for safe deployment in a clinical environment.

MRI and CT Fusion in Stereotactic Electroencephalography (SEEG).

Epilepsy is a neurological disorder characterized by spontaneous recurrent seizures. While 20% to 30% of epilepsy cases are untreatable with Anti-Epileptic Drugs, some of these cases can be addressed through surgical intervention. The success of such interventions greatly depends on accurately locating the epileptogenic tissue, a task achieved using diagnostic techniques like Stereotactic Electroencephalography (SEEG). SEEG utilizes multi-modal fusion to aid in electrode localization, using pre-surgical resonance and post-surgical computer tomography images as inputs. To ensure the absence of artifacts or misregistrations in the resultant images, a fusion method that accounts for electrode presence is required. We proposed an image fusion method in SEEG that incorporates electrode segmentation from computed tomography as a sampling mask during registration to address the fusion problem in SEEG. The method was validated using eight image pairs from the Retrospective Image Registration Evaluation Project (RIRE). After establishing a reference registration for the MRI and identifying eight points, we assessed the method's efficacy by comparing the Euclidean distances between these reference points and those derived using registration with a sampling mask. The results showed that the proposed method yielded a similar average error to the registration without a sampling mask, but reduced the dispersion of the error, with a standard deviation of 0.86 when a mask was used and 5.25 when no mask was used.

A super-voxel-based method for generating surrogate lung ventilation images from CT.

Purpose: This study aimed to develop and evaluate , a super-voxel-based method for surrogate computed tomography ventilation imaging (CTVI). Methods and Materials: The study used four-dimensional CT (4DCT) and single-photon emission computed tomography (SPECT) images and corresponding lung masks from 21 patients with lung cancer obtained from the Ventilation And Medical Pulmonary Image Registration Evaluation dataset. The lung volume of the exhale CT for each patient was segmented into hundreds of super-voxels using the Simple Linear Iterative Clustering (SLIC) method. These super-voxel segments were applied to the CT and SPECT images to calculate the mean density values (D mean) and mean ventilation values (Vent mean), respectively. The final CT-derived ventilation images were generated by interpolation from the D mean values to yield . For the performance evaluation, the voxel- and region-wise differences between and SPECT were compared using Spearman's correlation and the Dice similarity coefficient index. Additionally, images were generated using two deformable image registration (DIR)-based methods, and , and compared with the SPECT images. Results: The correlation between the D mean and Vent mean of the super-voxel was 0.59 ± 0.09, representing a moderate-to-high correlation at the super-voxel level. In the voxel-wise evaluation, the method achieved a stronger average correlation (0.62 ± 0.10) with SPECT, which was significantly better than the correlations achieved with the (0.33 ± 0.14, p < 0.05) and (0.23 ± 0.11, p < 0.05) methods. For the region-wise evaluation, the Dice similarity coefficient of the high functional region for (0.63 ± 0.07) was significantly higher than the corresponding values for the (0.43 ± 0.08, p < 0.05) and (0.42 ± 0.05, p < 0.05) methods. Conclusion: The strong correlation between and SPECT demonstrates the potential usefulness of this novel method of ventilation estimation for surrogate ventilation imaging.

Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning.

Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing approaches. The Learn2Reg challenge addresses these limitations by providing a multi-task medical image registration data set for comprehensive characterisation of deformable registration algorithms. A continuous evaluation will be possible at https:// learn2reg.grand-challenge.org. Learn2Reg covers a wide range of anatomies (brain, abdomen, and thorax), modalities (ultrasound, CT, MR), availability of annotations, as well as intra- and inter-patient registration evaluation. We established an easily accessible framework for training and validation of 3D registration methods, which enabled the compilation of results of over 65 individual method submissions from more than 20 unique teams. We used a complementary set of metrics, including robustness, accuracy, plausibility, and runtime, enabling unique insight into the current state-of-the-art of medical image registration. This paper describes datasets, tasks, evaluation methods and results of the challenge, as well as results of further analysis of transferability to new datasets, the importance of label supervision, and resulting bias. While no single approach worked best across all tasks, many methodological aspects could be identified that push the performance of medical image registration to new state-of-the-art performance. Furthermore, we demystified the common belief that conventional registration methods have to be much slower than deep-learning-based methods.

A distinctive ensemble deep learning model for brain tumor MRI image classification

Brain tumor detection is challenging for radiologists. The early detection of brain tumors is critical, and automated techniques are necessary to achieve this goal. In this study, an automated method is proposed to distinguish between malignant and non-cancerous brain Magnetic Resonance Images (MRI). An ensemble technique was proposed that includes two Deep Learning (DL) models, a 6-Convolutional Neural Network (6-CNN) model with an efficient end-to-end, and a pre-trained Residual Network50 (ResNet50) model. The MRI image classification was experimented in two directions: one using the average probabilities of the ensemble model, and the other considering the optimal weights of the ensemble model for a Support Vector Machine (SVM) classifier with different kernels. Two datasets, Harvard and Retrospective Image Registration Evaluation (RIDER), were used to evaluate the performance of the proposed model. The 6-CNN-ResNet-SVM model achieved the highest accuracy of 97.32% for 10-fold validation, with the remaining performance metrics being an Area Under the Curve (AUC) of 0.98%, sensitivity of 93.62%, specificity of 98%, False Negative Rate (FNR) of 0.06, and False Positive Rate (FPR) of 0.00 for the linear kernel. The proposed model can identify tumors more accurately and quickly than existing approaches.

Geometric Analysis and Dosimetric Evaluation of Various Deformable Image Registration (DIR) Algorithms for Abdominal Images with 3D Biomechanically Modelled Deformation

<h3>Purpose/Objective(s)</h3> The accuracy of DIR algorithms available on commercial radiotherapy planning platforms were examined and their clinical applicability geometrically and dosimetrically for abdominal sites were evaluated using biomechanically modelled deformation. <h3>Materials/Methods</h3> 5 pancreatic patients' planning CT were used in this study. Following the guideline of AAPM TG-132, a patient-specific QA workflow was developed and implemented to evaluate DIR for abdomen using digital phantoms, created with a commercial virtual phantom software (VPS). First, the planning CT images were deformed in superior-inferior direction to simulation respiratory motion in lung and abdomen, using the biomechanical model embedded in VPS. In addition, 5 mm motion in both anterior-posterior and lateral directions were added to region of interest (ROI), including stomach and duodenum, and small bowel. The added ROI deformations are consistent with the abdomen data observed in daily CBCT. The resulting 3D deformation vector field (DVF) and the corresponding deformed CT was then obtained, serving as ground truth. Second, the original planning CT and the deformed CT were imported to two commercial DIR systems (DS1 and DS2), to perform DIR with intensity-based algorithms, and derive DVFs. Quantitative differences between ground truth DVF and DS1 or DS2's DVFs, such as mean, max, and the distribution of DIR errors were tabulated. Then the original plan was re-calculated on the deformed CT, which was generated based on DS1 or DS2's DVF. The dose errors were then calculated on a pixel-to-pixel basis. The relationship between the DIR errors and the dose error ranges (25%-75% on the dose error distribution histogram) were investigated. As a pilot study, comprehensive reports were generated to show the DIR performances as patient specific QA records. <h3>Results</h3> The displacements on the ground truth DVF range from 0 to 37 mm. Geometric analysis showed DIR errors as tabulated below. Dosimetrically, as the DIR error increases, the dose error range also increase, for both DS1 and DS2. Dosimetric evaluation of the dose errors for ROIs were tabulated as below. <h3>Conclusion</h3> Preliminary evaluation of geometric and dosimetric accuracy of DIR on DS1 and DS2 were performed, with a realistic biomechanical model derived DVF serving as ground truth. A patient specific DIR QA workflow was developed for a pancreatic patient cohort, and it has potential to be implemented as a clinical workflow.

Evaluation of image registration for measuring deformation fields in soft tissue mechanics

AbstractHigh‐fidelity biomechanical models usually involve the mechanical characterisation of biological tissues using experimental methods based on optical measurements. In most experiments, strains are evaluated based on displacements of a few markers and represents an average within the region of interest (ROI). Full‐field measurements may improve description of non‐homogeneous materials such as soft tissues. The approach based on non‐rigid image registration is proposed and compared with standard digital image correlation (DIC) on a set of samples, including (i) complex heterogeneous deformations with sub‐pixel displacement, (ii) a typical uniaxial tension test of aorta, and (iii) an indentation test on skin. The possibility to extend the ROI to the whole sample and the exploitation of a natural tissue pattern represents the main assets of the proposed method whereas the results show similar accuracy as standard DIC when analysing sub‐pixel deformations. Therefore, displacement and strain fields measurement based on image registration is very promising to characterise heterogeneous specimens with irregular shapes and/or small dimensions, which are typical features of soft biological tissues.

Noisy multimodal brain image registration using markov random field model

In this paper, a new scheme is proposed to register brain Magnetic Resonance (MR) and Computed Tomography (CT) images under a noisy environment. The scheme is developed based on the notion of mutual information and this is extended to the feature space-based registration. The 2D joint histogram of the noisy MR and CT images has been computed and this is assumed to be the degraded version of the true 2D joint histogram that corresponds to the CT and MR images. The true 2D joint histogram is modeled as Markov Random Field (MRF) model and is estimated from the degraded one by formulating the problem in Maximum A Posterior (MAP) estimation framework. The MAP estimates are obtained by the combination of Simulated Annealing (SA) and Iterated Conditioned Mode (ICM) algorithms. The Mutual Information (MI) at different values of the parameter vector θ are computed from the estimated joint histogram and is maximized to determine the optimal registration parameter. The MI curve is modeled as multivariate Gaussian distribution and the registration parameter vector is obtained with a confidence interval. The scheme has been tested with the slices of patients from the Retrospective Image Registration Evaluation (RIRE) database. The efficacy of the scheme on MR and CT volumes is demonstrated by rendering the registered slices. The proposed algorithm is successfully tested with different degrees of Gaussian, Rician and Rayleigh noise conditions. Performance measures such as parameter estimation errors, error statistics and landmark based validation are used to demonstrate the efficacy of the proposed algorithm. The scheme is found to exhibit improved performance, especially at high noisy conditions as compared to the existing schemes.

Variability in commercially available deformable image registration: A multi‐institution analysis using virtual head and neck phantoms

PurposeThe purpose of this study was to evaluate the performance of three common deformable image registration (DIR) packages across algorithms and institutions.Methods and MaterialsThe Deformable Image Registration Evaluation Project (DIREP) provides ten virtual phantoms derived from computed tomography (CT) datasets of head‐and‐neck cancer patients over a single treatment course. Using the DIREP phantoms, DIR results from 35 institutions were submitted using either Velocity, MIM, or Eclipse. Submitted deformation vector fields (DVFs) were compared to ground‐truth DVFs to calculate target registration error (TRE) for six regions of interest (ROIs). Statistical analysis was performed to determine the variability between each DIR software package and the variability of users within each algorithm.ResultsOverall mean TRE was 2.04 ± 0.35 mm for Velocity, 1.10 ± 0.29 mm for MIM, and 2.35 ± 0.15 mm for Eclipse. The MIM mean TRE was significantly different than both Velocity and Eclipse for all ROIs. Velocity and Eclipse mean TREs were not significantly different except for when evaluating the registration of the cord or mandible. Significant differences between institutions were found for the MIM and Velocity platforms. However, these differences could be explained by variations in Velocity DIR parameters and MIM software versions.ConclusionsAverage TRE was shown to be <3 mm for all three software platforms. However, maximum errors could be larger than 2 cm indicating that care should be exercised when using DIR. While MIM performed statistically better than the other packages, all evaluated algorithms had an average TRE better than the largest voxel dimension. For the phantoms studied here, significant differences between algorithm users were minimal suggesting that the algorithm used may have more impact on DIR accuracy than the particular registration technique employed. A significant difference in TRE was discovered between MIM versions showing that DIR QA should be performed after software upgrades as recommended by TG‐132.

A multivariable study of deformable image registration evaluation metrics in 4DCT of thoracic cancer patients

Deformable image registration (DIR) accuracy is often validated using manually identified landmarks or known deformations generated using digital or physical phantoms. In daily practice, the application of these approaches is limited since they are time-consuming or require additional equipment. An alternative is the use of metrics automatically derived from the registrations, but their interpretation is not straightforward. In this work we aim to determine the suitability of DIR-derived metrics to validate the accuracy of 4 commonly used DIR algorithms. First, we investigated the DIR accuracy using a landmark-based metric (target registration error (TRE)) and a digital phantom-based metric (known deformation recovery error (KDE)). 4DCT scans of 16 thoracic cancer patients along with corresponding pairwise anatomical landmarks (AL) locations were collected from two public databases. Digital phantoms with known deformations were generated by each DIR algorithm to test all other algorithms and compute KDE. TRE and KDE were evaluated at AL. KDE was additionally quantified in coordinates randomly sampled (RS) inside the lungs. Second, we investigated the associations of 5 DIR-derived metrics (distance discordance metric (DDM), inverse consistency error (ICE), transitivity (TE), spatial (SS) and temporal smoothness (TS)) with DIR accuracy through uni- and multivariable linear regression models. TRE values were found higher compared to KDE values and these varied depending on the phantom used. The algorithm with the best accuracy achieved average values of TRE = 1.1 mm and KDE ranging from 0.3 to 0.8 mm. DDM was the best predictor of DIR accuracy, with moderate correlations (R 2 < 0.61). Poor correlations were obtained at AL for algorithms with better accuracy, which improved when evaluated at RS. Only slight correlation improvement was obtained with a multivariable analysis (R 2 < 0.64). DDM can be a useful metric to identify inaccuracies for different DIR algorithms without employing landmarks or digital phantoms.

PO-1629: A quantitative evaluation of deformable image registration based on Halcyon MVCBCT images

PO-1629: A quantitative evaluation of deformable image registration based on Halcyon MVCBCT images

A feasibility study to estimate optimal rigid-body registration using combinatorial rigid registration optimization (CORRO).

PurposeClinical image pairs provide the most realistic test data for image registration evaluation. However, the optimal registration is unknown. Using combinatorial rigid registration optimization (CORRO) we demonstrate a method to estimate the optimal alignment for rigid‐registration of clinical image pairs.MethodsExpert selected landmark pairs were selected for each CT/CBCT image pair for six cases representing head and neck, thoracic, and pelvic anatomic regions. Combination subsets of a k number of landmark pairs (k‐combination set) were generated without repeat to form a large set of k‐combination sets (k‐set) for k = 4,8,12. The rigid transformation between the image pairs was calculated for each k‐combination set. The mean and standard deviation of these transformations were used to derive final registration for each k‐set.ResultsThe standard deviation of registration output decreased as the k‐size increased for all cases. The joint entropy evaluated for each k‐set of each case was smaller than those from two commercially available registration programs indicating a stronger correlation between the image pair after CORRO was used. A joint histogram plot of all three algorithms showed high correlation between them. As further proof of the efficacy of CORRO the joint entropy of each member of 30 000 k‐combination sets in k = 4 were calculated for one of the thoracic cases. The minimum joint entropy was found to exist at the estimated mean of registration indicating CORRO converges to the optimal rigid‐registration results.ConclusionsWe have developed a methodology called CORRO that allows us to estimate optimal alignment for rigid‐registration of clinical image pairs using a large set landmark point. The results for the rigid‐body registration have been shown to be comparable to results from commercially available algorithms for all six cases. CORRO can serve as an excellent tool that can be used to test and validate rigid registration algorithms.

Computed Tomography/Magnetic Resonance Imaging (CT/MRI) Image Registration and Fusion Assessment for Accurate Glioblastoma Radiotherapy Treatment Planning

Background: In this study, computed tomography/magnetic resonance imaging (CT/MRI) image registration and fusion in the 3D conformal radiotherapy treatment planning of Glioblastoma brain tumor was investigated. Good CT/MRI image registration and fusion made a great impact on dose calculation and treatment planning accuracy. Indeed, the uncertainly associated with the registration and fusion methods must be well verified and communicated. Unfortunately, there is no standard procedure or mathematical formalism to perform this verification due to noise, distortion, and complicated anatomical situations. Objectives: This study aimed at assessing the effective contribution of MRI in Glioma radiotherapy treatment by improving the localization of target volumes and organs at risk (OARs). It is also a question to provide clinicians with some suitable metrics to evaluate the CT/MRI image registration and fusion results. Methods: Quantitative image registration and fusion evaluation were used in this study to compare Eclipse TPS tools and Elastix CT/MRI image registration fusion. Thus, Dice score coefficient (DSC), Jaccard similarity coefficient (JSC), and Hausdorff distance (HD) were found to be suitable metrics for the evaluation and comparison of the image registration and fusion methods of Eclipse TPS and Elastix. Results: The programmed tumor’s volumes (PTV) delineated on CT slices were approximately 1.38 times smaller than those delineated on CT/MRI fused images. Large differences were observed for the edema and the brainstem. It was also found that MRI considerably optimized the dose to be delivered to the optic nerve and brainstem. Conclusions: Image registration and fusion is a fundamental step for suitable and efficient Glioma treatment planning in 3D conformal radiotherapy that ensure accurate dose delivery and unnecessary OAR irradiation. MRI can provide accurate localization of targeted volumes leading to better irradiation control of Glioma tumor.

Flexible numerical simulation framework for dynamic PET-MR data

This paper presents a simulation framework for dynamic PET-MR. The main focus of this framework is to provide motion-resolved MR and PET data and ground truth motion information. This can be used in the optimisation and quantitative evaluation of image registration and in assessing the error propagation due to inaccuracies in motion estimation in complex motion-compensated reconstruction algorithms. Contrast and tracer kinetics can also be simulated and are available as ground truth information. To closely emulate medical examination, input and output of the simulation are files in standardised open-source raw data formats. This enables the use of existing raw data as a template input and ensures seamless integration of the output into existing reconstruction pipelines.The proposed framework was validated in PET-MR and image registration applications. It was used to simulate a FDG-PET-MR scan with cardiac and respiratory motion. Ground truth motion information could be utilised to optimise parameters for PET and synergistic PET-MR image registration. In addition, a free-breathing dynamic contrast enhancement (DCE) abdominal scan of a patient with hepatic lesions was simulated. In order to correct for breathing motion, a motion-corrected image reconstruction scheme was used and a Toft’s model was fit to the DCE data to obtain quantitative DCE-MRI parameters. Utilising the ground truth motion information, the dependency of quantitative DCE-MR images on the accuracy of the motion estimation was evaluated. We demonstrated that respiratory motion had to be available with an average accuracy of at least the spatial resolution of the DCE-MR images in order to ensure an improvement in lesions visualisation and quantification compared to no motion correction.The proposed framework provides a valuable tool with a wide range of scientific PET and MR applications and will be available as part of the open-source project Synergistic Image Reconstruction Framework (SIRF).

A quantitative evaluation of deformable image registration based on MV cone beam CT images: Impact of deformation magnitudes and image modalities

Background and purposeTo evaluate the impact of deformation magnitude and image modality on deformable-image-registration (DIR) accuracy using Halcyon megavoltage cone beam CT images (MVCBCT). Materials and methodsPlanning CT images of an anthropomorphic Head phantom were aligned rigidly with MVCBCT and re-sampled to achieve the same resolution, denoted as pCT. MVCBCT was warped with twenty simulated pre-known virtual deformation fields (Ti, i = 1–20) with increasing deformation magnitudes, yielding warped CBCT (wCBCT). The pCT and MVCBCT were registered to wCBCT respectively (Multi-modality and Uni-modality DIR), generating deformation vector fields Vi and Vi′ (i = 1–20). Vi and Vi′ were compared with Ti respectively to assess the DIR accuracy geometrically. In addition, Vi, Ti, and Vi′ were applied to pCT, generating deformed CT (dCTi), ground-truth CT (Gi) and deformed CT′ (dCTi′) respectively. The Hounsfield Unit (HU) on these virtual CT images were also compared. ResultsThe mean errors of vector displacement increased with the deformation magnitude. For deformation magnitudes between 2.82 mm and 7.71 mm, the errors of uni-modality DIR were 1.16 mm ~ 1.73 mm smaller than that of multi-modality (p = 0.0001, Wilcoxon signed rank test). DIR could reduce the maximum signed and absolute HU deviations from 70.8 HU to 11.4 HU and 208 HU to 46.2 HU respectively. ConclusionsAs deformation magnitude increases, DIR accuracy continues to deteriorate and uni-modality DIR consistently outperformed multi-modality DIR. DIR-based adaptive radiotherapy utilizing the noisy MVCBCT images is only conditionally applicable with caution.

多角度偏振成像仪在轨图像配准及性能评估

多角度偏振成像仪在轨图像配准及性能评估

Automatic large quantity landmark pairs detection in 4DCT lung images.

To automatically and precisely detect a large quantity of landmark pairs between two lung computed tomography (CT) images to support evaluation of deformable image registration (DIR). We expect that the generated landmark pairs will significantly augment the current lung CT benchmark datasets in both quantity and positional accuracy. A large number of landmark pairs were detected within the lung between the end-exhalation (EE) and end-inhalation (EI) phases of the lung four-dimensional computed tomography (4DCT) datasets. Thousands of landmarks were detected by applying the Harris-Stephens corner detection algorithm on the probability maps of the lung vasculature tree. A parametric image registration method (pTVreg) was used to establish initial landmark correspondence by registering the images at EE and EI phases. A multi-stream pseudo-siamese (MSPS) network was then developed to further improve the landmark pair positional accuracy by directly predicting three-dimensional (3D) shifts to optimally align the landmarks in EE to their counterparts in EI. Positional accuracies of the detected landmark pairs were evaluated using both digital phantoms and publicly available landmark pairs. Dense sets of landmark pairs were detected for 10 4DCT lung datasets, with an average of 1886 landmark pairs per case. The mean and standard deviation of target registration error (TRE) were 0.47±0.45mm with 98% of landmark pairs having a TRE smaller than 2mm for 10 digital phantom cases. Tests using 300 manually labeled landmark pairs in 10 lung 4DCT benchmark datasets (DIRLAB) produced TRE results of 0.73±0.53mm with 97% of landmark pairs having a TRE smaller than 2mm. A new method was developed to automatically and precisely detect a large quantity of landmark pairs between lung CT image pairs. The detected landmark pairs could be used as benchmark datasets for more accurate and informative quantitative evaluation of DIR algorithms.

EP-2010 A QA method for evaluation of deformable image registration in head and neck adaptive radiotherapy

EP-2010 A QA method for evaluation of deformable image registration in head and neck adaptive radiotherapy

The VAMPIRE challenge: A multi-institutional validation study of CT ventilation imaging.

CT ventilation imaging (CTVI) is being used to achieve functional avoidance lung cancer radiation therapy in three clinical trials (NCT02528942, NCT02308709, NCT02843568). To address the need for common CTVI validation tools, we have built the Ventilation And Medical Pulmonary Image Registration Evaluation (VAMPIRE) Dataset, and present the results of the first VAMPIRE Challenge to compare relative ventilation distributions between different CTVI algorithms and other established ventilation imaging modalities. The VAMPIRE Dataset includes 50 pairs of 4DCT scans and corresponding clinical or experimental ventilation scans, referred to as reference ventilation images (RefVIs). The dataset includes 25 humans imaged with Galligas 4DPET/CT, 21 humans imaged with DTPA-SPECT, and 4 sheep imaged with Xenon-CT. For the VAMPIRE Challenge, 16 subjects were allocated to a training group (with RefVI provided) and 34 subjects were allocated to a validation group (with RefVI blinded). Seven research groups downloaded the Challenge dataset and uploaded CTVIs based on deformable image registration (DIR) between the 4DCT inhale/exhale phases. Participants used DIR methods broadly classified into B-splines, Free-form, Diffeomorphisms, or Biomechanical modeling, with CT ventilation metrics based on the DIR evaluation of volume change, Hounsfield Unit change, or various hybrid approaches. All CTVIs were evaluated against the corresponding RefVI using the voxel-wise Spearman coefficient , and Dice similarity coefficients evaluated for low function lung ( ) and high function lung ( ). A total of 37 unique combinations of DIR method and CT ventilation metric were either submitted by participants directly or derived from participant-submitted DIR motion fields using the in-house software, VESPIR. The and DSC results reveal a high degree of inter-algorithm and intersubject variability among the validation subjects, with algorithm rankings changing by up to ten positions depending on the choice of evaluation metric. The algorithm with the highest overall cross-modality correlations used a biomechanical model-based DIR with a hybrid ventilation metric, achieving a median (range) of 0.49 (0.27-0.73) for , 0.52 (0.36-0.67) for , and 0.45 (0.28-0.62) for . All other algorithms exhibited at least one negative value, and/or one DSC value less than 0.5. The VAMPIRE Challenge results demonstrate that the cross-modality correlation between CTVIs and the RefVIs varies not only with the choice of CTVI algorithm but also with the choice of RefVI modality, imaging subject, and the evaluation metric used to compare relative ventilation distributions. This variability may arise from the fact that each of the different CTVI algorithms and RefVI modalities provides a distinct physiologic measurement. Ultimately this variability, coupled with the lack of a "gold standard," highlights the ongoing importance of further validation studies before CTVI can be widely translated from academic centers to the clinic. It is hoped that the information gleaned from the VAMPIRE Challenge can help inform future validation efforts.

Technical Note: A method for quality assurance of landmark sets for use in evaluation of deformable image registration accuracy of lung parenchyma.

To develop a quality control method to improve the accuracy of corresponding landmark sets used for deformable image registration (DIR) evaluation in the lung parenchyma. An iterative workflow was developed as a method for quality assurance of landmark sets. Starting with the initial landmark set for a given image pair, a landmark-based deformation was applied to one of the images. A difference image and a color overlay were generated using the deformed image and the other image of the pair. Inspection of these generated images at locations of landmarks allowed for the identification of misplaced landmarks. The observer responsible for creating the initial landmark set was tasked with review and revision of points flagged by the quality assurance procedure. Using the updated landmark sets, the process was repeated until all points were acceptable to the reviewer. Eighteen landmark sets, containing a mean (SD) of 170 (31) landmarks, were created using CT images from non-small cell lung cancer patients exhibiting large geometric changes and atelectasis resolution, making landmark specification challenging. Following the quality assurance procedure, the final landmark sets contained a mean (SD) of 165 (25) landmarks, as points too difficult to match were removed and points were added to regions deficient in landmarks. For landmark sets in which changes were made, maximum and mean differences in landmark positions before and after quality assurance ranged between 8.7-81.5mm and 0.3-9.6mm, respectively. An effective method for improving the accuracy of landmark correspondence was presented. This quality assurance approach enables more accurate evaluation of DIR for lung parenchyma in clinical image pairs in the absence of a ground truth deformation and may be applicable to other feature-rich anatomical sites.