Reproducibility Of Radiomic Features Research Articles

Effects of Interobserver Segmentation Variability and Intensity Discretization on MRI-Based Radiomic Feature Reproducibility of Lipoma and Atypical Lipomatous Tumor

Segmentation and image intensity discretization impact on radiomics workflow. The aim of this study is to investigate the influence of interobserver segmentation variability and intensity discretization methods on the reproducibility of MRI-based radiomic features in lipoma and atypical lipomatous tumor (ALT). Thirty patients with lipoma or ALT were retrospectively included. Three readers independently performed manual contour-focused segmentation on T1-weighted and T2-weighted sequences, including the whole tumor volume. Additionally, a marginal erosion was applied to segmentations to evaluate its influence on feature reproducibility. After image pre-processing, with included intensity discretization employing both fixed bin number and width approaches, 1106 radiomic features were extracted from each sequence. Intraclass correlation coefficient (ICC) 95% confidence interval lower bound ≥ 0.75 defined feature stability. In contour-focused vs. margin shrinkage segmentation, the rates of stable features extracted from T1-weighted and T2-weighted images ranged from 92.68 to 95.21% vs. 90.69 to 95.66% after fixed bin number discretization and from 95.75 to 97.65% vs. 95.39 to 96.47% after fixed bin width discretization, respectively, with no difference between the two segmentation approaches (p ≥ 0.175). Higher stable feature rates and higher feature ICC values were found when implementing discretization with fixed bin width compared to fixed bin number, regardless of the segmentation approach (p < 0.001). In conclusion, MRI radiomic features of lipoma and ALT are reproducible regardless of the segmentation approach and intensity discretization method, although a certain degree of interobserver variability highlights the need for a preliminary reliability analysis in future studies.

Multi-sequence MRI radiomics of colorectal liver metastases: Which features are reproducible across readers?

PurposeTo assess the inter-reader reproducibility of radiomics features on multiple MRI sequences after segmentations of colorectal liver metastases (CRLM). Method30 CRLM (in 23 patients) were manually delineated by three readers on MRI before the start of chemotherapy on the contrast enhanced T1-weighted images (CE-T1W) in the portal venous phase, T2-weighted images (T2W) and b800 diffusion weighted images (DWI). DWI delineations were copied to the ADC-maps. 107 radiomics features were extracted per sequence. The intraclass correlation coefficient (ICC) was calculated per feature. Features were considered reproducible if ICC > 0.9. Results90% of CE-T1W features were reproducible with a median ICC of 0.98 (range 0.76–1.00). 81% of DWI features were robust with median ICC = 0.97 (range 0.38–1.00). The T2W features had a median ICC of 0.96 (range 0.55–0.99) and were reproducible in 80%. ADC showed the lowest number of reproducible features with 58% and median ICC = 0.91 (range 0.38–0.99) When considering the lower bound of the ICC 95% confidence intervals, 58%, 66%, 54% and 29% reached 0.9 for the CE-T1W, DWI, T2W and ADC features, respectively. The feature class with the best reproducibility differed per sequence. ConclusionsThe majority of MRI radiomics features from CE-T1W, T2W, DWI and ADC in colorectal liver metastases were robust for segmentation variability between readers. The CE-T1W yielded slightly better reproducibility results compared to DWI and T2W. The ADC features seem more susceptible to reader differences compared to the other three sequences.

Creating a 3D-printed pelvic phantom to explore the impact of Magnetic Resonance (MR) scanner noise in radiomics analysis, a technical note.

Reproducibility of radiomics features necessitates that scanner noise be considered prior to feature extraction. Phantom research provides the opportunity for such ‘ground truth’ measurements, without the additional complication of patient-related factors. The aim of this technical note was to create a 3D printed Magnetic Resonance Imaging (MRI)-compatible pelvic phantom that can be used for subsequent analysis of the impact of scanner noise on the reproducibility of radiomics features.A 3D printed phantom of a male pelvis was created using fused deposition modelling technology. It was 3D printed using the high density MRI-compatible acrylonitrile butadiene styrene (ABS). The ‘negative’ mould created was then filled with silicone, and the prostate gland and femoral heads were also simulated.We successfully created an MRI-compatible 3D printed pelvic phantom, with a test scan. The phantom will subsequently be utilised to determine the impact of MRI scanner noise on radiomics feature reproducibility.

The reproducibility and predictivity of radiomic features extracted from dynamic contrast-enhanced computed tomography of hepatocellular carcinoma.

To assess the reproducibility of radiomic features (RFs) extracted from dynamic contrast-enhanced computed tomography (DCE-CT) scans of patients diagnosed with hepatocellular carcinoma (HCC) with regards to inter-observer variability and acquisition timing after contrast injection. The predictive ability of reproducible RFs for differentiating between the degrees of HCC differentiation is also investigated. We analyzed a set of DCE-CT scans of 39 patients diagnosed with HCC. Two radiologists independently segmented the scans, and RFs were extracted from each sequence of the DCE-CT scans. The same lesion was segmented across the DCE-CT sequences of each patient's scan. From each lesion, 127 commonly used RFs were extracted. The reproducibility of RFs was assessed with regard to (i) inter-observer variability, by evaluating the reproducibility of RFs between the two radiologists; and (ii) timing of acquisition following contrast injection (inter- and intra-imaging phase). The reproducibility of RFs was assessed using the concordance correlation coefficient (CCC), with a cut-off value of 0.90. Reproducible RFs were used for building XGBoost classification models for the differentiation of HCC differentiation. Inter-observer analyses across the different contrast-enhancement phases showed that the number of reproducible RFs was 29 (22.8%), 52 (40.9%), and 36 (28.3%) for the non-contrast enhanced, late arterial, and portal venous phases, respectively. Intra- and inter-sequence analyses revealed that the number of reproducible RFs ranged between 1 (0.8%) and 47 (37%), inversely related with time interval between the sequences. XGBoost algorithms built using reproducible RFs in each phase were found to be high predictive ability of the degree of HCC tumor differentiation. The reproducibility of many RFs was significantly impacted by inter-observer variability, and a larger number of RFs were impacted by the difference in the time of acquisition after contrast injection. Our findings highlight the need for quality assessment to ensure that scans are analyzed in the same physiologic imaging phase in quantitative imaging studies, or that phase-wide reproducible RFs are selected. Overall, the study emphasizes the importance of reproducibility and quality control when using RFs as biomarkers for clinical applications.

Magnetic resonance imaging preprocessing and radiomic features for classification of autosomal dominant polycystic kidney disease genotype.

Our study aims to investigate the impact of preprocessing on magnetic resonance imaging (MRI) radiomic features extracted from the noncystic kidney parenchyma of patients with autosomal dominant polycystic kidney disease (ADPKD) in the task of classifying PKD1 versus PKD2 genotypes, which differ with regard to cyst burden and disease outcome. The effect of preprocessing on radiomic features was investigated using a single T2-weighted fat saturated (T2W-FS) MRI scan from PKD1 and PKD2 subjects (29 kidneys in total) from the Consortium for Radiologic Imaging Studies of Polycystic Kidney Disease study. Radiomic feature reproducibility using the intraclass correlation coefficient (ICC) was computed across MRI normalizations (-score, reference-tissue, and original image), gray-level discretization, and upsampling and downsampling pixel schemes. A second dataset for genotype classification from 136 subjects T2W-FS MRI images previously enrolled in the HALT Progression of Polycystic Kidney Disease study was matched for age, gender, and Mayo imaging classification class. Genotype classification was performed using a logistic regression classifier and radiomic features extracted from (1)the noncystic kidney parenchyma and (2)the entire kidney. The area under the receiver operating characteristic curve (AUC) was used to evaluate the classification performance across preprocessing methods. Radiomic features extracted from the noncystic kidney parenchyma were sensitive to preprocessing parameters, with varying reproducibility depending on the parameter. The percentage of features with good-to-excellent ICC scores ranged from 14% to 58%. AUC values ranged between 0.47 to 0.68 and 0.56 to 0.73 for the noncystic kidney parenchyma and entire kidney, respectively. Reproducibility of radiomic features extracted from the noncystic kidney parenchyma was dependent on the preprocessing parameters used, and the effect on genotype classification was sensitive to preprocessing parameters. The results suggest that texture features may be indicative of genotype expression in ADPKD.

Photon-counting detector CT-based virtual monoenergetic reconstructions: repeatability and reproducibility of radiomics features of an organic phantom and human myocardium

BackgroundPhoton-counting detector computed tomography (PCD-CT) may influence imaging characteristics for various clinical conditions due to higher signal and contrast-to-noise ratio in virtual monoenergetic images (VMI). Radiomics analysis relies on quantification of image characteristics. We evaluated the impact of different VMI reconstructions on radiomic features in in vitro and in vivo PCD-CT datasets.MethodsAn organic phantom consisting of twelve samples (four oranges, four onions, and four apples) was scanned five times. Twenty-three patients who had undergone coronary computed tomography angiography on a first generation PCD-CT system with the same image acquisitions were analyzed. VMIs were reconstructed at 6 keV levels (40, 55, 70, 90, 120, and 190 keV). The phantoms and the patients’ left ventricular myocardium (LVM) were segmented for all reconstructions. Ninety-three original radiomic features were extracted. Repeatability and reproducibility were evaluated through intraclass correlations coefficient (ICC) and post hoc paired samples ANOVA t test.ResultsThere was excellent repeatability for radiomic features in phantom scans (all ICC = 1.00). Among all VMIs, 36/93 radiomic features (38.7%) in apples, 28/93 (30.1%) in oranges, and 33/93 (35.5%) in onions were not significantly different. For LVM, the percentage of stable features was high between VMIs ≥ 90 keV (90 versus 120 keV, 77.4%; 90 versus 190 keV, 83.9%; 120 versus 190 keV, 89.3%), while comparison to lower VMI levels led to fewer reproducible features (40 versus 55 keV, 8.6%).ConclusionsVMI levels influence the stability of radiomic features in an organic phantom and patients’ LVM; stability decreases considerably below 90 keV.Relevance statementSpectral reconstructions significantly influence radiomic features in vitro and in vivo, necessitating standardization and careful attention to these reconstruction parameters before clinical implementation.Key points• Radiomic features have an excellent repeatability within the same PCD-CT acquisition and reconstruction.• Differences in VMI lead to decreased reproducibility for radiomic features.• VMI ≥ 90 keV increased the reproducibility of the radiomic features.Graphical

Reproducibility and location-stability of radiomic features derived from cone-beam computed tomography: a phantom study.

This study aims to determine the reproducibility and location-stability of cone-beam computed tomography (CBCT) radiomic features. Centrifugal tubes with six concentrations of K2HPO4 solutions (50, 100, 200, 400, 600, and 800 mg ml-1) were imaged within a customized phantom. For each concentration, images were captured twice as test and retest sets. Totally, 69 radiomic features were extracted by LIFEx. The reproducibility was assessed between the test and retest sets. We used the concordance correlation coefficient (CCC) to screen qualified features and then compared the differences in the numbers of them under 24 series (four locations groups * six concentrations). The location-stability was assessed using the Kruskal-Wallis test under different concentration sets; likewise, the numbers of qualified features under six test sets were analyzed. There were 20 and 23 qualified features in the reproducibility and location-stability experiments, respectively. In the reproducibility experiment, the performance of the peripheral groups and high-concentration sets was significantly better than the center groups and low-concentration sets. The effect of concentration on the location-stability of features was not monotonic, and the number of qualified features in the low-concentration sets was greater than that in the high-concentration sets. No features were qualified in both experiments. The density and location of the target object can affect the number of reproducible radiomic features, and its density can also affect the number of location-stable radiomic features. The problem of feature reliability should be treated cautiously in radiomic research on CBCT.

Physics-Informed Discretization for Reproducible and Robust Radiomic Feature Extraction Using Quantitative MRI.

Given the limited repeatability and reproducibility of radiomic features derived from weighted magnetic resonance imaging (MRI), there may be significant advantages to using radiomics in conjunction with quantitative MRI. This study introduces a novel physics-informed discretization (PID) method for reproducible radiomic feature extraction and evaluates its performance using quantitative MRI sequences including magnetic resonance fingerprinting (MRF) and apparent diffusion coefficient (ADC) mapping. A multiscanner, scan-rescan dataset comprising whole-brain 3D quantitative (MRF T1, MRF T2, and ADC) and weighted MRI (T1w MPRAGE, T2w SPACE, and T2w FLAIR) from 5 healthy subjects was prospectively acquired. Subjects underwent 2 repeated acquisitions on 3 distinct 3 T scanners each, for a total of 6 scans per subject (30 total scans). First-order statistical (n = 23) and second-order texture (n = 74) radiomic features were extracted from 56 brain tissue regions of interest using the proposed PID method (for quantitative MRI) and conventional fixed bin number (FBN) discretization (for quantitative MRI and weighted MRI). Interscanner radiomic feature reproducibility was measured using the intraclass correlation coefficient (ICC), and the effect of image sequence (eg, MRF T1 vs T1w MPRAGE), as well as image discretization method (ie, PID vs FBN), on radiomic feature reproducibility was assessed using repeated measures analysis of variance. The robustness of PID and FBN discretization to segmentation error was evaluated by simulating segmentation differences in brainstem regions of interest. Radiomic features with ICCs greater than 0.75 following simulated segmentation were determined to be robust to segmentation. First-order features demonstrated higher reproducibility in quantitative MRI than weighted MRI sequences, with 30% (n = 7/23) features being more reproducible in MRF T1 and MRF T2 than weighted MRI. Gray level co-occurrence matrix (GLCM) texture features extracted from MRF T1 and MRF T2 were significantly more reproducible using PID compared with FBN discretization; for all quantitative MRI sequences, PID yielded the highest number of texture features with excellent reproducibility (ICC > 0.9). Comparing texture reproducibility of quantitative and weighted MRI, a greater proportion of MRF T1 (n = 225/370, 61%) and MRF T2 (n = 150/370, 41%) texture features had excellent reproducibility (ICC > 0.9) compared with T1w MPRAGE (n = 148/370, 40%), ADC (n = 115/370, 32%), T2w SPACE (n = 98/370, 27%), and FLAIR (n = 102/370, 28%). Physics-informed discretization was also more robust than FBN discretization to segmentation error, as 46% (n = 103/222, 46%) of texture features extracted from quantitative MRI using PID were robust to simulated 6 mm segmentation shift compared with 19% (n = 42/222, 19%) of weighted MRI texture features extracted using FBN discretization. The proposed PID method yields radiomic features extracted from quantitative MRI sequences that are more reproducible and robust than radiomic features extracted from weighted MRI using conventional (FBN) discretization approaches. Quantitative MRI sequences also demonstrated greater scan-rescan robustness and first-order feature reproducibility than weighted MRI.

Achieving imaging and computational reproducibility on multiparametric MRI radiomics features in brain tumor diagnosis: phantom and clinical validation.

The Image Biomarker Standardization Initiative has helped improve the computational reproducibility of MRI radiomics features. Nonetheless, the MRI sequences and features with high imaging reproducibility are yet to be established. To determine reproducible multiparametric MRI radiomics features across test-retest, multi-scanner, and computational reproducibility comparisons, and to evaluate their clinical value in brain tumor diagnosis. To assess reproducibility, T1-weighted imaging (T1WI), T2-weighted imaging (T2WI), and diffusion-weighted imaging (DWI) were acquired from three 3-T MRI scanners using standardized phantom, and radiomics features were extracted using two computational algorithms. Reproducible radiomics features were selected when the concordance correlation coefficient value above 0.9 across multiple sessions, scanners, and computational algorithms. Random forest classifiers were trained with reproducible features (n = 117) and validated in a clinical cohort (n = 50) to evaluate whether features with high reproducibility improved the differentiation of glioblastoma from primary central nervous system lymphomas (PCNSLs). Radiomics features from T2WI demonstrated higher repeatability (65-94%) than those from DWI (38-48%) or T1WI (2-92%). Across test-retest, multi-scanner, and computational comparisons, T2WI provided 41 reproducible features, DWI provided six, and T1WI provided two. The performance of the classification model with reproducible features was higher than that using non-reproducible features in both training set (AUC, 0.916 vs. 0.877) and validation set (AUC, 0.957 vs. 0.869). Radiomics features with high reproducibility across multiple sessions, scanners, and computational algorithms were identified, and they showed higher diagnostic performance than non-reproducible radiomics features in the differentiation of glioblastoma from PCNSL. By identifying the radiomics features showing higher multi-machine reproducibility, our results also demonstrated higher radiomics diagnostic performance in the differentiation of glioblastoma from PCNSL, paving the way for further research designs and clinical application in neuro-oncology. • Highly reproducible radiomics features across multiple sessions, scanners, and computational algorithms were identified using phantom and applied to clinical diagnosis. • Radiomics features from T2-weighted imaging were more reproducible than those from T1-weighted and diffusion-weighted imaging. • Radiomics features with good reproducibility had better diagnostic performance for brain tumors than features with poor reproducibility.

Improving radiomics reproducibility using deep learning-based image conversion of CT reconstruction algorithms in hepatocellular carcinoma patients.

CT reconstruction algorithms affect radiomics reproducibility. In this study, we evaluate the effect of deep learning-based image conversion on CT reconstruction algorithms. This study included 78 hepatocellular carcinoma (HCC) patients who underwent four-phase liver CTs comprising non-contrast, late arterial (LAP), portal venous (PVP), and delayed phase (DP), reconstructed using both filtered back projection (FBP) and advanced modeled iterative reconstruction (ADMIRE). PVP images were used to train a convolutional neural network (CNN) model to convert images from FBP to ADMIRE and vice versa. LAP, PVP, and DP images were used for validation and testing. Radiomic features were extracted for each patient with a semi-automatic segmentation tool. We used concordance correlation coefficients (CCCs) to evaluate the radiomics reproducibility for original FBP (oFBP) vs. original ADMIRE (oADMIRE), oFBP vs. converted FBP (cFBP), and oADMIRE vs. converted ADMIRE (cADMIRE). In the test group including 30 patients, the CCC and proportion of reproducible features (CCC ≥ 0.85) for oFBP vs. oADMIRE were 0.65 and 32.9% (524/1595) for LAP, 0.65 and 35.9% (573/1595) for PVP, and 0.69 and 43.8% (699/1595) for DP. For oFBP vs. cFBP, the values increased to 0.92 and 83.9% (1339/1595) for LAP, 0.89 and 71.0% (1133/1595) for PVP, and 0.90 and 79.7% (1271/1595) for DP. Similarly, for oADMIRE vs. cADMIRE, the values increased to 0.87 and 68.1% (1086/1595) for LAP, 0.91 and 82.1% (1309/1595) for PVP, and 0.89 and 76.2% (1216/1595) for DP. CNN-based image conversion between CT reconstruction algorithms improved the radiomics reproducibility of HCCs. This study demonstrates that using a CNN-based image conversion technique significantly improves the reproducibility of radiomic features in HCCs, highlighting its potential for enhancing radiomics research in HCC patients. Radiomics reproducibility of HCC was improved via CNN-based image conversion between two different CT reconstruction algorithms. This is the first clinical study to demonstrate improvements across a range of radiomic features in HCC patients. This study promotes the reproducibility and generalizability of different CT reconstruction algorithms in radiomics research.

The impact of pre-processing and disease characteristics on reproducibility of T2-weighted MRI radiomics features

PurposeTo evaluate the reproducibility of radiomics features derived via different pre-processing settings from paired T2-weighted imaging (T2WI) prostate lesions acquired within a short interval, to select the setting that yields the highest number of reproducible features, and to evaluate the impact of disease characteristics (i.e., clinical variables) on features reproducibility.Materials and methodsA dataset of 50 patients imaged using T2WI at 2 consecutive examinations was used. The dataset was pre-processed using 48 different settings. A total of 107 radiomics features were extracted from manual delineations of 74 lesions. The inter-scan reproducibility of each feature was measured using the intra-class correlation coefficient (ICC), with ICC values > 0.75 considered good. Statistical differences were assessed using Mann–Whitney U and Kruskal–Wallis tests.ResultsThe pre-processing parameters strongly influenced the reproducibility of radiomics features of T2WI prostate lesions. The setting that yielded the highest number of features (25 features) with high reproducibility was the relative discretization with a fixed bin number of 64, no signal intensity normalization, and outlier filtering by excluding outliers. Disease characteristics did not significantly impact the reproducibility of radiomics features.ConclusionThe reproducibility of T2WI radiomics features was significantly influenced by pre-processing parameters, but not by disease characteristics. The selected pre-processing setting yielded 25 reproducible features.

Erratum: “Evaluation of repeatability and reproducibility of radiomic features produced by the fan‐beam kV‐CT on a novel ring gantry‐based PET/CT linear accelerator” https://10.1002/mp.16399

Erratum: “Evaluation of repeatability and reproducibility of radiomic features produced by the fan‐beam kV‐CT on a novel ring gantry‐based PET/CT linear accelerator” https://10.1002/mp.16399

CNN-based automatic segmentations and radiomics feature reliability on contrast-enhanced ultrasound images for renal tumors.

To investigate the feasibility and efficiency of automatic segmentation of contrast-enhanced ultrasound (CEUS) images in renal tumors by convolutional neural network (CNN) based models and their further application in radiomic analysis. From 94 pathologically confirmed renal tumor cases, 3355 CEUS images were extracted and randomly divided into training set (3020 images) and test set (335 images). According to the histological subtypes of renal cell carcinoma, the test set was further split into clear cell renal cell carcinoma (ccRCC) set (225 images), renal angiomyolipoma (AML) set (77 images) and set of other subtypes (33 images). Manual segmentation was the gold standard and serves as ground truth. Seven CNN-based models including DeepLabV3+, UNet, UNet++, UNet3+, SegNet, MultilResUNet and Attention UNet were used for automatic segmentation. Python 3.7.0 and Pyradiomics package 3.0.1 were used for radiomic feature extraction. Performance of all approaches was evaluated by the metrics of mean intersection over union (mIOU), dice similarity coefficient (DSC), precision, and recall. Reliability and reproducibility of radiomics features were evaluated by the Pearson coefficient and the intraclass correlation coefficient (ICC). All seven CNN-based models achieved good performance with the mIOU, DSC, precision and recall ranging between 81.97%-93.04%, 78.67%-92.70%, 93.92%-97.56%, and 85.29%-95.17%, respectively. The average Pearson coefficients ranged from 0.81 to 0.95, and the average ICCs ranged from 0.77 to 0.92. The UNet++ model showed the best performance with the mIOU, DSC, precision and recall of 93.04%, 92.70%, 97.43% and 95.17%, respectively. For ccRCC, AML and other subtypes, the reliability and reproducibility of radiomic analysis derived from automatically segmented CEUS images were excellent, with the average Pearson coefficients of 0.95, 0.96 and 0.96, and the average ICCs for different subtypes were 0.91, 0.93 and 0.94, respectively. This retrospective single-center study showed that the CNN-based models had good performance on automatic segmentation of CEUS images for renal tumors, especially the UNet++ model. The radiomics features extracted from automatically segmented CEUS images were feasible and reliable, and further validation by multi-center research is necessary.

REliability of consensus-based segMentatIoN in raDiomic feature reproducibility (REMIND): A word of caution

ObjectiveTo evaluate the reliability of consensus-based segmentation in terms of reproducibility of radiomic features. MethodsIn this retrospective study, three tumor data sets were investigated: breast cancer (n = 30), renal cell carcinoma (n = 30), and pituitary macroadenoma (n = 30). MRI was utilized for breast and pituitary data sets, while CT was used for renal data set. 12 readers participated in the segmentation process. Consensus segmentation was created by making corrections on a previous region or volume of interest. Four experiments were designed to evaluate the reproducibility of radiomic features. Reliability was assessed with intraclass correlation coefficient (ICC) with two cut-off values: 0.75 and 0.9. ResultsConsidering the lower bound of the 95% confidence interval and the ICC threshold of 0.90, at least 61% of the radiomic features were not reproducible in the inter-consensus analysis. In the susceptibility experiment, at least half (54%) became non-reproducible when the first reader is replaced with a different reader. In the intra-consensus analysis, at least about one-third (32%) were non-reproducible when the same second reader segmented the image over the same first reader two weeks later. Compared to inter-reader analysis based on independent single readers, the inter-consensus analysis did not statistically significantly improve the rates of reproducible features in all data sets and analyses. ConclusionsDespite the positive connotation of the word “consensus”, it is essential to REMIND that consensus-based segmentation has significant reproducibility issues. Therefore, the usage of consensus-based segmentation alone should be avoided unless a reliability analysis is performed, even if it is not practical in clinical settings.

Generative adversarial network with radiomic feature reproducibility analysis for computed tomography denoising

Background:Most computed tomography (CT) denoising algorithms have been evaluated using image quality analysis (IQA) methods developed for natural image, which do not adequately capture the texture details in medical imaging. Radiomics is an emerging image analysis technique that extracts texture information to provide a more objective basis for medical imaging diagnostics, overcoming the subjective nature of traditional methods. By utilizing the difficulty of reproducing radiomics features under different imaging protocols, we can more accurately evaluate the performance of CT denoising algorithms. Method:We introduced radiomic feature reproducibility analysis as an evaluation metric for a denoising algorithm. Also, we proposed a low-dose CT denoising method based on a generative adversarial network (GAN), which outperformed well-known CT denoising methods. Results:Although the proposed model produced excellent results visually, the traditional image assessment metrics such as peak signal-to-noise ratio and structural similarity failed to show distinctive performance differences between the proposed method and the conventional ones. However, radiomic feature reproducibility analysis provided a distinctive assessment of the CT denoising performance. Furthermore, radiomic feature reproducibility analysis allowed fine-tuning of the hyper-parameters of the GAN. Conclusion:We demonstrated that the well-tuned GAN architecture outperforms the well-known CT denoising methods. Our study is the first to introduce radiomics reproducibility analysis as an evaluation metric for CT denoising. We look forward that the study may bridge the gap between traditional objective and subjective evaluations in the clinical medical imaging field.

Evaluation of repeatability and reproducibility of radiomic features produced by the fan-beam kV-CT on a novel ring gantry-based PET/CT linear accelerator.

The RefleXion X1 is a novel radiotherapy delivery system on a ring gantry equipped with fan-beam kV-CT and PET imaging subsystems. The day-to-day scanning variability of radiomics features must be evaluated before any attempt to utilize radiomics features. This study aims to characterize the repeatability and reproducibility of radiomic features produced by the RefleXion X1 kV-CT. The Credence Cartridge Radiomics (CCR) phantom includes six cartridges of varied materials. It was scanned 10 times on the RefleXion X1 kVCT imaging subsystem over a 3-month period using the two most frequently used scanning protocols (BMS and BMF). Fifty-five radiomic features were extracted for each ROI on each CT scan and analyzed using LifeX software. The coefficient of variation (COV) was computed to evaluate the repeatability. Intraclass correlation coefficient (ICC) and concordance correlation coefficient (CCC) were used to evaluate the repeatability and reproducibility of the scanned images using 0.9 as the threshold. This process is repeated on a GE PET-CT scanner using several built-in protocols as a comparison. On average, 87% of the features on both scan protocols on the RefleXion X1 kVCT imaging subsystem can be considered repeatable as they met COV<10% criteria. On GE PET-CT, this number is similar at 86%. When we tighten the criteria to COV<5%, the RefleXion X1 kVCT imaging subsystem showed much better repeatability with 81% of features on average whereas GE PET-CT showed only 73.5% on average. About 91% and 89% of the features with ICC>0.9 respectively for BMS and BMF protocols on RefleXion X1. On the other hand, the percentage of features with ICC>0.9 on GE PET-CT ranges from 67% to 82%. The RefleXion X1 kVCT imaging subsystem showed excellent intra-scanner reproducibility between the scanning protocols much better than the GE PET CT scanner. For the inter-scanner reproducibility, the percentage of features with CCC>0.9 ranged from 49% to 80%. between X1 and GE PET-CT scanning protocols. Clinically useful CT radiomic features produced by the RefleXion X1 kVCT imaging subsystem are reproducible and stable over time, demonstrating its utility as a quantitative imaging platform.

Identification of reproducible radiomic features from on-board volumetric images: A multi-institutional phantom study.

Radiomics analysis using on-board volumetric images has attracted research attention as a method for predicting prognosis during treatment; however, the lack of standardization is still one of the main concerns. This study investigated the factors that influence the reproducibility of radiomic features extracted from on-board volumetric images using an anthropomorphic radiomics phantom. Furthermore, a phantom experiment was conducted with different treatment machines from multiple institutions as external validation to identify reproducible radiomic features. The phantom was designed to be 35 × 20 × 20cm with eight types of heterogeneous spheres (⌀=1, 2, and 3cm). On-board volumetric images were acquired using 15 treatment machines from eight institutions. Of these, kilovoltage cone-beam computed tomography (kV-CBCT) image data acquired from four treatment machines at one institution were used as an internal evaluation dataset to explore the reproducibility of radiomic features. The remaining image data, including kV-CBCT, megavoltage-CBCT (MV-CBCT), and megavoltage computed tomography (MV-CT) provided by seven different institutions (11 treatment machines), were used as an external validation dataset. A total of 1,302 radiomic features, including 18 first-order, 75 texture, 465 (i.e., 93 × 5) Laplacian of Gaussian (LoG) filter-based, and 744 (i.e., 93 × 8) wavelet filter-based features, were extracted within the spheres. The intraclass correlation coefficient (ICC) was calculated to explore feature repeatability and reproducibility using an internal evaluation dataset. Subsequently, the coefficient of variation (COV) was calculated to validate the feature variability of external institutions. An absolute ICC exceeding 0.85 or COV under 5% was considered indicative of a highly reproducible feature. For internal evaluation, ICC analysis showed that the median percentage of radiomic features with high repeatability was 95.2%. The ICC analysis indicated that the median percentages of highly reproducible features for inter-tube current, reconstruction algorithm, and treatment machine were decreased by 20.8%, 29.2%, and 33.3%, respectively. For external validation, the COV analysis showed that the median percentage of reproducible features was 31.5%. A total of 16 features, including nine LoG filter-based and seven wavelet filter-based features, were indicated as highly reproducible features. The gray-level run-length matrix (GLRLM) was classified as containing the most frequent features (N=8), followed by the gray-level dependence matrix (N=7) and gray-level co-occurrence matrix (N=1) features. We developed the standard phantom for radiomics analysis of kV-CBCT, MV-CBCT, and MV-CT images. With this phantom, we revealed that the differences in the treatment machine and image reconstruction algorithm reduce the reproducibility of radiomic features from on-board volumetric images. Specifically, the most reproducible features for external validation were LoG or wavelet filter-based GLRLM features. However, the acceptability of the identified features should be examined in advance at each institution before applying the findings to prognosis prediction.

RESOLVE-based radiomics in cervical cancer: improved image quality means better feature reproducibility?

To compare the reproducibility of apparent diffusion coefficient (ADC)-based radiomic features between readout-segmented echo-planar diffusion-weighted imaging (RESOLVE) and single-shot echo-planar diffusion-weighted imaging (SS-EPI DWI) in cervical cancer. The RESOLVE and SS-EPI DWI images of 36 patients with histopathologically confirmed cervical cancer were collected retrospectively. Two observers independently delineated the whole tumour on RESOLVE and SS-EPI DWI, and then copied them to the corresponding ADC maps. Shape, first-order, and texture features were extracted from ADC maps in the original and filtered (Laplacian of Gaussian [LoG] and wavelet) images. Thereafter, 1,316 features were generated in each RESOLVE and SS-EPI DWI, respectively. The reproducibility of radiomic features was assessed using intraclass correlation coefficient (ICC). In the original images, RESOLVE showed 92.86%, 66.67%, and 86.67% of features with excellent reproducibility in shape, first-order, and texture features, while SS-EPI DWI showed 85.71%, 72.22%, and 60% of features with excellent reproducibility, respectively. In the LoG and wavelet filtered images, RESOLVE had 56.77% and 65.32% of features with excellent reproducibility and SS-EPI DWI had 44.95% and 61.96% of features with excellent reproducibility, respectively. Compared with SS-EPI DWI, the feature reproducibility of RESOLVE was better in cervical cancer, especially for texture features. The filtered images cannot improve the feature reproducibility compared with the original images for both SS-EPI DWI and RESOLVE.

Feasibility and intra-and interobserver reproducibility of quantitative susceptibility mapping with radiomic features for intracranial dissecting intramural hematomas and atherosclerotic calcifications

Quantitative susceptibility mapping (QSM) for 61 patients with dissecting intramural hematomas (n = 36) or atherosclerotic calcifications (n = 25) in intracranial vertebral arteries were collected to assess intra- and interobserver reproducibility in a 3.0-T MR system between January 2015 and December 2017. Two independent observers each segmented regions of interest for lesions twice. The reproducibility was evaluated using intra-class correlation coefficients (ICC) and within-subject coefficients of variation (wCV) for means and concordance correlation coefficients (CCC) and ICC for radiomic features (CCC and ICC > 0.85) were used. Mean QSM values were 0.277 ± 0.092 ppm for dissecting intramural hematomas and − 0.208 ± 0.078 ppm for atherosclerotic calcifications. ICCs and wCVs were 0.885–0.969 and 6.5–13.7% in atherosclerotic calcifications and 0.712–0.865 and 12.4–18.7% in dissecting intramural hematomas, respectively. A total of 9 and 19 reproducible radiomic features were observed in dissecting intramural hematomas and atherosclerotic calcifications, respectively. QSM measurements in dissecting intramural hematomas and atherosclerotic calcifications were feasible and reproducible between intra- and interobserver comparisons, and some reproducible radiomic features were demonstrated.

Reproducibility of 2D versus 3D radiomics for quantitative assessment of fetal lung development: a retrospective fetal MRI study

PurposeTo investigate the reproducibility of radiomics features extracted from two-dimensional regions of interest (2D ROIs) versus whole lung (3D) ROIs in repeated in-vivo fetal magnetic resonance imaging (MRI) acquisitions.MethodsThirty fetal MRI scans including two axial T2-weighted acquisitions of the lungs were analysed. 2D (lung at the level of the carina) and 3D (whole lung) ROIs were manually segmented using ITK-Snap. Ninety-five radiomics features were extracted from 2 and 3D ROIs in initial and repeat acquisitions using Pyradiomics. Radiomics feature intra-class correlation coefficients (ICC) were calculated between 2 and 3D ROIs in the initial acquisition, and between 2 and 3D ROIs in repeated acquisitions, respectively.ResultsMRI data of 11 (36.7%) female and 19 (63.3%) male fetuses acquired at a median 25 + 0 gestational weeks plus days (GW) (interquartile range [IQR] 23 + 4 − 27 + 0 GW) were assessed. Median radiomics feature ICC between 2 and 3D ROIs in the initial MRI acquisition was 0.733 (IQR 0.313–0.814, range 0.018–0.970). ICCs between radiomics features extracted using 3D ROIs in initial and repeat acquisitions (median 0.908 [IQR 0.824–0.929, range 0.335–0.996]) were significantly higher compared to 2D ROIs (0.771 [0.699–0.835, 0.048–0.965]) (p < 0.001).ConclusionFetal MRI radiomics features extracted from 3D whole lung segmentation masks showed significantly higher reproducibility across repeat acquisitions compared to 2D ROIs. Therefore, fetal MRI whole lung radiomics features are robust diagnostic and potentially prognostic tools in the image-based in-vivo quantitative assessment of lung development.