Reproducibility Of Radiomic Features Research Articles

Enhancing the Clinical Utility of Radiomics: Addressing the Challenges of Repeatability and Reproducibility in CT and MRI.

Radiomics, which integrates the comprehensive characterization of imaging phenotypes with machine learning algorithms, is increasingly recognized for its potential in the diagnosis and prognosis of oncological conditions. However, the repeatability and reproducibility of radiomic features are critical challenges that hinder their widespread clinical adoption. This review aims to address the paucity of discussion regarding the factors that influence the reproducibility and repeatability of radiomic features and their subsequent impact on the application of radiomic models. We provide a synthesis of the literature on the repeatability and reproducibility of CT/MR-based radiomic features, examining sources of variation, the number of reproducible features, and the availability of individual feature repeatability indices. We differentiate sources of variation into random effects, which are challenging to control but can be quantified through simulation methods such as perturbation, and biases, which arise from scanner variability and inter-reader differences and can significantly affect the generalizability of radiomic model performance in diverse settings. Four suggestions for repeatability and reproducibility studies are suggested: (1) detailed reporting of variation sources, (2) transparent disclosure of calculation parameters, (3) careful selection of suitable reliability indices, and (4) comprehensive reporting of reliability metrics. This review underscores the importance of random effects in feature selection and harmonizing biases between development and clinical application settings to facilitate the successful translation of radiomic models from research to clinical practice.

Reproducibility of Radiomic Features in Glial Brain Tumors.

In our recent research, we have effectively demonstrated the feasibility of classifying magnetic resonance images (MRI) of glial tumors into four histological types utilizing standardized volume of interest (VOI), radiomics and machine learning. This research aims to determine the reproducibility of our approach when the locations of VOI are changed. We were able to demonstrate high reproducibility of ML results when the same feature selection methodology was employed across different VOIs. However, the reproducibility of radiomic features and their sets among various VOIs was not ensured for the sample size (n = 85) studied. The limited reproducibility of radiomic features should be taken into account when evaluating radiomics studies in glial tumors.

Effect of Deep Learning Image Reconstruction Algorithms on Radiomic Features of Pulmonary Nodules in Ultra-Low-Dose CT.

The purpose of this study is to explore the impact of deep learning image reconstruction (DLIR) algorithm on the quantification of radiomic features in ultra-low-dose computed tomography (ULD-CT) compared with adaptive statistical iterative reconstruction-Veo (ASIR-V). One hundred eighty-three patients with pulmonary nodules underwent standard-dose computed tomography (SDCT) (4.30 ± 0.36 mSv) and ULD-CT (UL-A, 0.57 ± 0.09 mSv or UL-B, 0.33 ± 0.04 mSv). SDCT was the reference standard using (ASIR-V) at 50% strength (50%ASIR-V). ULD-CT was reconstructed with 50%ASIR-V, DLIR at medium and high strength (DLIR-M, DLIR-H). Radiomics analysis extracted 102 features, and the intraclass correlation coefficient (ICC) quantified reproducibility between ULD-CT and SDCT reconstructed by 50%ASIR-V, DLIR-M, and DLIR-H for each feature. Among 102 radiomic features, the percentages of reproducibility of 50%ASIR-V, DLIR-M, and DLIR-H were 48.04% (49/102), 49.02% (50/102), and 52.94% (54/102), respectively. Shape and first order features demonstrated high reproducibility across different reconstruction algorithms and radiation doses, with mean ICC values exceeding 0.75. In texture features, DLIR-M and DLIR-H showed improved mean ICC values for pure ground glass nodules (pGGNs) from 0.69 ± 0.23 to 0.75 ± 0.18 and 0.81 ± 0.12, respectively, compared with 50%ASIR-V. Similarly, the mean ICC values for solid nodules (SNs) increased from 0.60 ± 0.19 to 0.66 ± 0.14 and 0.69 ± 0.13, respectively. Additionally, the mean ICC values of texture features for pGGNs and SNs in both ULD-CT groups decreased with reduced radiation dose. DLIR can improve the reproducibility of radiomic features at ultra-low doses compared with ASIR-V. In addition, pGGNs showed better reproducibility at ultra-low doses than SNs.

Magnetic Resonance Image Radiomic Reproducibility: The Impact of Preprocessing on Extracted Features from Gross and High-Risk Clinical Tumor Volumes in Cervical Cancer Patients before Brachytherapy.

Radiomic feature reproducibility assessment is critical in radiomics-based image biomarker discovery. This study aims to evaluate the impact of preprocessing parameters on the reproducibility of magnetic resonance image (MRI) radiomic features extracted from gross tumor volume (GTV) and high-risk clinical tumor volume (HR-CTV) in cervical cancer (CC) patients. This study included 99 patients with pathologically confirmed cervical cancer who underwent an MRI prior to receiving brachytherapy. The GTV and HR-CTV were delineated on T2-weighted MRI and inputted into 3D Slicer for radiomic analysis. Before feature extraction, all images were preprocessed to a combination of several parameters of Laplacian of Gaussian (1 and 2), resampling (0.5 and 1), and bin width (5, 10, 25, and 50). The reproducibility of radiomic features was analyzed using the intra-class correlation coefficient (ICC). Almost all shapes and first-order features had ICC values > 0.95. Most second-order texture features were not reproducible (ICC < 0.95) in GTV and HR-CTV. Furthermore, 20% of all neighboring gray-tone difference matrix texture features had ICC > 0.90 in both GTV and HR-CTV. The results presented here showed that MRI radiomic features are vulnerable to changes in preprocessing, and this issue must be understood and applied before any clinical decision-making. Features with ICC > 0.90 were considered the most reproducible features. Shape and first-order radiomic features were the most reproducible features in both GTV and HR-CTV. Our results also showed that GTV and HR-CTV radiomic features had similar changes against preprocessing sets.

Role of artificial intelligence applied to ultrasound in gynecology oncology: A systematic review.

The aim of this paper was to explore the role of artificial intelligence (AI) applied to ultrasound imaging in gynecology oncology. Web of Science, PubMed, and Scopus databases were searched. All studies were imported to RAYYAN QCRI software. The overall quality of the included studies was assessed using QUADAS-AI tool. Fifty studies were included, of these 37/50 (74.0%) on ovarian masses or ovarian cancer, 5/50 (10.0%) on endometrial cancer, 5/50 (10.0%) on cervical cancer, and 3/50 (6.0%) on other malignancies. Most studies were at high risk of bias for subject selection (i.e., sample size, source, or scanner model were not specified; data were not derived from open-source datasets; imaging preprocessing was not performed) and index test (AI models was not externally validated) and at low risk of bias for reference standard (i.e., the reference standard correctly classified the target condition) and workflow (i.e., the time between index test and reference standard was reasonable). Most studies presented machine learning models (33/50, 66.0%) for the diagnosis and histopathological correlation of ovarian masses, while others focused on automatic segmentation, reproducibility of radiomics features, improvement of image quality, prediction of therapy resistance, progression-free survival, and genetic mutation. The current evidence supports the role of AI as a complementary clinical and research tool in diagnosis, patient stratification, and prediction of histopathological correlation in gynecological malignancies. For example, the high performance of AI models to discriminate between benign and malignant ovarian masses or to predict their specific histology can improve the diagnostic accuracy of imaging methods.

A comparative analysis of preclinical computed tomography radiomics using cone-beam and micro-computed tomography scanners

Background and purposeRadiomics analysis extracts quantitative data (features) from medical images. These features could potentially reflect biological characteristics and act as imaging biomarkers within precision medicine. However, there is a lack of cross-comparison and validation of radiomics outputs which is paramount for clinical implementation. In this study, we compared radiomics outputs across two computed tomography (CT)-based preclinical scanners. Materials and methodsCone beam CT (CBCT) and µCT scans were acquired using different preclinical CT imaging platforms. The reproducibility of radiomics features on each scanner was assessed using a phantom across imaging energies (40 & 60 kVp) and segmentation volumes (44–238 mm3). Retrospective mouse scans were used to compare feature reliability across varying tissue densities (lung, heart, bone), scanners and after voxel size harmonisation. Reliable features had an intraclass correlation coefficient (ICC) > 0.8. ResultsFirst order and GLCM features were the most reliable on both scanners across different volumes. There was an inverse relationship between tissue density and feature reliability, with the highest number of features in lung (CBCT=580, µCT=734) and lowest in bone (CBCT=110, µCT=560). Comparable features for lung and heart tissues increased when voxel sizes were harmonised. We have identified tissue-specific preclinical radiomics signatures in mice for the lung (133), heart (35), and bone (15). ConclusionsPreclinical CBCT and µCT scans can be used for radiomics analysis to support the development of meaningful radiomics signatures. This study demonstrates the importance of standardisation and emphasises the need for multi-centre studies.

Reproducibility Analysis of Radiomic Features on T2-weighted MR Images after Processing and Segmentation Alterations in Neuroblastoma Tumors.

Purpose To evaluate the reproducibility of radiomics features extracted from T2-weighted MR images in patients with neuroblastoma. Materials and Methods A retrospective study included 419 patients (mean age, 29 months ± 34 [SD]; 220 male, 199 female) with neuroblastic tumors diagnosed between 2002 and 2023, within the scope of the PRedictive In-silico Multiscale Analytics to support cancer personalized diaGnosis and prognosis, Empowered by imaging biomarkers (ie, PRIMAGE) project, involving 746 T2/T2*-weighted MRI sequences at diagnosis and/or after initial chemotherapy. Images underwent processing steps (denoising, inhomogeneity bias field correction, normalization, and resampling). Tumors were automatically segmented, and 107 shape, first-order, and second-order radiomics features were extracted, considered as the reference standard. Subsequently, the previous image processing settings were modified, and volumetric masks were applied. New radiomics features were extracted and compared with the reference standard. Reproducibility was assessed using the concordance correlation coefficient (CCC); intrasubject repeatability was measured using the coefficient of variation (CoV). Results When normalization was omitted, only 5% of the radiomics features demonstrated high reproducibility. Statistical analysis revealed significant changes in the normalization and resampling processes (P < .001). Inhomogeneities removal had the least impact on radiomics (83% of parameters remained stable). Shape features remained stable after mask modifications, with a CCC greater than 0.90. Mask modifications were the most favorable changes for achieving high CCC values, with a radiomics features stability of 70%. Only 7% of second-order radiomics features showed an excellent CoV of less than 0.10. Conclusion Modifications in the T2-weighted MRI preparation process in patients with neuroblastoma resulted in changes in radiomics features, with normalization identified as the most influential factor for reproducibility. Inhomogeneities removal had the least impact on radiomics features. Keywords: Pediatrics, MR Imaging, Oncology, Radiomics, Reproducibility, Repeatability, Neuroblastic Tumors Supplemental material is available for this article. © RSNA, 2024 See also the commentary by Safdar and Galaria in this issue.

Reproducible Radiomics Features from Multi-MRI-Scanner Test-Retest-Study: Influence on Performance and Generalizability of Models.

Radiomics models trained on data from one center typically show a decline of performance when applied to data from external centers, hindering their introduction into large-scale clinical practice. Current expert recommendations suggest to use only reproducible radiomics features isolated by multiscanner test-retest experiments, which might help to overcome the problem of limited generalizability to external data. To evaluate the influence of using only a subset of robust radiomics features, defined in a prior in vivo multi-MRI-scanner test-retest-study, on the performance and generalizability of radiomics models. Retrospective. Patients with monoclonal plasma cell disorders. Training set (117 MRIs from center 1); internal test set (42 MRIs from center 1); external test set (143 MRIs from center 2-8). 1.5T and 3.0T; T1-weighted turbo spin echo. The task for the radiomics models was to predict plasma cell infiltration, determined by bone marrow biopsy, noninvasively from MRI. Radiomics machine learning models, including linear regressor, support vector regressor (SVR), and random forest regressor (RFR), were trained on data from center 1, using either all radiomics features, or using only reproducible radiomics features. Models were tested on an internal (center 1) and a multicentric external data set (center 2-8). Pearson correlation coefficient r and mean absolute error (MAE) between predicted and actual plasma cell infiltration. Fisher's z-transformation, Wilcoxon signed-rank test, Wilcoxon rank-sum test; significance level P < 0.05. When using only reproducible features compared with all features, the performance of the SVR on the external test set significantly improved (r = 0.43 vs. r = 0.18 and MAE = 22.6 vs. MAE = 28.2). For the RFR, the performance on the external test set deteriorated when using only reproducible instead of all radiomics features (r = 0.33 vs. r = 0.44, P = 0.29 and MAE = 21.9 vs. MAE = 20.5, P = 0.10). Using only reproducible radiomics features improves the external performance of some, but not all machine learning models, and did not automatically lead to an improvement of the external performance of the overall best radiomics model. Stage 2.

Multicenter PET image harmonization using generative adversarial networks

PurposeTo improve reproducibility and predictive performance of PET radiomic features in multicentric studies by cycle-consistent generative adversarial network (GAN) harmonization approaches.MethodsGAN-harmonization was developed to harmonize whole-body PET scans to perform image style and texture translation between different centers and scanners. GAN-harmonization was evaluated by application to two retrospectively collected open datasets and different tasks. First, GAN-harmonization was performed on a dual-center lung cancer cohort (127 female, 138 male) where the reproducibility of radiomic features in healthy liver tissue was evaluated. Second, GAN-harmonization was applied to a head and neck cancer cohort (43 female, 154 male) acquired from three centers. Here, the clinical impact of GAN-harmonization was analyzed by predicting the development of distant metastases using a logistic regression model incorporating first-order statistics and texture features from baseline 18F-FDG PET before and after harmonization.ResultsImage quality remained high (structural similarity: left kidney ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 0.800, right kidney ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 0.806, liver ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 0.780, lung ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 0.838, spleen ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 0.793, whole-body ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 0.832) after image harmonization across all utilized datasets. Using GAN-harmonization, inter-site reproducibility of radiomic features in healthy liver tissue increased at least by ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 5 ± 14% (first-order), ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 16 ± 7% (GLCM), ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 19 ± 5% (GLRLM), ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 16 ± 8% (GLSZM), ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 17 ± 6% (GLDM), and ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 23 ± 14% (NGTDM). In the head and neck cancer cohort, the outcome prediction improved from AUC 0.68 (95% CI 0.66–0.71) to AUC 0.73 (0.71–0.75) by application of GAN-harmonization.ConclusionsGANs are capable of performing image harmonization and increase reproducibility and predictive performance of radiomic features derived from different centers and scanners.

Abstract PO1-07-07: Radiomics features for distinguishing true recurrence versus new primary tumor following breast-conserving treatment

Abstract Background Despite modern surgical and irradiation techniques, ipsilateral breast tumor recurrence (IBTR) accounts for 5-15% of all cancer recurrences in women treated with breast conservative treatment. Historically, the methods to define true recurrence (TR) and new primary tumor (NP) for IBTR mainly rely on clinical and pathological criteria, limiting the accuracy of the discerption and causing misclassification. This study aimed to develop a preoperative, noninvasive model for distinguishing IBTR by integrating clinicopathological review with paired dynamic contrast-enhanced breast magnetic resonance imaging (DCE-MRI) at diagnosis and following in-breast recurrence. MethodsWe retrospectively extracted radiomics features from MRI to develop a radiomic cohort of IBTR (n =46), among which all patients underwent paired preoperative DCE-MRI. This radiomic cohort was divided into a training cohort (n =27) and a validation cohort (n =19) with stratified random sampling. Classification of IBTR as TR or NP on the basis of tumor location, histologic subtype, estrogen receptor, and HER2 status was set as the gold standard. The least absolute shrinkage and selection operator (LASSO) regression and logistic regression were utilized to perform radiomics feature selection and model training, respectively. The clinical utility of the model was determined via decision curve analysis (DCA). Results We selected three radiomics features (first-order feature Kurtosis from the first postcontrast phase on IBTR DCE-MRI, GLCM feature Imc2 from the first postcontrast phase on IBTR DCE-MRI, and the delta feature Correlation from the precontrast phase of the change between IBTR and primary tumor DCE-MRI) to develop an IBTR-classification predicting radiomic model, which performed well in the validation cohort (AUC 0.867, 95% confidence interval (CI) 0.694-1). Further investigation for sensitivity (73.3%) and specificity (100%) verified a favorable concordance between the radiomic classification and the conventional standard, with a diagnostic accuracy of 79%. Conclusions: Our study demonstrated the feasibility of the radiomics model in predicting IBTR classification and provided preoperative information about the nature of “recurrence”. This might have important implications in surgical approaches and multidisciplinary care for IBTR. Further efforts are needed to improve the reproducibility of radiomics features and models in multiple centers. Citation Format: Feilin Qu, Guan-Hua Su, Jin-Hui Li, Chao You, Zhi-Ming Shao. Radiomics features for distinguishing true recurrence versus new primary tumor following breast-conserving treatment [abstract]. In: Proceedings of the 2023 San Antonio Breast Cancer Symposium; 2023 Dec 5-9; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2024;84(9 Suppl):Abstract nr PO1-07-07.

Influence of image preprocessing on the segmentation-based reproducibility of radiomic features: in vivo experiments on discretization and resampling parameters

To systematically investigate the impact of image preprocessing parameters on the segmentation-based reproducibility of magnetic resonance imaging (MRI) radiomic features. The MRI scans of 50 patients were included from the multi-institutional Brain Tumor Segmentation 2021 public glioma dataset. Whole tumor volumes were manually segmented by two independent readers, with the participation of eight readers. Radiomic features were extracted from two sequences: T2-weighted (T2) and contrast-enhanced T1-weighted (T1ce). Two methods were considered for discretization: bin count (i.e., relative discretization) and bin width (i.e., absolute discretization). Ten discretization (five for each method) and five resampling parameters were varied while other parameters were fixed. The intraclass correlation coefficient (ICC) was used for reliability analysis based on two commonly used cut-off values (0.75 and 0.90). Image preprocessing parameters had a significant impact on the segmentation-based reproducibility of radiomic features. The bin width method yielded more reproducible features than the bin count method. In discretization experiments using the bin width on both sequences, according to the ICC cut-off values of 0.75 and 0.90, the rate of reproducible features ranged from 70% to 84% and from 35% to 57%, respectively, with an increasing percentage trend as parameter values decreased (from 84 to 5 for T2; 100 to 6 for T1ce). In the resampling experiments, these ranged from 53% to 74% and from 10% to 20%, respectively, with an increasing percentage trend from lower to higher parameter values (physical voxel size; from 1 x 1 x 1 to 2 x 2 x 2 mm3). The segmentation-based reproducibility of radiomic features appears to be substantially influenced by discretization and resampling parameters. Our findings indicate that the bin width method should be used for discretization and lower bin width and higher resampling values should be used to allow more reproducible features.

Contrast Agent Dynamics Determine Radiomics Profiles in Oncologic Imaging.

The reproducibility of radiomics features extracted from CT and MRI examinations depends on several physiological and technical factors. The aim was to evaluate the impact of contrast agent timing on the stability of radiomics features using dynamic contrast-enhanced perfusion CT (dceCT) or MRI (dceMRI) in prostate and lung cancers. Radiomics features were extracted from dceCT or dceMRI images in patients with biopsy-proven peripheral prostate cancer (pzPC) or biopsy-proven non-small cell lung cancer (NSCLC), respectively. Features that showed significant differences between contrast phases were identified using linear mixed models. An L2-penalized logistic regression classifier was used to predict class labels for pzPC and unaffected prostate regions-of-interest (ROIs). Nine pzPC and 28 NSCLC patients, who were imaged with dceCT and/or dceMRI, were included in this study. After normalizing for individual enhancement patterns by defining seven individual phases based on a reference vessel, 19, 467 and 128 out of 1204 CT features showed significant temporal dynamics in healthy prostate parenchyma, prostate tumors and lung tumors, respectively. CT radiomics-based classification accuracy of healthy and tumor ROIs was highly dependent on contrast agent phase. For dceMRI, 899 and 1027 out of 1118 features were significantly dependent on time after contrast agent injection for prostate and lung tumors. CT and MRI radiomics features in both prostate and lung tumors are significantly affected by interindividual differences in contrast agent dynamics.

Stability of radiomic features from positron emission tomography images: a phantom study comparing advanced reconstruction algorithms and ordered subset expectation maximization.

In this study, we compared the repeatability and reproducibility of radiomic features obtained from positron emission tomography (PET) images according to the reconstruction algorithm used-advanced reconstruction algorithms, such as HYPER iterative (IT), HYPER deep learning reconstruction (DLR), and HYPER deep progressive reconstruction (DPR), or traditional Ordered Subset Expectation Maximization (OSEM)-to understand the potential variations and implications of using advanced reconstruction techniques in PET-based radiomics. We used a heterogeneous phantom with acrylic spherical beads (4- or 8-mm diameter) filled with 18F. PET images were acquired and reconstructed using OSEM, IT, DLR, and DPR. Original and wavelet radiomic features were calculated using SlicerRadiomics. Radiomic feature repeatability was assessed using the Coefficient of Variance (COV) and intraclass correlation coefficient (ICC), and inter-acquisition time reproducibility was assessed using the concordance correlation coefficient (CCC). For the 4- and 8-mm diameter beads phantom, the proportion of radiomic features with a COV < 10% was equivocal or higher for the advanced reconstruction algorithm than for OSEM. ICC indicated that advanced methods generally outperformed OSEM in repeatability, except for the original features of the 8-mm beads phantom. In the inter-acquisition time reproducibility analysis, the combinations of 3 and 5min exhibited the highest reproducibility in both phantoms, with IT and DPR showing the highest proportion of radiomic features with CCC > 0.8. Advanced reconstruction methods provided enhanced stability of radiomic features compared with OSEM, suggesting their potential for optimal image reconstruction in PET-based radiomics, offering potential benefits in clinical diagnostics and prognostics.

CT Cadaveric dataset for Radiomics features stability assessment in lumbar vertebrae

Radiomics features (RFs) studies have showed limitations in the reproducibility of RFs in different acquisition settings. To date, reproducibility studies using CT images mainly rely on phantoms, due to the harness of patient exposure to X-rays. The provided CadAIver dataset has the aims of evaluating how CT scanner parameters effect radiomics features on cadaveric donor. The dataset comprises 112 unique CT acquisitions of a cadaveric truck acquired on 3 different CT scanners varying KV, mA, field-of-view, and reconstruction kernel settings. Technical validation of the CadAIver dataset comprises a comprehensive univariate and multivariate GLM approach to assess stability of each RFs extracted from lumbar vertebrae. The complete dataset is publicly available to be applied for future research in the RFs field, and could foster the creation of a collaborative open CT image database to increase the sample size, the range of available scanners, and the available body districts.

Impact of harmonization on the reproducibility of MRI radiomic features when using different scanners, acquisition parameters, and image pre-processing techniques: a phantom study

This study investigated the impact of ComBat harmonization on the reproducibility of radiomic features extracted from magnetic resonance images (MRI) acquired on different scanners, using various data acquisition parameters and multiple image pre-processing techniques using a dedicated MRI phantom. Four scanners were used to acquire an MRI of a nonanatomic phantom as part of the TCIA RIDER database. In fast spin-echo inversion recovery (IR) sequences, several inversion durations were employed, including 50, 100, 250, 500, 750, 1000, 1500, 2000, 2500, and 3000 ms. In addition, a 3D fast spoiled gradient recalled echo (FSPGR) sequence was used to investigate several flip angles (FA): 2, 5, 10, 15, 20, 25, and 30 degrees. Nineteen phantom compartments were manually segmented. Different approaches were used to pre-process each image: Bin discretization, Wavelet filter, Laplacian of Gaussian, logarithm, square, square root, and gradient. Overall, 92 first-, second-, and higher-order statistical radiomic features were extracted. ComBat harmonization was also applied to the extracted radiomic features. Finally, the Intraclass Correlation Coefficient (ICC) and Kruskal-Wallis’s (KW) tests were implemented to assess the robustness of radiomic features. The number of non-significant features in the KW test ranged between 0–5 and 29–74 for various scanners, 31–91 and 37–92 for three times tests, 0–33 to 34–90 for FAs, and 3–68 to 65–89 for IRs before and after ComBat harmonization, with different image pre-processing techniques, respectively. The number of features with ICC over 90% ranged between 0–8 and 6–60 for various scanners, 11–75 and 17–80 for three times tests, 3–83 to 9–84 for FAs, and 3–49 to 3–63 for IRs before and after ComBat harmonization, with different image pre-processing techniques, respectively. The use of various scanners, IRs, and FAs has a great impact on radiomic features. However, the majority of scanner-robust features is also robust to IR and FA. Among the effective parameters in MR images, several tests in one scanner have a negligible impact on radiomic features. Different scanners and acquisition parameters using various image pre-processing might affect radiomic features to a large extent. ComBat harmonization might significantly impact the reproducibility of MRI radiomic features.Graphical

Synthesis of virtual monoenergetic images from kilovoltage peak images using wavelet loss enhanced CycleGAN for improving radiomics features reproducibility.

Dual-energy computed tomography (CT) can provide a range of image information beyond conventional CT through virtual monoenergetic images (VMIs). The purpose of this study was to investigate the impact of material decomposition in detector-based spectral CT on radiomics features and effectiveness of using deep learning-based image synthesis to improve the reproducibility of radiomics features. In this paper, spectral CT image data from 45 esophageal cancer patients were collected for investigation retrospectively. First, we computed the correlation coefficient of radiomics features between conventional kilovoltage peak (kVp) CT images and VMI. Then, a wavelet loss-enhanced CycleGAN (WLL-CycleGAN) with paired loss terms was developed to synthesize virtual monoenergetic CT images from the corresponding conventional single-energy CT (SECT) images for improving radiomics reproducibility. Finally, the radiomic features in 6 different categories, including gray-level co-occurrence matrix (GLCM), gray-level difference matrix (GLDM), gray-level run-length matrix (GLRLM), gray-level size-zone matrix (GLSZM), neighborhood gray-tone difference matrix (NGTDM), and wavelet, were extracted from the gross tumor volumes from conventional single energy CT, synthetic virtual monoenergetic CT images, and virtual monoenergetic CT images. Comparison between errors in the VMI and synthetic VMI (sVMI) suggested that the performance of our proposed deep learning method improved the radiomic feature accuracy. Material decomposition of dual-layer dual-energy CT (DECT) can substantially influence the reproducibility of the radiomic features, and the degree of impact is feature dependent. The average reduction of radiomics errors for 15 patients in testing sets was 96.9% for first-order, 12.1% for GLCM, 12.9% for GLDM, 15.7% for GLRLM, 50.3% for GLSZM, 53.4% for NGTDM, and 6% for wavelet features. The work revealed that material decomposition has a significant effect on the radiomic feature values. The deep learning-based method reduced the influence of material decomposition in VMIs and might improve the robustness and reproducibility of radiomic features in esophageal cancer. Quantitative results demonstrated that our proposed wavelet loss-enhanced paired CycleGAN outperforms the original CycleGAN.

CT and MRI radiomics of bone and soft-tissue sarcomas: an updated systematic review of reproducibility and validation strategies.

To systematically review radiomic feature reproducibility and model validation strategies in recent studies dealing with CT and MRI radiomics of bone and soft-tissue sarcomas, thus updating a previous version of this review which included studies published up to 2020. A literature search was conducted on EMBASE and PubMed databases for papers published between January 2021 and March 2023. Data regarding radiomic feature reproducibility and model validation strategies were extracted and analyzed. Out of 201 identified papers, 55 were included. They dealt with radiomics of bone (n= 23) or soft-tissue (n = 32) tumors. Thirty-two (out of 54 employing manual or semiautomatic segmentation, 59%) studies included a feature reproducibility analysis. Reproducibility was assessed based on intra/interobserver segmentation variability in 30 (55%) and geometrical transformations of the region of interest in 2 (4%) studies. At least one machine learning validation technique was used for model development in 34 (62%) papers, and K-fold cross-validation was employed most frequently. A clinical validation of the model was reported in 38 (69%) papers. It was performed using a separate dataset from the primary institution (internal test) in 22 (40%), an independent dataset from another institution (external test) in 14 (25%) and both in 2 (4%) studies. Compared to papers published up to 2020, a clear improvement was noted with almost double publications reporting methodological aspects related to reproducibility and validation. Larger multicenter investigations including external clinical validation and the publication of databases in open-access repositories could further improve methodology and bring radiomics from a research area to the clinical stage. An improvement in feature reproducibility and model validation strategies has been shown in this updated systematic review on radiomics of bone and soft-tissue sarcomas, highlighting efforts to enhance methodology and bring radiomics from a research area to the clinical stage. • 2021-2023 radiomic studies on CT and MRI of musculoskeletal sarcomas were reviewed. • Feature reproducibility was assessed in more than half (59%) of the studies. • Model clinical validation was performed in 69% of the studies. • Internal (44%) and/or external (29%) test datasets were employed for clinical validation.

Effects of Interobserver Segmentation Variability and Intensity Discretization on MRI-Based Radiomic Feature Reproducibility of Lipoma and Atypical Lipomatous Tumor

Segmentation and image intensity discretization impact on radiomics workflow. The aim of this study is to investigate the influence of interobserver segmentation variability and intensity discretization methods on the reproducibility of MRI-based radiomic features in lipoma and atypical lipomatous tumor (ALT). Thirty patients with lipoma or ALT were retrospectively included. Three readers independently performed manual contour-focused segmentation on T1-weighted and T2-weighted sequences, including the whole tumor volume. Additionally, a marginal erosion was applied to segmentations to evaluate its influence on feature reproducibility. After image pre-processing, with included intensity discretization employing both fixed bin number and width approaches, 1106 radiomic features were extracted from each sequence. Intraclass correlation coefficient (ICC) 95% confidence interval lower bound ≥ 0.75 defined feature stability. In contour-focused vs. margin shrinkage segmentation, the rates of stable features extracted from T1-weighted and T2-weighted images ranged from 92.68 to 95.21% vs. 90.69 to 95.66% after fixed bin number discretization and from 95.75 to 97.65% vs. 95.39 to 96.47% after fixed bin width discretization, respectively, with no difference between the two segmentation approaches (p ≥ 0.175). Higher stable feature rates and higher feature ICC values were found when implementing discretization with fixed bin width compared to fixed bin number, regardless of the segmentation approach (p < 0.001). In conclusion, MRI radiomic features of lipoma and ALT are reproducible regardless of the segmentation approach and intensity discretization method, although a certain degree of interobserver variability highlights the need for a preliminary reliability analysis in future studies.

Multi-sequence MRI radiomics of colorectal liver metastases: Which features are reproducible across readers?

PurposeTo assess the inter-reader reproducibility of radiomics features on multiple MRI sequences after segmentations of colorectal liver metastases (CRLM). Method30 CRLM (in 23 patients) were manually delineated by three readers on MRI before the start of chemotherapy on the contrast enhanced T1-weighted images (CE-T1W) in the portal venous phase, T2-weighted images (T2W) and b800 diffusion weighted images (DWI). DWI delineations were copied to the ADC-maps. 107 radiomics features were extracted per sequence. The intraclass correlation coefficient (ICC) was calculated per feature. Features were considered reproducible if ICC > 0.9. Results90% of CE-T1W features were reproducible with a median ICC of 0.98 (range 0.76–1.00). 81% of DWI features were robust with median ICC = 0.97 (range 0.38–1.00). The T2W features had a median ICC of 0.96 (range 0.55–0.99) and were reproducible in 80%. ADC showed the lowest number of reproducible features with 58% and median ICC = 0.91 (range 0.38–0.99) When considering the lower bound of the ICC 95% confidence intervals, 58%, 66%, 54% and 29% reached 0.9 for the CE-T1W, DWI, T2W and ADC features, respectively. The feature class with the best reproducibility differed per sequence. ConclusionsThe majority of MRI radiomics features from CE-T1W, T2W, DWI and ADC in colorectal liver metastases were robust for segmentation variability between readers. The CE-T1W yielded slightly better reproducibility results compared to DWI and T2W. The ADC features seem more susceptible to reader differences compared to the other three sequences.

Creating a 3D-printed pelvic phantom to explore the impact of Magnetic Resonance (MR) scanner noise in radiomics analysis, a technical note.

Reproducibility of radiomics features necessitates that scanner noise be considered prior to feature extraction. Phantom research provides the opportunity for such ‘ground truth’ measurements, without the additional complication of patient-related factors. The aim of this technical note was to create a 3D printed Magnetic Resonance Imaging (MRI)-compatible pelvic phantom that can be used for subsequent analysis of the impact of scanner noise on the reproducibility of radiomics features.A 3D printed phantom of a male pelvis was created using fused deposition modelling technology. It was 3D printed using the high density MRI-compatible acrylonitrile butadiene styrene (ABS). The ‘negative’ mould created was then filled with silicone, and the prostate gland and femoral heads were also simulated.We successfully created an MRI-compatible 3D printed pelvic phantom, with a test scan. The phantom will subsequently be utilised to determine the impact of MRI scanner noise on radiomics feature reproducibility.