Reproducibility Of Radiomic Features Research Articles

A Novel Benchmarking Approach to Assess the Agreement among Radiomic Tools.

Background The translation of radiomic models into clinical practice is hindered by the limited reproducibility of features across software and studies. Standardization is needed to accelerate this process and to bring radiomics closer to clinical deployment. Purpose To assess the standardization level of seven radiomic software programs and investigate software agreement as a function of built-in image preprocessing (eg, interpolation and discretization), feature aggregation methods, and the morphological characteristics (ie, volume and shape) of the region of interest (ROI). Materials and Methods The study was organized into two phases: In phase I, the two Image Biomarker Standardization Initiative (IBSI) phantoms were used to evaluate the IBSI compliance of seven software programs. In phase II, the reproducibility of all IBSI-standardized radiomic features across tools was assessed with two custom Italian multicenter Shared Understanding of Radiomic Extractors (ImSURE) digital phantoms that allowed, in conjunction with a systematic feature extraction, observations on whether and how feature matches between program pairs varied depending on the preprocessing steps, aggregation methods, and ROI characteristics. Results In phase I, the software programs showed different levels of completeness (ie, the number of computable IBSI benchmark values). However, the IBSI-compliance assessment revealed that they were all standardized in terms of feature implementation. When considering additional preprocessing steps, for each individual program, match percentages fell by up to 30%. In phase II, the ImSURE phantoms showed that software agreement was dependent on discretization and aggregation as well as on ROI shape and volume factors. Conclusion The agreement of radiomic software varied in relation to factors that had already been standardized (eg, interpolation and discretization methods) and factors that need standardization. Both dependences must be resolved to ensure the reproducibility of radiomic features and to pave the way toward the clinical adoption of radiomic models. Published under a CC BY 4.0 license. Online supplemental material is available for this article. See also the editorial by Steiger in this issue. An earlier incorrect version appeared online and in print. This article was corrected on March 2, 2022.

Robustness of CT radiomics features: consistency within and between single-energy CT and dual-energy CT

ObjectivesTo evaluate inter- and intra- scan mode and scanner repeatability and reproducibility of radiomics features within and between single-energy CT (SECT) and dual-energy CT (DECT).MethodsA standardized phantom with sixteen rods of clinical-relevant densities was scanned on seven DECT-capable scanners and three SECT-only scanners. The acquisition parameters were selected to present typical abdomen-pelvic examinations with the same voxel size. Images of SECT at 120 kVp and corresponding 120 kVp-like virtual monochromatic images (VMIs) in DECT which were generated according to scanners were analyzed. Regions of interest were drawn with rigid registrations to avoid variations due to segmentation. Radiomics features were extracted via Pyradiomics platform. Test-retest repeatability was evaluated by Bland-Altman analysis for repeated scans. Intra-scanner reproducibility for different scan modes was tested by intraclass correlation coefficient (ICC) and concordance correlation coefficient (CCC). Inter-scanner reproducibility among different scanners for same scan mode was assessed by coefficient of variation (CV) and quartile coefficient of dispersion (QCD).ResultsThe test-retest analysis presented that 92.91% and 87.02% of the 94 assessed features were repeatable for SECT 120kVp and DECT 120 kVp-like VMIs, respectively. The intra-scanner analysis for SECT 120kVp vs DECT 120 kVp-like VMIs demonstrated that 10.76% and 10.28% of features were with ICC > 0.90 and CCC > 0.90, respectively. The inter-scanner analysis showed that 17.09% and 27.73% of features for SECT 120kVp were with CV < 10% and QCD < 10%, and 15.16% and 32.78% for DECT 120 kVp-like VMIs, respectively.ConclusionsThe majority of radiomics features were non-reproducible within and between SECT and DECT.Key Points• Although the test-retest analysis showed high repeatability for radiomics features, the overall reproducibility of radiomics features within and between SECT and DECT was low.• Only about one-tenth of radiomics features extracted from SECT images and corresponding DECT images did match each other, even their average photon energy levels were considered alike, indicating that the scan mode potentially altered the radiomics features.• Less than one-fifth of radiomics features were reproducible among multiple SECT and DECT scanners, regardless of their fixed acquisition and reconstruction parameters, suggesting the necessity of scanning protocol adjustment and post-scan harmonization process.

The impact of phantom design and material-dependence on repeatability and reproducibility of CT-based radiomics features.

To understand the design of radiomics phantom and material-dependence on repeatability and reproducibility of computed tomography (CT) radiomics features. A radiomics phantom consisting of various materials with uniformity, textural, and biological components, was constructed. The phantom was scanned with different manufacturer CT scanners and the scans were repeated multiple times on the same scanner with different acquisition settings as kVp, mAs, orientation, field of view (FOV), slice thickness, pitch, reconstruction kernels, and acquisition mode. A total of 72 phantom scans were included. For each scan, 18 different regions of interest (ROI) were contoured and 708 radiomics features were extracted from each ROI via an open source radiomics tool, IBEX. To relate the phantom data to patient data, the radiomics features from different phantom materials were compared with those extracted from 50 patients' images of five disease sites as brain, head-and-neck, breast, liver, and lung cases using box-plots comparison and principal component analysis (PCA). The temporal stability of imaging features was then evaluated with respect to a controlled scenario (test-retest) via the intra-class correlation coefficient (ICC). The reproducibility of radiomics features with respect to different scanners or acquisition settings were further evaluated with concordance correlation coefficients (CCC). Among all phantom materials, the biological component had feature values closest to human tissues, especially for tumors in brain and liver. The textural component showed similar ranges of variation to lung lesions, particularly for cartridges of rice, cereal, and the 3D-printed textural phantom with fine and rough grid. It also showed that certain materials, such as polystyrene foam, plaster, and peanuts, did not have comparable values to human tissue and could be excluded for future phantom design. High repeatability was observed in the test-retest study as indicated by an ICC value of 0.998±0.020. All materials were used for feature stability analysis. For the inter-scanner study, shape-related features were the most-reliable category with 94% of features having CCC≥0.9, while gradient orientation histogram (GOH) were the least-reliable with only 14.6% meeting the criteria. For the intra-scanner study, the reproducibility of CT-based radiomics features showed material-dependence. In general, the instability of radiomics features introduced by kVp, mAs, pitch, acquisition mode, and orientation were relatively mild. However, the homogeneous materials were more vulnerable to those changes compared to materials with textural patterns. Regardless of material compositions, resolution parameters like FOV and slice thickness, could have large impact on feature stability. Switching between standard and bone reconstruction kernels could also result significant changes to feature reproducibility. We have built a radiomics phantom using materials that cover a wide span of tumor textures seen in oncological CT images. The designed phantom presents a preliminary opportunity for investigating reproducibility of radiomics features and the reproducibility can be material dependent. Thus, in the radiomics quality assurance design, it is important to choose appropriate materials that can provide a close range of radiomics features to patients with specific disease sites dependency taken into consideration.

Role of hepatic metastatic lesion size on inter-reader reproducibility of CT-based radiomics features.

To evaluate the effect of hepatic metastatic lesion size on inter-reader reproducibility of CT-based 2D radiomics imaging features. Computerized tomography (CT) scans of 59 liver metastases from 34 patients with colorectal cancer were evaluated. Image segmentation was performed manually by three readers blinded to each other's results. For each radiomics feature, we created two datasets by sorting measurements according to size, i.e., (i) from the smallest to the largest lesion and (ii) from the largest to the smallest lesion. The Lin concordance correlation coefficient (CCC) was employed to analyze the reproducibility of radiomics features. In particular, the CCC was computed as a function of a number of elements in the dataset, by gradually adding lesions from each sorted dataset. To evaluate the effect of lesion size, we analyzed the difference between these two functions thus assessing the contribution of small and large lesions into the reproducibility of radiomics features. Inter-reader reproducibility of CT-based 2D radiomics features assessed using Lin's CCC demonstrates tumor-size dependence. For example, the Lin CCC for GLCM contrast equals 0.88 (95% C.I. 0.84 to 0.92, p < 0.003) and could change by an additional + / - 0.06 depending on the presence of large or small lesions. Groups of "large" and "small" lesions show different inter-reader reproducibility. The inter-reader reproducibility from the mixed group consisting of "large" and "small" lesions depends on the lesion-size distribution and can vary widely. This finding could partially explain variability in reproducibility of radiomics features in the literature. • Groups of "large" and "small" lesions show different inter-reader reproducibility. • The inter-reader reproducibility from the mixed group consisting of "large" and "small" lesions depends on the lesion-size distribution.

Reproducibility of radiomic features in CT images of NSCLC patients: an integrative analysis on the impact of acquisition and reconstruction parameters

BackgroundWe investigated to what extent tube voltage, scanner model, and reconstruction algorithm affect radiomic feature reproducibility in a single-institution retrospective database of computed tomography images of non-small-cell lung cancer patients.MethodsThis study was approved by the Institutional Review Board (UID 2412). Images of 103 patients were considered, being acquired on either among two scanners, at 100 or 120 kVp. For each patient, images were reconstructed with six iterative blending levels, and 1414 features were extracted from each reconstruction. At univariate analysis, Wilcoxon-Mann-Whitney test was applied to evaluate feature differences within scanners and voltages, whereas the impact of the reconstruction was established with the overall concordance correlation coefficient (OCCC). A multivariable mixed model was also applied to investigate the independent contribution of each acquisition/reconstruction parameter. Univariate and multivariable analyses were combined to analyse feature behaviour.ResultsScanner model and voltage did not affect features significantly. The reconstruction blending level showed a significant impact at both univariate analysis (154/1414 features yielding an OCCC < 0.85) and multivariable analysis, with most features (1042/1414) revealing a systematic trend with the blending level (multiple comparisons adjusted p < 0.05). Reproducibility increased in association to image processing with smooth filters, nonetheless specific investigation in relation to clinical endpoints should be performed to ensure that textural information is not removed.ConclusionsCombining univariate and multivariable models is allowed to identify features for which corrections may be applied to reduce the trend with the algorithm and increase reproducibility. Subsequent clustering may be applied to eliminate residual redundancy.

The stability and repeatability of radiomics features based on lung diffusion-weighted imaging

Objective: To evaluate the feasibility, robustness and reproducibility of radiomics features derived from lung diffusion-weighted imaging (DWI). Methods: Thirty patients with pulmonary nodules/masses who underwent magnetic resonance imaging examination in the Department of Radiology, the First Affiliated Hospital of Guangzhou Medical University, from January 4 2019 to May 5 2019, including 16 males and 14 females, aged from 27 to 69 (57±11) years, were prospectively collected. Planar echo imaging (EPI) -DWI and fast spin-echo (TSE) -DWI scans were performed under free-breathing conditions. Each scan was repeated at an interval of 5 minutes, and the corresponding apparent diffusion coefficient (ADC) maps were reconstructed. Each DWI and ADC sequence (a total of eight groups of images) were manually segmented by two radiologists, and a total of 396 radiomics features in 6 categories were extracted from each group of images. Consistency correlation coefficient (CCC) and dynamic range (DR) were used to evaluate the robustness of features between two scans, and stable features were defined as both CCC values and DR values ≥0.85. Intra-observer and interobserver reproducibility were evaluated by intra-group correlation coefficient (ICC), and ICC values≥0.75 was considered to be good reproducibility. Results: Regardless of EPI or TSE technique, the number of robust features extracted fromDWI (TSE: n=197, EPI: n=169) were higher than that of the corresponding ADC (TSE: n=126, EPI: n=148). The proportion of robust features of TSE-DWI、EPI-DWI、TSE-ADC、EPI-ADC was 49.7% (197/396), 42.7% (169/396), 31.8% (126/396) and 37.4% (148/396), respectively. Of the 396 features, 54 (13.6%) of them demonstrated great robustness (CCC and DR≥0.85) and interobserver and interobserver reproducibility (ICC≥0.75) across all sequences. Conclusions: Radiomics features derived from lung DWI showed robustness and reproducibility. Different sequences and different feature clusters have different proportions of stable features, and some features have good robustness and reproducibility between different scans, different observers, and even different sequences.

The Effects of Automatic Segmentations on Preoperative Lymph Node Status Prediction Models With Ultrasound Radiomics for Patients With Early Stage Cervical Cancer.

Introduction: The purpose of this study is to investigate the effects of automatic segmentation algorithms on the performance of ultrasound (US) radiomics models in predicting the status of lymph node metastasis (LNM) for patients with early stage cervical cancer preoperatively. Methods: US images of 148 cervical cancer patients were collected and manually contoured by two senior radiologists. The four deep learning-based automatic segmentation models, namely U-net, context encoder network (CE-net), Resnet, and attention U-net were constructed to segment the tumor volumes automatically. Radiomics features were extracted and selected from manual and automatically segmented regions of interest (ROIs) to predict the LNM of these cervical cancer patients preoperatively. The reliability and reproducibility of radiomics features and the performances of prediction models were evaluated. Results: A total of 449 radiomics features were extracted from manual and automatic segmented ROIs with Pyradiomics. Features with an intraclass coefficient (ICC) > 0.9 were all 257 (57.2%) from manual and automatic segmented contours. The area under the curve (AUCs) of validation models with radiomics features extracted from manual, attention U-net, CE-net, Resnet, and U-net were 0.692, 0.755, 0.696, 0.689, and 0.710, respectively. Attention U-net showed best performance in the LNM prediction model with a lowest discrepancy between training and validation. The AUCs of models with automatic segmentation features from attention U-net, CE-net, Resnet, and U-net were 9.11%, 0.58%, –0.44%, and 2.61% higher than AUC of model with manual contoured features, respectively. Conclusion: The reliability and reproducibility of radiomics features, as well as the performance of radiomics models, were affected by manual segmentation and automatic segmentations.

OD126 - Reproducibility of radiomic features in CT images of NSCLC patients

OD126 - Reproducibility of radiomic features in CT images of NSCLC patients

Deep Learning-Based Image Conversion Improves the Reproducibility of Computed Tomography Radiomics Features: A Phantom Study.

This study aimed to evaluate the usefulness of deep learning-based image conversion to improve the reproducibility of computed tomography (CT) radiomics features. This study was conducted using an abdominal phantom with liver nodules. We developed an image conversion algorithm using a residual feature aggregation network to reproduce radiomics features with CT images under various CT protocols and reconstruction kernels. External validation was performed using images from different scanners, consisting of 8 different protocols. To evaluate the variability of radiomics features, regions of interest (ROIs) were drawn by targeting the liver parenchyma, vessels, paraspinal area, and liver nodules. We extracted 18 first-order, 68 second-order, and 688 wavelet radiomics features. Measurement variability was assessed using the concordance correlation coefficient (CCC), compared with the ground-truth image. In the ROI-based analysis, there was an 83.3% improvement of CCC (80/96; 4 ROIs with 3 categories of radiomics features and 8 protocols) in synthetic images compared with the original images. Among them, the 56 CCC pairs showed a significant increase after image synthesis. In the radiomics feature-based analysis, 62.0% (3838 of 6192; 774 radiomics features with 8 protocols) features showed increased CCC after image synthesis, and a significant increase was noted in 26.9% (1663 of 6192) features. In particular, the first-order feature (79.9%, 115/144) showed better improvement in terms of the reproducibility of radiomics feature than the second-order (59.9%, 326/544) or wavelet feature (61.7%, 3397/5504). Our study demonstrated that a deep learning model for image conversion can improve the reproducibility of radiomics features across various CT protocols, reconstruction kernels, and CT scanners.

Repeatability and Reproducibility of Computed Tomography Radiomics for Pulmonary Nodules: A Multicenter Phantom Study.

BackgroundRadiomics can yield minable information from medical images, which can facilitate computer-aided diagnosis. However, the lack of repeatability and reproducibility of radiomic features (RFs) may hinder their generalizability in clinical applications.ObjectivesThe aims of this study were to explore 3 main sources of variability in RFs, investigate their influencing magnitudes and patterns, and identify a subset of robust RFs for further studies.Materials and MethodsA chest phantom with nodules was scanned with different computed tomography (CT) scanners repeatedly with varying acquisition and reconstruction parameters (April-May 2019) to evaluate 3 sources of variability: test-retest, inter-CT, and intra-CT protocol variability. The robustness of the RFs was measured using the concordance correlation coefficient, dynamic range, and intraclass correlation coefficient (ICC). The influencing magnitudes and patterns were analyzed using the Friedman test and Spearman rank correlation coefficient. Stable and informative RFs were selected, and their redundancy was eliminated using hierarchical clustering. Clinical validation was also performed to verify the clinical effectiveness and potential enhancement of the generalizability of radiomics research.ResultsA total of 1295 RFs that showed all 3 sources of variability were included. The reconstruction kernel and the iteration level showed the greatest (ICC, 0.35 ± 0.31) and the least (ICC, 0.63 ± 0.27) influence on magnitudes. The different sources of variability showed relatively consistent patterns of influence (false discovery rate <0.001). Finally, we obtained a subset of 19 stable, informative, and nonredundant RFs under all 3 sources of variability. These RFs exhibited clinical effectiveness and showed better prediction performance than unstable RFs in the validation dataset (P = 0.017, Delong test).ConclusionsThe stability of RFs was affected to different degrees by test-retest and differences in CT manufacturers and models and CT acquisition and reconstruction parameters, but the influences of these factors showed relatively consistent patterns. We also obtained a subset of 19 stable, informative, and nonredundant RFs that should be preferably used to enhance the generalizability of further radiomics research.

A systematic review and quality of reporting checklist for repeatability and reproducibility of radiomic features.

Although quantitative image biomarkers (radiomics) show promising value for cancer diagnosis, prognosis, and treatment assessment, these biomarkers still lack reproducibility. In this systematic review, we aimed to assess the progress in radiomics reproducibility and repeatability in the recent years. Four hundred fifty-one abstracts were retrieved according to the original PubMed search pattern with the publication dates ranging from 2017/05/01 to 2020/12/01. Each abstract including the keywords was independently screened by four observers. Forty-two full-text articles were selected for further analysis. Patient population data, radiomic feature classes, feature extraction software, image preprocessing, and reproducibility results were extracted from each article. To support the community with a standardized reporting strategy, we propose a specific reporting checklist to evaluate the feasibility to reproduce each study. Many studies continue to under-report essential reproducibility information: all but one clinical and all but two phantom studies missed to report at least one important item reporting image acquisition. The studies included in this review indicate that all radiomic features are sensitive to image acquisition, reconstruction, tumor segmentation, and interpolation. However, the amount of sensitivity is feature dependent, for instance, textural features were, in general, less robust than statistical features. Radiomics repeatability, reproducibility, and reporting quality can substantially be improved regarding feature extraction software and settings, image preprocessing and acquisition, cutoff values for stable feature selection. Our proposed radiomics reporting checklist can serve to simplify and improve the reporting and, eventually, guarantee the possibility to fully replicate and validate radiomic studies.

Virtual Monoenergetic Images of Dual-Energy CT-Impact on Repeatability, Reproducibility, and Classification in Radiomics.

Simple SummaryVirtual monoenergetic images from dual-energy CT are incrementally used in routine clinical practice. Thus, radiomic analysis will be more often performed on these images in the future. This study characterized the test–retest repeatability and reproducibility of radiomic features from virtual monoenergetic images and their impact on machine-learning-based lesion classification. The results of this study provide a basis to improve radiomic analyses and identify the role of feature stability in classification tasks when using virtual monoenergetic imaging with different scan or reconstruction parameters in multicenter clinical studies.The purpose of this study was to (i) evaluate the test–retest repeatability and reproducibility of radiomic features in virtual monoenergetic images (VMI) from dual-energy CT (DECT) depending on VMI energy (40, 50, 75, 120, 190 keV), radiation dose (5 and 15 mGy), and DECT approach (dual-source and split-filter DECT) in a phantom (ex vivo), and (ii) to assess the impact of VMI energy and feature repeatability on machine-learning-based classification in vivo in 72 patients with 72 hypodense liver lesions. Feature repeatability and reproducibility were determined by concordance–correlation–coefficient (CCC) and dynamic range (DR) ≥0.9. Test–retest repeatability was high within the same VMI energies and scan conditions (percentage of repeatable features ranging from 74% for SFDE mode at 40 keV and 15 mGy to 86% for DSDE at 190 keV and 15 mGy), while reproducibility varied substantially across different VMI energies and DECTs (percentage of reproducible features ranging from 32.8% for SFDE at 5 mGy comparing 40 with 190 keV to 99.2% for DSDE at 15 mGy comparing 40 with 50 keV). No major differences were observed between the two radiation doses (<10%) in all pair-wise comparisons. In vivo, machine learning classification using penalized regression and random forests resulted in the best discrimination of hemangiomas and metastases at low-energy VMI (40 keV), and for cysts at high-energy VMI (120 keV). Feature selection based on feature repeatability did not improve classification performance. Our results demonstrate the high repeatability of radiomics features when keeping scan and reconstruction conditions constant. Reproducibility diminished when using different VMI energies or DECT approaches. The choice of optimal VMI energy improved lesion classification in vivo and should hence be adapted to the specific task.

Radiomics feature stability of open-source software evaluated on apparent diffusion coefficient maps in head and neck cancer

Radiomics is a promising technique for discovering image based biomarkers of therapy response in cancer. Reproducibility of radiomics features is a known issue that is addressed by the image biomarker standardisation initiative (IBSI), but it remains challenging to interpret previously published radiomics signatures. This study investigates the reproducibility of radiomics features calculated with two widely used radiomics software packages (IBEX, MaZda) in comparison to an IBSI compliant software package (PyRadiomics). Intensity histogram, shape and textural features were extracted from 334 diffusion weighted magnetic resonance images of 59 head and neck cancer (HNC) patients from the PREDICT-HN observational radiotherapy study. Based on name and linear correlation, PyRadiomics shares 83 features with IBEX and 49 features with MaZda, a sub-set of well correlated features are considered reproducible (IBEX: 15 features, MaZda: 18 features). We explore the impact of including non-reproducible radiomics features in a HNC radiotherapy response model. It is possible to classify equivalent patient groups using radiomic features from either software, but only when restricting the model to reliable features using a correlation threshold method. This is relevant for clinical biomarker validation trials as it provides a framework to assess the reproducibility of reported radiomic signatures from existing trials.

Stability and Reproducibility of Radiomic Features Based Various Segmentation Technique on MR Images of Hepatocellular Carcinoma (HCC).

Hepatocellular carcinoma (HCC) is considered as a complex liver disease and ranked as the eighth-highest mortality rate with a prevalence of 2.4% in Malaysia. Magnetic resonance imaging (MRI) has been acknowledged for its advantages, a gold technique for diagnosing HCC, and yet the false-negative diagnosis from the examinations is inevitable. In this study, 30 MR images from patients diagnosed with HCC is used to evaluate the robustness of semi-automatic segmentation using the flood fill algorithm for quantitative features extraction. The relevant features were extracted from the segmented MR images of HCC. Four types of features extraction were used for this study, which are tumour intensity, shape feature, textural feature and wavelet feature. A total of 662 radiomic features were extracted from manual and semi-automatic segmentation and compared using intra-class relation coefficient (ICC). Radiomic features extracted using semi-automatic segmentation utilized flood filling algorithm from 3D-slicer had significantly higher reproducibility (average ICC = 0.952 ± 0.009, p < 0.05) compared with features extracted from manual segmentation (average ICC = 0.897 ± 0.011, p > 0.05). Moreover, features extracted from semi-automatic segmentation were more robust compared to manual segmentation. This study shows that semi-automatic segmentation from 3D-Slicer is a better alternative to the manual segmentation, as they can produce more robust and reproducible radiomic features.

Generative models improve radiomics reproducibility in low dose CTs: a simulation study

Radiomics is an active area of research in medical image analysis, however poor reproducibility of radiomics has hampered its application in clinical practice. This issue is especially prominent when radiomic features are calculated from noisy images, such as low dose computed tomography (CT) scans. In this article, we investigate the possibility of improving the reproducibility of radiomic features calculated on noisy CTs by using generative models for denoising. Our work concerns two types of generative models—encoder–decoder network (EDN) and conditional generative adversarial network (CGAN). We then compared their performance against a more traditional ‘non-local means’ denoising algorithm. We added noise to sinograms of full dose CTs to mimic low dose CTs with two levels of noise: low-noise CT and high-noise CT. Models were trained on high-noise CTs and used to denoise low-noise CTs without re-training. We tested the performance of our model in real data, using a dataset of same-day repeated low dose CTs in order to assess the reproducibility of radiomic features in denoised images. EDN and the CGAN achieved similar improvements on the concordance correlation coefficients (CCC) of radiomic features for low-noise images from 0.87 [95%CI, (0.833, 0.901)] to 0.92 [95%CI, (0.909, 0.935)] and for high-noise images from 0.68 [95%CI, (0.617, 0.745)] to 0.92 [95%CI, (0.909, 0.936)], respectively. The EDN and the CGAN improved the test-retest reliability of radiomic features (mean CCC increased from 0.89 [95%CI, (0.881, 0.914)] to 0.94 [95%CI, (0.927, 0.951)]) based on real low dose CTs. These results show that denoising using EDN and CGANs could be used to improve the reproducibility of radiomic features calculated from noisy CTs. Moreover, images at different noise levels can be denoised to improve the reproducibility using the above models without need for re-training, provided the noise intensity is not excessively greater that of the high-noise CTs. To the authors’ knowledge, this is the first effort to improve the reproducibility of radiomic features calculated on low dose CT scans by applying generative models.

MRI Radiomic Features of Peritumoral Edema May Predict the Recurrence Sites of Glioblastoma Multiforme

Background: The purpose of this study was to investigate whether there is radiological heterogeneity within peritumoral edema and identify the reproducible radiomic features predictive of the sites of recurrence of glioblastoma multiforme (GBM), which may be of value to optimize patients’ management. Methods: We retrospectively evaluated the clinical information and MR images (T1-weighted and fluid-attenuated inversion recovery sequences) of 22 patients with histologically diagnosed GBM. Kaplan-Meier’s method was used for survival analysis. Peritumoral edema regions were manually segmented by an expert into recurrence and non-recurrence subregions. A set of 94 radiomic features were obtained from both subregions using a three-dimensional slicer. Features with significant differences between the two subregions were identified using the paired t test. Subsequently, the data of two patients from The Cancer Genome Atlas (TCGA) database were used to evaluate the clinical value of the selected features. Findings: Ten features with significant differences between the recurrence and non-recurrence subregions were identified and verified on two individual patients from TCGA database with pathologically confirmed diagnosis of GBM. Interpretation: Our results suggested that heterogeneity does exist in peritumoral edema, indicating that the radiomic features of peritumoral edema from routine MR images can be utilized to predict the sites of GBM recurrence. Our findings may further guide the surgical treatment strategy for GBM. Funding Information: This work was supported by grants from the Natural Science Fund of Guangdong Province (No. 2017A030313597), “Climbing Program” Special Fund of Guangdong Province (No. pdjh2019b0100, No. pdjh2020b0112) and Southern Medical University (No. LX2016N006, No. KJ20161102, No.201912121004S, No.201912121013，No. S202012121088, No. X202012121354). Declaration of Interests: The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article. Ethics Approval Statement: Ethics committee approval and patient consent were obtained.

Impact of Preprocessing and Harmonization Methods on the Removal of Scanner Effects in Brain MRI Radiomic Features.

Simple SummaryAs a rapid-development research field, radiomics-based analysis has been applied to many clinical problems. However, the reproducibility of the radiomics studies remain challenging especially when data suffers from scanner effects, a kind of non-biological variations introduced by different image acquiring settings. This study aims to investigate how the image preprocessing methods (N4 bias field correction and image resampling) and the harmonization methods (intensity normalization methods working on images and ComBat method working on radiomic features) help to remove the scanner effects and improve the radiomics reproducibility in brain MRI radiomics.In brain MRI radiomics studies, the non-biological variations introduced by different image acquisition settings, namely scanner effects, affect the reliability and reproducibility of the radiomics results. This paper assesses how the preprocessing methods (including N4 bias field correction and image resampling) and the harmonization methods (either the six intensity normalization methods working on brain MRI images or the ComBat method working on radiomic features) help to remove the scanner effects and improve the radiomic feature reproducibility in brain MRI radiomics. The analyses were based on in vitro datasets (homogeneous and heterogeneous phantom data) and in vivo datasets (brain MRI images collected from healthy volunteers and clinical patients with brain tumors). The results show that the ComBat method is essential and vital to remove scanner effects in brain MRI radiomic studies. Moreover, the intensity normalization methods, while not able to remove scanner effects at the radiomic feature level, still yield more comparable MRI images and improve the robustness of the harmonized features to the choice among ComBat implementations.

CT and MRI radiomics of bone and soft-tissue sarcomas: a systematic review of reproducibility and validation strategies

BackgroundFeature reproducibility and model validation are two main challenges of radiomics. This study aims to systematically review radiomic feature reproducibility and predictive model validation strategies in studies dealing with CT and MRI radiomics of bone and soft-tissue sarcomas. The ultimate goal is to promote achieving a consensus on these aspects in radiomic workflows and facilitate clinical transferability.ResultsOut of 278 identified papers, forty-nine papers published between 2008 and 2020 were included. They dealt with radiomics of bone (n = 12) or soft-tissue (n = 37) tumors. Eighteen (37%) studies included a feature reproducibility analysis. Inter-/intra-reader segmentation variability was the theme of reproducibility analysis in 16 (33%) investigations, outnumbering the analyses focused on image acquisition or post-processing (n = 2, 4%). The intraclass correlation coefficient was the most commonly used statistical method to assess reproducibility, which ranged from 0.6 and 0.9. At least one machine learning validation technique was used for model development in 25 (51%) papers, and K-fold cross-validation was the most commonly employed. A clinical validation of the model was reported in 19 (39%) papers. It was performed using a separate dataset from the primary institution (i.e., internal validation) in 14 (29%) studies and an independent dataset related to different scanners or from another institution (i.e., independent validation) in 5 (10%) studies.ConclusionsThe issues of radiomic feature reproducibility and model validation varied largely among the studies dealing with musculoskeletal sarcomas and should be addressed in future investigations to bring the field of radiomics from a preclinical research area to the clinical stage.

Effect of adaptive statistical iterative reconstruction-V (ASiR-V) levels on ultra-low-dose CT radiomics quantification in pulmonary nodules.

The weightings of iterative reconstruction algorithm can affect CT radiomic quantification. But, the effect of ASiR-V levels on the reproducibility of CT radiomic features between ultra-low-dose computed tomography (ULDCT) and low-dose computed tomography (LDCT) is still unknown. The purpose of study is to investigate whether adaptive statistical iterative reconstruction-V (ASiR-V) levels affect radiomic feature quantification using ULDCT and to assess the reproducibility of radiomic features between ULDCT and LDCT. Sixty-three patients with pulmonary nodules underwent LDCT (0.70±0.16 mSv) and ULDCT (0.15±0.02 mSv). LDCT was reconstructed with ASiR-V 50%, and ULDCT with ASiR-V 50%, 70%, and 90%. Radiomics analysis was applied, and 107 features were extracted. The concordance correlation coefficient (CCC) was calculated to describe agreement among ULDCTs and between ULDCT and LDCT for each feature. The proportion of features with CCC >0.9 among ULDCTs and between ULDCT and LDCT, and the mean CCC for all features between ULDCT and LDCT were also compared. Sixty-three solid nodules (SNs) and 48 pure ground-glass nodules (pGGNs) were analyzed. There was no difference for the proportion of features in SNs among ULDCTs and between ULDCT and LDCT (P>0.05). The proportion of features in pGGNs were highest for ULDCT70% vs. 90% (78.5%) and ULDCT90% vs. LDCT50% (50.5%). In SNs, the mean CCC for ULDCT90% vs. LDCT50% was 0.67±0.26, not different with that for ULDCT50% vs. LDCT50% (0.68±0.24) and ULDCT70% vs. LDCT50% (0.64±0.21) (P>0.05). In pGGNs, the mean CCC for ULDCT90% vs. LDCT50% was 0.79±0.19, higher than that for ULDCT50% vs. LDCT50% (0.61±0.28) and ULDCT70% vs. LDCT50% (0.76±0.24) (P<0.05). ASiR-V levels significantly affected ULDCT radiomic feature quantification in pulmonary nodules, with stronger effects in pGGNs than in SNs. The reproducibility of radiomic features was highest between ULDCT90% and LDCT50%.

Radiomics complements clinical, radiological, and technical features to assess local control of colorectal cancer lung metastases treated with radiofrequency ablation.

Radiofrequency ablation (RFA) of lung metastases of colorectal origin can improve patient survival and quality of life. Our aim was to identify pre- and per-RFA features predicting local control of lung metastases following RFA. This case-control single-center retrospective study included 119 lung metastases treated with RFA in 48 patients (median age: 60 years). Clinical, technical, and radiological data before and on early CT scan (at 48 h) were retrieved. After CT scan preprocessing, 64 radiomics features were extracted from pre-RFA and early control CT scans. Log-rank tests were used to detect categorical variables correlating with post-RFA local tumor progression-free survival (LTPFS). Radiomics prognostic scores (RPS) were developed on reproducible radiomics features using Monte-Carlo cross-validated LASSO Cox regressions. Twenty-six of 119 (21.8%) nodules demonstrated local progression (median delay: 11.2 months). In univariate analysis, four non-radiomics variables correlated with post-RFA-LTPFS: nodule size (> 15 mm, p < 0.001), chosen electrode (with difference between covered array and nodule diameter < 20 mm or non-expandable electrode, p = 0.03), per-RFA intra-alveolar hemorrhage (IAH, p = 0.002), and nodule location into the ablation zone (not seen or in contact with borders, p = 0.005). The highest prognostic performance was reached with the multivariate model including a RPS built on 4 radiomics features from pre-RFA and early revaluation CT scans (cross-validated concordance index= 0.74) in which this RPS remained an independent predictor (cross-validated HR = 3.49, 95% confidence interval = [1.76 - 6.96]). Technical, radiological, and radiomics features of the lung metastases before RFA and of the ablation zone at 48 h can help discriminate nodules at risk of local progression that could benefit from complementary local procedure. • The highest prognostic performance to predict post-RFA LTPFS was reached with a parsimonious model including a radiomics score built with 4 radiomics features. • Nodule size, difference between electrode diameter, use of non-expandable electrode, per-RFA hemorrhage, and a tumor not seen or in contact with the ablation zone borders at 48-h CT were correlated with post-RFA LTPFS.