Hand-crafted Radiomics Features Research Articles

The Role of Radiomics in Fibrosis Crohn's Disease: A Review.

Inflammatory bowel disease (IBD) is a global health concern that has been on the rise in recent years. In addition, imaging is the established method of care for detecting, diagnosing, planning treatment, and monitoring the progression of IBD. While conventional imaging techniques are limited in their ability to provide comprehensive information, cross-sectional imaging plays a crucial role in the clinical management of IBD. However, accurately characterizing, detecting, and monitoring fibrosis in Crohn's disease remains a challenging task for clinicians. Recent advances in artificial intelligence technology, machine learning, computational power, and radiomic emergence have enabled the automated evaluation of medical images to generate prognostic biomarkers and quantitative diagnostics. Radiomics analysis can be achieved via deep learning algorithms or by extracting handcrafted radiomics features. As radiomic features capture pathophysiological and biological data, these quantitative radiomic features have been shown to offer accurate and rapid non-invasive tools for IBD diagnostics, treatment response monitoring, and prognosis. For these reasons, the present review aims to provide a comprehensive review of the emerging radiomics methods in intestinal fibrosis research that are highlighted and discussed in terms of challenges and advantages.

A multi-stage fusion framework to classify breast lesions using deep learning and radiomics features computed from four-view mammograms.

Developing computer aided diagnosis (CAD) schemes of mammograms to classify between malignant and benign breast lesions has attracted a lot of research attention over the last several decades. However, unlike radiologists who make diagnostic decisions based on the fusion of image features extracted from multi-view mammograms, most CAD schemes are single-view-based schemes, which limit CAD performance and clinical utility. This study aims to develop and test a novel CAD framework that optimally fuses information extracted from ipsilateral views of bilateral mammograms using both deep transfer learning (DTL) and radiomics feature extraction methods. An image dataset containing 353 benign and 611 malignant cases is assembled. Each case contains four images: the craniocaudal (CC) and mediolateral oblique (MLO) view of the left and right breast. First, we extract four matching regions of interest (ROIs) from images that surround centers of two suspicious lesion regions seen in CC and MLO views, as well as matching ROIs in the contralateral breasts. Next, the handcrafted radiomics (HCRs) features and VGG16 model-generated automated features are extracted from each ROI resulting in eight feature vectors. Then, after reducing feature dimensionality and quantifying the bilateral and ipsilateral asymmetry of four ROIs to yield four new feature vectors, we test four fusion methods to build three support vector machine (SVM) classifiers by an optimal fusion of asymmetrical image features extracted from four view images. Using a 10-fold cross-validation method, results show that a SVM classifier trained using an optimal fusion of four view images yields the highest classification performance (AUC=0.876±0.031), which significantly outperforms SVM classifiers trained using one projection view alone, AUC=0.817±0.026 and 0.792±0.026 for the CC and MLO view of bilateral mammograms, respectively (p<0.001). The study demonstrates that the shift from single-view CAD to four-view CAD and the inclusion of both DTL and radiomics features significantly increases CAD performance in distinguishing between malignant and benign breast lesions.

Differentiation of acute and chronic vertebral compression fractures using conventional CT based on deep transfer learning features and hand-crafted radiomics features

BackgroundWe evaluated the diagnostic efficacy of deep learning radiomics (DLR) and hand-crafted radiomics (HCR) features in differentiating acute and chronic vertebral compression fractures (VCFs).MethodsA total of 365 patients with VCFs were retrospectively analysed based on their computed tomography (CT) scan data. All patients completed MRI examination within 2 weeks. There were 315 acute VCFs and 205 chronic VCFs. Deep transfer learning (DTL) features and HCR features were extracted from CT images of patients with VCFs using DLR and traditional radiomics, respectively, and feature fusion was performed to establish the least absolute shrinkage and selection operator. The MRI display of vertebral bone marrow oedema was used as the gold standard for acute VCF, and the model performance was evaluated using the receiver operating characteristic (ROC).To separately evaluate the effectiveness of DLR, traditional radiomics and feature fusion in the differential diagnosis of acute and chronic VCFs, we constructed a nomogram based on the clinical baseline data to visualize the classification evaluation. The predictive power of each model was compared using the Delong test, and the clinical value of the nomogram was evaluated using decision curve analysis (DCA).ResultsFifty DTL features were obtained from DLR, 41 HCR features were obtained from traditional radiomics, and 77 features fusion were obtained after feature screening and fusion of the two. The area under the curve (AUC) of the DLR model in the training cohort and test cohort were 0.992 (95% confidence interval (CI), 0.983-0.999) and 0.871 (95% CI, 0.805-0.938), respectively. While the AUCs of the conventional radiomics model in the training cohort and test cohort were 0.973 (95% CI, 0.955-0.990) and 0.854 (95% CI, 0.773-0.934), respectively. The AUCs of the features fusion model in the training cohort and test cohort were 0.997 (95% CI, 0.994-0.999) and 0.915 (95% CI, 0.855-0.974), respectively. The AUCs of nomogram constructed by the features fusion in combination with clinical baseline data were 0.998 (95% CI, 0.996–0.999) and 0.946 (95% CI, 0.906–0.987) in the training cohort and test cohort, respectively. The Delong test showed that the differences between the features fusion model and the nomogram in the training cohort and the test cohort were not statistically significant (P values were 0.794 and 0.668, respectively), and the differences in the other prediction models in the training cohort and the test cohort were statistically significant (P < 0.05). DCA showed that the nomogram had high clinical value.ConclusionThe features fusion model can be used for the differential diagnosis of acute and chronic VCFs, and its differential diagnosis ability is improved when compared with that when either radiomics is used alone. At the same time, the nomogram has a high predictive value for acute and chronic VCFs and can be a potential decision-making tool to assist clinicians, especially when a patient is unable to undergo spinal MRI examination.

Radiomics-Guided Global-Local Transformer for Weakly Supervised Pathology Localization in Chest X-Rays.

Before the recent success of deep learning methods for automated medical image analysis, practitioners used handcrafted radiomic features to quantitatively describe local patches of medical images. However, extracting discriminative radiomic features relies on accurate pathology localization, which is difficult to acquire in real-world settings. Despite advances in disease classification and localization from chest X-rays, many approaches fail to incorporate clinically-informed domainspecific radiomic features. For these reasons, we propose a Radiomics-Guided Transformer (RGT) that fuses global image information with local radiomics-guided auxiliary information to provide accurate cardiopulmonary pathology localization and classification without any bounding box annotations. RGT consists of an image Transformer branch, a radiomics Transformer branch, and fusion layers that aggregate image and radiomics information. Using the learned self-attention of its image branch, RGT extracts a bounding box for which to compute radiomic features, which are further processed by the radiomics branch; learned image and radiomic features are then fused and mutually interact via cross-attention layers. Thus, RGT utilizes a novel end-to-end feedback loop that can bootstrap accurate pathology localization only using image-level disease labels. Experiments on the NIH ChestXRay dataset demonstrate that RGT outperforms prior works in weakly supervised disease localization (by an average margin of 3.6% over various intersection-over-union thresholds) and classification (by 1.1% in average area under the receiver operating characteristic curve). We publicly release our codes and pre-trained models at https://github.com/VITAGroup/chext.

Radiogenomic association of deep MR imaging features with genomic profiles and clinical characteristics in breast cancer

BackgroundIt has been believed that traditional handcrafted radiomic features extracted from magnetic resonance imaging (MRI) of tumors are normally shallow and low-ordered. Recent advancement in deep learning technology shows that the high-order deep radiomic features extracted automatically from tumor images can capture tumor heterogeneity in a more efficient way. We hypothesize that MRI-based deep radiomic phenotypes have significant associations with molecular profiles of breast cancer tumors. We aim to identify deep radiomic features (DRFs) from MRI, evaluate their significance in predicting breast cancer (BC) clinical characteristics and explore their associations with multi-level genomic factors.MethodsA denoising autoencoder was built to retrospectively extract 4,096 DRFs from 110 BC patients’ MRI. Visualization and clustering were applied to these DRFs. Linear Mixed Effect models were used to test their associations with multi-level genomic features (GFs) (risk genes, gene signatures, and biological pathway activities) extracted from the same patients’ mRNA expression profile. A Least Absolute Shrinkage and Selection Operator model was used to identify the most predictive DRFs for each clinical characteristic (tumor size (T), lymph node metastasis (N), estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2) status).ResultsThirty-six conventional radiomic features (CRFs) for 87 of the 110 BC patients provided by a previous study were used for comparison. More than 1,000 DRFs were associated with the risk genes, gene signatures, and biological pathways activities (adjusted P-value < 0.05). DRFs produced better performance in predicting T, N, ER, PR, and HER2 status (AUC > 0.9) using DRFs. These DRFs showed significant powers of stratifying patients, linking to relevant biological and clinical characteristics. As a contrast, only eight risk genes were associated with CRFs. The RFs performed worse in predicting clinical characteristics than DRFs.ConclusionsThe deep learning-based auto MRI features perform better in predicting BC clinical characteristics, which are more significantly associated with GFs than traditional semi-auto MRI features. Our radiogenomic approach for identifying MRI-based imaging signatures may pave potential pathways for the discovery of genetic mechanisms regulating specific tumor phenotypes and may enable a more rapid innovation of novel imaging modalities, hence accelerating their translation to personalized medicine.

Brain metastasis magnetic resonance imaging-based deep learning for predicting epidermal growth factor receptor (EGFR) mutation and subtypes in metastatic non-small cell lung cancer.

The preoperative identification of epidermal growth factor receptor (EGFR) mutations and subtypes based on magnetic resonance imaging (MRI) of brain metastases (BM) is necessary to facilitate individualized therapy. This study aimed to develop a deep learning model to preoperatively detect EGFR mutations and identify the location of EGFR mutations in patients with non-small cell lung cancer (NSCLC) and BM. We included 160 and 72 patients who underwent contrast-enhanced T1-weighted (T1w-CE) and T2-weighted (T2W) MRI at Liaoning Cancer Hospital and Institute (center 1) and Shengjing Hospital of China Medical University (center 2) to form a training cohort and an external validation cohort, respectively. A multiscale feature fusion network (MSF-Net) was developed by adaptively integrating features based on different stages of residual network (ResNet) 50 and by introducing channel and spatial attention modules. The external validation set from center 2 was used to assess the performance of MSF-Net and to compare it with that of handcrafted radiomics features. Receiver operating characteristic (ROC) curves, accuracy, precision, recall, and F1-score were used to evaluate the effectiveness of the models. Gradient-weighted class activation mapping (Grad-CAM) was used to demonstrate the attention of the MSF-Net model. The developed MSF-Net generated a better diagnostic performance than did the handcrafted radiomics in terms of the microaveraged area under the curve (AUC) (MSF-Net: 0.91; radiomics: 0.80) and macroaveraged AUC (MSF-Net: 0.90; radiomics: 0.81) for predicting EGFR mutations and subtypes. This study provides an end-to-end and noninvasive imaging tool for the preoperative prediction of EGFR mutation status and subtypes based on BM, which may be helpful for facilitating individualized clinical treatment plans.

NIMG-73. PREDICTING PERITUMORAL GLIOBLASTOMA INFILTRATION USING QUANTITATIVE MAGNETIC RESONANCE FINGERPRINTING RADIOMICS

Abstract PURPOSE Standard clinical magnetic resonance (MR) imaging of glioblastoma (GB) cannot identify malignant infiltration into the peritumoral non-enhancing region. This work implements quantitative MR fingerprinting (MRF) based radiomics to predict infiltrative regions of GB. METHODS Pre-operative MRF T1 and T2 maps along with multiparametric MR (mpMR) images (T1w, T2w, T1w-Gd, FLAIR, and ADC) from GB patients (n = 10) were analyzed. All subjects had histologically confirmed sites of non-enhancing peritumoral infiltration as identified by intra-operative 5-ALA fluorescence-guided tissue resection. These locations (n = 40) were manually annotated on pre-operative FLAIR by a board-certified neuroradiologist and labeled as “infiltration” (INF). For each patient, another FLAIR region ( &amp;gt; 3 cm from the enhancing tumor margin) was identified and labeled as “edema” (ED). Following image co-registration, 693 handcrafted radiomic voxel-based features were extracted from confirmed infiltration (INF) and edema (ED) voxels using a 3D 5x5x5 voxel sliding kernel. Feature selection was performed using the minimum redundancy maximal relevance (MRMR) algorithm, then INF and ED voxel features were used to train two cross-validated binary support vector machine (SVM) models for voxel-wise infiltration prediction: 1) MRF only model and 2) combined MRF and mpMRI model. Following MRMR selection, the combined model included features from T1, T1w, T2w, T1w-Gd, FLAIR, and ADC images. Model performance was evaluated using one withheld subject with multiple known infiltration sites. Balanced test accuracy and receiver operating characteristic (ROC) curve analysis was used to evaluate classification performance. RESULTS The MRF only and combined model achieved balanced test accuracies of 77.87% and 73.21% and AUCs of 0.91 and 0.97, respectively. CONCLUSIONS This study demonstrates that MRF-based radiomic features can predict GB tumor infiltration with high accuracy and provide complementary value to standard clinical MRI features. Our results indicate the high potential of employing MRF to guide personalized GB treatment strategies.

Comparison of Traditional Radiomics, Deep Learning Radiomics and Fusion Methods for Axillary Lymph Node Metastasis Prediction in Breast Cancer

Accurate identification of axillary lymph node (ALN) status in breast cancer patients is important for determining treatment options and avoiding axillary overtreatments. Our study aims to comprehensively compare the performance of the traditional radiomics model, deep learning radiomics model, and the fusion models in evaluating breast cancer ALN status based on dynamic contrast-enhanced-magnetic resonance imaging (DCE-MRI) images. The handcrafted radiomics features and deep features were extracted from 3062 DCE-MRI images. The feature selection was performed by applying mutual information and feature recursive elimination algorithms. The traditional radiomics model and deep learning radiomics model were built using the optimal features and machine learning classifiers, respectively. The fusion models for distinguishing axillary lymph node status were constructed using two fusion strategies. The performance of the models with MRI-reported lymphadenopathy or suspicious nodes to evaluate axillary lymph node status was also compared. The decision fusion model, with the integration of the radiomics features and deep learning features at the decision level, achieved an area under the curve (AUC) of 0.91 (95% confidence interval (CI): 0.879-0.937), which was higher than that of the traditional radiomics model and deep learning radiomics model. The results of the decision fusion model with clinical characteristic yielded an AUC of 0.93 (95% CI: 0.899-0.951), which was also superior to other models incorporating clinical characteristic. This study demonstrates the effectiveness of the fusion models for predicting axillary lymph node metastasis in breast cancer.

Deterministic small-scale undulations of image-based risk predictions from the deep learning of lung tumors in motion.

Deep learning (DL) models that use medical images to predict clinical outcomes are poised for clinical translation. For tumors that reside in organs that move, however, the impact of motion (i.e., degenerated object appearance or blur) on DL model accuracy remains unclear. We examine the impact of tumor motion on an image-based DL framework that predicts local failure risk after lung stereotactic body radiotherapy (SBRT). We input pre-therapy free breathing (FB) computed tomography (CT) images from 849 patients treated with lung SBRT into a multitask deep neural network to generate an image fingerprint signature (or DL score) that predicts time-to-event local failure outcomes. The network includes a convolutional neural network encoder for extracting imaging features and building a task-specific fingerprint, a decoder for estimating handcrafted radiomic features, and a task-specific network for generating image signature for radiotherapy outcome prediction. The impact of tumor motion on the DL scores was then examined for a holdout set of 468 images from 39 patients comprising: (1) FB CT, (2) four-dimensional (4D) CT, and (3) maximum-intensity projection (MIP) images. Tumor motion was estimated using a 3D vector of the maximum distance traveled, and its association with DL score variance was assessed by linear regression. The variance and amplitude in 4D CT image-derived DL scores were associated with tumor motion (R2 =0.48 and 0.46, respectively). Specifically, DL score variance was deterministic and represented by sinusoidal undulations in phase with the respiratory cycle. DL scores, but not tumor volumes, peaked near end-exhalation. The mean of the scores derived from 4D CT images and the score obtained from FB CT images were highly associated (Pearson r=0.99). MIP-derived DL scores were significantly higher than 4D- or FB-derived risk scores (p<0.0001). An image-based DL risk score derived from a series of 4D CT images varies in a deterministic, sinusoidal trajectory in a phase with the respiratory cycle. These results indicate that DL models of tumors in motion can be robust to fluctuations in object appearance due to movement and can guide standardization processes in the clinical translation of DL models for patients with lung cancer.

Radiomics Analysis of Lymph Nodes with Esophageal Squamous Cell Carcinoma Based on Deep Learning.

Purpose To assess the role of multiple radiomic features of lymph nodes in the preoperative prediction of lymph node metastasis (LNM) in patients with esophageal squamous cell carcinoma (ESCC). Methods Three hundred eight patients with pathologically confirmed ESCC were retrospectively enrolled (training cohort, n = 216; test cohort, n = 92). We extracted 207 handcrafted radiomic features and 1000 deep radiomic features of lymph nodes from their computed tomography (CT) images. The t-test and least absolute shrinkage and selection operator (LASSO) were used to reduce the dimensions and select key features. Handcrafted radiomics, deep radiomics, and clinical features were combined to construct models. Models I (handcrafted radiomic features), II (Model I plus deep radiomic features), and III (Model II plus clinical features) were built using three machine learning methods: support vector machine (SVM), adaptive boosting (AdaBoost), and random forest (RF). The best model was compared with the results of two radiologists, and its performance was evaluated in terms of sensitivity, specificity, accuracy, area under the curve (AUC), and receiver operating characteristic (ROC) curve analysis. Results No significant differences were observed between cohorts. Ten handcrafted and 12 deep radiomic features were selected from the extracted features (p < 0.05). Model III could discriminate between patients with and without LNM better than the diagnostic results of the two radiologists. Conclusion The combination of handcrafted radiomic features, deep radiomic features, and clinical features could be used clinically to assess lymph node status in patients with ESCC.

1139P A CT-based radiomic readout on the differential diagnosis of pneumonitis in stage IV non-small cell lung cancer patients treated with immune checkpoint inhibitors

About 4-10% patients who receive immunotherapy are subjected to Immunotherapy-Induced Pneumonitis (IIP) which is a lethal complication that requires immediate interruption of immunotherapy as well as corticosteroid administration. Less than 10% of the patients also develop other types of pneumonitis (OP), (e.g. infection or radiation-induced). Making the differential diagnosis of pneumonitis (IIP vs. OP) is a challenging task due to similar radiological patterns and low incidence rates. This study was performed to develop and evaluate a prediction model using handcrafted radiomic features and their associated clinical metadata to estimate the risk of IIP in patients with stage IV Non-Small Cell Lung Cancer (NSCLC) who developed pneumonitis after immunotherapy.

Computer-aided diagnostic models to classify lymph node metastasis and lymphoma involvement in enlarged cervical lymph nodes using PET/CT.

It is a clinical problem to identify histological component in enlarged cervical lymph nodes, particularly in differentiation between lymph node metastasis and lymphoma involvement. To construct two kinds of deep learning (DL)-based computer-aided diagnosis (CAD) systems including DL-convolutional neural networks (DL-CNN) and DL-machine learning for pathological diagnosis of cervical lymph nodes by positron emission tomography (PET)/computed tomography (CT) images. We collected CT, PET, and PET/CT images series from 165 patients with enlarged cervical lymph nodes receiving examinations from January 2014 to June 2018. Six CNNs pretrained on ImageNet as DL architectures were used for two kinds of DL-based CAD models, including DL-CNN and DL-machine learning models. The DL-CNN models were constructed via transfer learning for classification of lymphomatous and metastatic lymph nodes. The DL-machine learning models were developed by DL-based features extractors and support vector machine (SVM) classifier. As for DL-SVM models, we also evaluate the effect of handcrafted radiomics features in combination of DL-based features. The DL-CNN model with ResNet50 architecture on PET/CT images had the best diagnostic performance among all six algorithms with an area under the receiver operating characteristic curve (AUC) of 0.845 and accuracy of 78.13% in the testing cohort. The DL-SVM model on ResNet50 extractor showed great performance for the testing cohort with an AUC of 0.901, accuracy of 86.96%, sensitivity of 76.09%, and specificity of 94.20%. The combination of DL-based and handcrafted features yielded the improvement of diagnostic performance. Our DL-based CAD systems on PET/CT images were developed for classifying metastatic and lymphomatous involvement with favorable diagnostic performance in enlarged cervical lymph nodes. Further clinical practice of our systems may improve quality of the following therapeutic interventions and optimize patients' outcomes.

A deep learning-based radiomics approach to predict head and neck tumor regression for adaptive radiotherapy

Early regression—the regression in tumor volume during the initial phase of radiotherapy (approximately 2 weeks after treatment initiation)—is a common occurrence during radiotherapy. This rapid radiation-induced tumor regression may alter target coordinates, necessitating adaptive radiotherapy (ART). We developed a deep learning-based radiomics (DLR) approach to predict early head and neck tumor regression and thereby facilitate ART. Primary gross tumor volume (GTVp) was monitored in 96 patients and nodal GTV (GTVn) in 79 patients during treatment. All patients underwent two computed tomography (CT) scans: one before the start of radiotherapy for initial planning and one during radiotherapy for boost planning. Patients were assigned to regression and nonregression groups according to their median tumor regression rate (ΔGTV/treatment day from initial to boost CT scan). We input a GTV image into the convolutional neural network model, which was pretrained using natural image datasets, via transfer learning. The deep features were extracted from the last fully connected layer. To clarify the prognostic power of the deep features, machine learning models were trained. The models then predicted the regression and nonregression of GTVp and GTVn and evaluated the predictive performance by 0.632 + bootstrap area under the curve (AUC). Predictive performance for GTVp regression was highest using the InceptionResNetv2 model (mean AUC = 0.75) and that for GTVn was highest using NASNetLarge (mean AUC = 0.73). Both models outperformed the handcrafted radiomics features (mean AUC = 0.63 for GTVp and 0.61 for GTVn) or clinical factors (0.64 and 0.67, respectively). DLR may facilitate ART for improved radiation side-effects and target coverage.

CT Reconstruction Kernels and the Effect of Pre- and Post-Processing on the Reproducibility of Handcrafted Radiomic Features.

Handcrafted radiomics features (HRFs) are quantitative features extracted from medical images to decode biological information to improve clinical decision making. Despite the potential of the field, limitations have been identified. The most important identified limitation, currently, is the sensitivity of HRF to variations in image acquisition and reconstruction parameters. In this study, we investigated the use of Reconstruction Kernel Normalization (RKN) and ComBat harmonization to improve the reproducibility of HRFs across scans acquired with different reconstruction kernels. A set of phantom scans (n = 28) acquired on five different scanner models was analyzed. HRFs were extracted from the original scans, and scans were harmonized using the RKN method. ComBat harmonization was applied on both sets of HRFs. The reproducibility of HRFs was assessed using the concordance correlation coefficient. The difference in the number of reproducible HRFs in each scenario was assessed using McNemar’s test. The majority of HRFs were found to be sensitive to variations in the reconstruction kernels, and only six HRFs were found to be robust with respect to variations in reconstruction kernels. The use of RKN resulted in a significant increment in the number of reproducible HRFs in 19 out of the 67 investigated scenarios (28.4%), while the ComBat technique resulted in a significant increment in 36 (53.7%) scenarios. The combination of methods resulted in a significant increment in 53 (79.1%) scenarios compared to the HRFs extracted from original images. Since the benefit of applying the harmonization methods depended on the data being harmonized, reproducibility analysis is recommended before performing radiomics analysis. For future radiomics studies incorporating images acquired with similar image acquisition and reconstruction parameters, except for the reconstruction kernels, we recommend the systematic use of the pre- and post-processing approaches (respectively, RKN and ComBat).

MaasPenn Radiomics Reproducibility Score: A Novel Quantitative Measure for Evaluating the Reproducibility of CT-Based Handcrafted Radiomic Features.

Simple SummaryThe reproducibility of handcrafted radiomic features (HRFs) has been reported to be affected by variations in imaging acquisition and reconstruction parameters. However, to date, these effects have not been understood or quantified. In this study, we analyzed a significantly large number of scenarios in an effort to quantify the effects of variations on the reproducibility of HRFs. In addition, we assessed the performance of ComBat harmonization in each of the 31,375 investigated scenarios. We developed a novel score that can be considered the first attempt to objectively assess the number of reproducible HRFs in different scenario. Following further validation, the score could be used to decide on the inclusion of data acquired differently, as well as the assessment of the generalizability of developed radiomic signatures.The reproducibility of handcrafted radiomic features (HRFs) has been reported to be affected by variations in imaging parameters, which significantly affect the generalizability of developed signatures and translation to clinical practice. However, the collective effect of the variations in imaging parameters on the reproducibility of HRFs remains unclear, with no objective measure to assess it in the absence of reproducibility analysis. We assessed these effects of variations in a large number of scenarios and developed the first quantitative score to assess the reproducibility of CT-based HRFs without the need for phantom or reproducibility studies. We further assessed the potential of image resampling and ComBat harmonization for removing these effects. Our findings suggest a need for radiomics-specific harmonization methods. Our developed score should be considered as a first attempt to introduce comprehensive metrics to quantify the reproducibility of CT-based handcrafted radiomic features. More research is warranted to demonstrate its validity in clinical contexts and to further improve it, possibly by the incorporation of more realistic situations, which better reflect real patients’ situations.

A CT-based deep learning radiomics nomogram for predicting the response to neoadjuvant chemotherapy in patients with locally advanced gastric cancer: A multicenter cohort study

SummaryBackgroundAccurate prediction of treatment response to neoadjuvant chemotherapy (NACT) in individual patients with locally advanced gastric cancer (LAGC) is essential for personalized medicine. We aimed to develop and validate a deep learning radiomics nomogram (DLRN) based on pretreatment contrast-enhanced computed tomography (CT) images and clinical features to predict the response to NACT in patients with LAGC.Methods719 patients with LAGC were retrospectively recruited from four Chinese hospitals between Dec 1st, 2014 and Nov 30th, 2020. The training cohort and internal validation cohort (IVC), comprising 243 and 103 patients, respectively, were randomly selected from center I; the external validation cohort1 (EVC1) comprised 207 patients from center II; and EVC2 comprised 166 patients from another two hospitals. Two imaging signatures, reflecting the phenotypes of the deep learning and handcrafted radiomics features, were constructed from the pretreatment portal venous-phase CT images. A four-step procedure, including reproducibility evaluation, the univariable analysis, the LASSO method, and the multivariable logistic regression analysis, was applied for feature selection and signature building. The integrated DLRN was then developed for the added value of the imaging signatures to independent clinicopathological factors for predicting the response to NACT. The prediction performance was assessed with respect to discrimination, calibration, and clinical usefulness. Kaplan-Meier survival curves based on the DLRN were used to estimate the disease-free survival (DFS) in the follow-up cohort (n = 300).FindingsThe DLRN showed satisfactory discrimination of good response to NACT and yielded the areas under the receiver operating curve (AUCs) of 0.829 (95% CI, 0.739–0.920), 0.804 (95% CI, 0.732–0.877), and 0.827 (95% CI, 0.755–0.900) in the internal and two external validation cohorts, respectively, with good calibration in all cohorts (p > 0.05). Furthermore, the DLRN performed significantly better than the clinical model (p < 0.001). Decision curve analysis confirmed that the DLRN was clinically useful. Besides, DLRN was significantly associated with the DFS of patients with LAGC (p < 0.05).InterpretationA deep learning-based radiomics nomogram exhibited a promising performance for predicting therapeutic response and clinical outcomes in patients with LAGC, which could provide valuable information for individualized treatment.

Testing the Ability of Convolutional Neural Networks to Learn Radiomic Features

Background and objectiveRadiomics and deep learning have emerged as two distinct approaches to medical image analysis. However, their relative expressive power remains largely unknown. Theoretically, hand-crafted radiomic features represent a mere subset of features that neural networks can approximate, thus making deep learning a more powerful approach. On the other hand, automated learning of hand-crafted features may require a prohibitively large number of training samples. Here we directly test the ability of convolutional neural networks (CNNs) to learn and predict the intensity, shape, and texture properties of tumors as defined by standardized radiomic features. MethodsConventional 2D and 3D CNN architectures with an increasing number of convolutional layers were trained to predict the values of 16 standardized radiomic features from real and synthetic PET images of tumors, and tested. In addition, several ImageNet-pretrained advanced networks were tested. A total of 4000 images were used for training, 500 for validation, and 500 for testing. ResultsFeatures quantifying size and intensity were predicted with high accuracy, while shape irregularity and heterogeneity features had very high prediction errors and generalized poorly. For example, mean normalized prediction error of tumor diameter with a 5-layer CNN was 4.23 ± 0.25, while the error for tumor sphericity was 15.64 ± 0.93. We additionally found that learning shape features required an order of magnitude more samples compared to intensity and size features. ConclusionsOur findings imply that CNNs trained to perform various image-based clinical tasks may generally under-utilize the shape and texture information that is more easily captured by radiomics. We speculate that to improve the CNN performance, shape and texture features can be computed explicitly and added as auxiliary variables to the networks, or supplied as synthetic inputs.

Improving mammography lesion classification by optimal fusion of handcrafted and deep transfer learning features

Objective. Handcrafted radiomics features or deep learning model-generated automated features are commonly used to develop computer-aided diagnosis schemes of medical images. The objective of this study is to test the hypothesis that handcrafted and automated features contain complementary classification information and fusion of these two types of features can improve CAD performance. Approach. We retrospectively assembled a dataset involving 1535 lesions (740 malignant and 795 benign). Regions of interest (ROI) surrounding suspicious lesions are extracted and two types of features are computed from each ROI. The first one includes 40 radiomic features and the second one includes automated features computed from a VGG16 network using a transfer learning method. A single channel ROI image is converted to three channel pseudo-ROI images by stacking the original image, a bilateral filtered image, and a histogram equalized image. Two VGG16 models using pseudo-ROIs and 3 stacked original ROIs without pre-processing are used to extract automated features. Five linear support vector machines (SVM) are built using the optimally selected feature vectors from the handcrafted features, two sets of VGG16 model-generated automated features, and the fusion of handcrafted and each set of automated features, respectively. Main Results. Using a 10-fold cross-validation, the fusion SVM using pseudo-ROIs yields the highest lesion classification performance with area under ROC curve (AUC = 0.756 ± 0.042), which is significantly higher than those yielded by other SVMs trained using handcrafted or automated features only (p < 0.05). Significance. This study demonstrates that both handcrafted and automated futures contain useful information to classify breast lesions. Fusion of these two types of features can further increase CAD performance.

Deep learning-based AI model for signet-ring cell carcinoma diagnosis and chemotherapy response prediction in gastric cancer.

We aimed to develop a noninvasive artificial intelligence (AI) model to diagnose signet-ring cell carcinoma (SRCC) of gastric cancer (GC) and identify patients with SRCC who could benefit from postoperative chemotherapy based on preoperative contrast-enhanced computed tomography (CT). A total of 855 GC patients with 855 single GCs were included, of which 249 patients were diagnosed as SRCC by histopathologic examinations. The AI model was generated with clinical, handcrafted radiomic, and deep learning features. Model diagnostic performance was measured by area under the receiver operating characteristic curve (AUC), sensitivity, and specificity, while predictive performance was measured by Kaplan-Meier curves. In the test cohort (n=257), the AUC, sensitivity, and specificity of our AI model for diagnosing SRCC were 0.786 (95% CI: 0.721-0.845), 77.3%, and 69.2%, respectively. For the entire cohort, patients with AI-predicted high risk had a significantly shorter median OS compared with those with low risk (median overall survival [OS], 38.8vs. 64.2 months, p=0.009). Importantly, in pathologically confirmed advanced SRCC patients, AI-predicted high-risk status was indicative of a shorter overall survival (median overall survival [OS], 31.0 vs. 54.4 months, p=0.036) and marked chemotherapy resistance, whereas AI-predicted low-risk status had substantial chemotherapy benefit (median OS [without vs. with chemotherapy], 26.0 vs. not reached, p=0.013). The CT-based AI model demonstrated good performance for diagnosing SRCC, stratifying patient prognosis, and predicting chemotherapy responses. Advanced SRCC patients with AI-predicted low-risk status may benefit substantially from adjuvant chemotherapy.

Survival Prediction in Glioblastoma Using Combination of Deep Learning and Hand-Crafted Radiomic Features in MRI Images

Survival Prediction in Glioblastoma Using Combination of Deep Learning and Hand-Crafted Radiomic Features in MRI Images