Hand-crafted Radiomics Features Research Articles

CT-based deep learning radiomics nomogram for the prediction of pathological grade in bladder cancer: a multicenter study

BackgroundTo construct and assess a computed tomography (CT)-based deep learning radiomics nomogram (DLRN) for predicting the pathological grade of bladder cancer (BCa) preoperatively.MethodsWe retrospectively enrolled 688 patients with BCa (469 in the training cohort, 219 in the external test cohort) who underwent surgical resection. We extracted handcrafted radiomics (HCR) features and deep learning (DL) features from three-phase CT images (including corticomedullary-phase [C-phase], nephrographic-phase [N-phase] and excretory-phase [E-phase]). We constructed predictive models using 11 machine learning classifiers, and we developed a DLRN by combining the radiomic signature with clinical factors. We assessed performance and clinical utility of the models with reference to the area under the curve (AUC), calibration curve, and decision curve analysis (DCA).ResultsThe support vector machine (SVM) classifier model based on HCR and DL combined features was the best radiomic signature, with AUC values of 0.953 and 0.943 in the training cohort and the external test cohort, respectively. The AUC values of the clinical model in the training cohort and the external test cohort were 0.752 and 0.745, respectively. DLRN performed well on both data cohorts (training cohort: AUC = 0.961; external test cohort: AUC = 0.947), and outperformed the clinical model and the optimal radiomic signature.ConclusionThe proposed CT-based DLRN showed good diagnostic capability in distinguishing between high and low grade BCa.

Development of a multi-feature-combined model: proof-of-concept with application to local failure prediction of post-SBRT or surgery early-stage NSCLC patients.

To develop a Multi-Feature-Combined (MFC) model for proof-of-concept in predicting local failure (LR) in NSCLC patients after surgery or SBRT using pre-treatment CT images. This MFC model combines handcrafted radiomic features, deep radiomic features, and patient demographic information in an integrated machine learning workflow. The MFC model comprised three key steps. (1) Extraction of 92 handcrafted radiomic features from the GTV segmented on pre-treatment CT images. (2) Extraction of 512 deep radiomic features from pre-trained U-Net encoder. (3) The extracted handcrafted radiomic features, deep radiomic features, along with 4 patient demographic information (i.e., gender, age, tumor volume, and Charlson comorbidity index), were concatenated as a multi-dimensional input to the classifiers for LR prediction. Two NSCLC patient cohorts from our institution were investigated: (1) the surgery cohort includes 83 patients with segmentectomy or wedge resection (7 LR), and (2) the SBRT cohort includes 84 patients with lung SBRT (9 LR). The MFC model was developed and evaluated independently for both cohorts, and was subsequently compared against the prediction models based on only handcrafted radiomic features (R models), patient demographic information (PI models), and deep learning modeling (DL models). ROC with AUC was adopted to evaluate model performance with leave-one-out cross-validation (LOOCV) and 100-fold Monte Carlo random validation (MCRV). The t-test was performed to identify the statistically significant differences. In LOOCV, the AUC range (surgery/SBRT) of the MFC model was 0.858-0.895/0.868-0.913, which was higher than the three other models: 0.356-0.480/0.322-0.650 for PI models, 0.559-0.618/0.639-0.682 for R models, and 0.809/0.843 for DL models. In 100-fold MCRV, the MFC model again showed the highest AUC results (surgery/SBRT): 0.742-0.825/0.888-0.920, which were significantly higher than PI models: 0.464-0.564/0.538-0.628, R models: 0.557-0.652/0.551-0.732, and DL models: 0.702/0.791. We successfully developed an MFC model that combines feature information from multiple sources for proof-of-concept prediction of LR in patients with surgical and SBRT early-stage NSCLC. Initial results suggested that incorporating pre-treatment patient information from multiple sources improves the ability to predict the risk of local failure.

Multi-task deep learning-based radiomic nomogram for prognostic prediction in locoregionally advanced nasopharyngeal carcinoma

PurposePrognostic prediction is crucial to guide individual treatment for locoregionally advanced nasopharyngeal carcinoma (LA-NPC) patients. Recently, multi-task deep learning was explored for joint prognostic prediction and tumor segmentation in various cancers, resulting in promising performance. This study aims to evaluate the clinical value of multi-task deep learning for prognostic prediction in LA-NPC patients.MethodsA total of 886 LA-NPC patients acquired from two medical centers were enrolled including clinical data, [18F]FDG PET/CT images, and follow-up of progression-free survival (PFS). We adopted a deep multi-task survival model (DeepMTS) to jointly perform prognostic prediction (DeepMTS-Score) and tumor segmentation from FDG-PET/CT images. The DeepMTS-derived segmentation masks were leveraged to extract handcrafted radiomics features, which were also used for prognostic prediction (AutoRadio-Score). Finally, we developed a multi-task deep learning-based radiomic (MTDLR) nomogram by integrating DeepMTS-Score, AutoRadio-Score, and clinical data. Harrell's concordance indices (C-index) and time-independent receiver operating characteristic (ROC) analysis were used to evaluate the discriminative ability of the proposed MTDLR nomogram. For patient stratification, the PFS rates of high- and low-risk patients were calculated using Kaplan–Meier method and compared with the observed PFS probability.ResultsOur MTDLR nomogram achieved C-index of 0.818 (95% confidence interval (CI): 0.785–0.851), 0.752 (95% CI: 0.638–0.865), and 0.717 (95% CI: 0.641–0.793) and area under curve (AUC) of 0.859 (95% CI: 0.822–0.895), 0.769 (95% CI: 0.642–0.896), and 0.730 (95% CI: 0.634–0.826) in the training, internal validation, and external validation cohorts, which showed a statistically significant improvement over conventional radiomic nomograms. Our nomogram also divided patients into significantly different high- and low-risk groups.ConclusionOur study demonstrated that MTDLR nomogram can perform reliable and accurate prognostic prediction in LA-NPC patients, and also enabled better patient stratification, which could facilitate personalized treatment planning.

Diffusion tensor imaging-based machine learning for IDH wild-type glioblastoma stratification to reveal the biological underpinning of radiomic features.

This study addresses the lack of systematic investigation into the prognostic value of hand-crafted radiomic features derived from diffusion tensor imaging (DTI) in isocitrate dehydrogenase (IDH) wild-type glioblastoma (GBM), as well as the limited understanding of the biological interpretation of individual DTI radiomic features and metrics. To develop and validate a DTI-based radiomic model for predicting prognosis in patients with IDH wild-type GBM and reveal the biological underpinning of individual DTI radiomic features and metrics. The DTI-based radiomic signature was an independent prognostic factor (p < 0.001). Incorporating the radiomic signature into a clinical model resulted in a radiomic-clinical nomogram that predicted survival better than either the radiomic model or clinical model alone, with a better calibration and classification accuracy. Four categories of pathways (synapse, proliferation, DNA damage response, and complex cellular functions) were significantly correlated with the DTI-based radiomic features and DTI metrics. The prognostic radiomic features derived from DTI are driven by distinct pathways involved in synapse, proliferation, DNA damage response, and complex cellular functions of GBM.

Deep versus Handcrafted Tensor Radiomics Features: Prediction of Survival in Head and Neck Cancer Using Machine Learning and Fusion Techniques.

Although handcrafted radiomics features (RF) are commonly extracted via radiomics software, employing deep features (DF) extracted from deep learning (DL) algorithms merits significant investigation. Moreover, a "tensor'' radiomics paradigm where various flavours of a given feature are generated and explored can provide added value. We aimed to employ conventional and tensor DFs, and compare their outcome prediction performance to conventional and tensor RFs. 408 patients with head and neck cancer were selected from TCIA. PET images were first registered to CT, enhanced, normalized, and cropped. We employed 15 image-level fusion techniques (e.g., dual tree complex wavelet transform (DTCWT)) to combine PET and CT images. Subsequently, 215 RFs were extracted from each tumor in 17 images (or flavours) including CT only, PET only, and 15 fused PET-CT images through the standardized-SERA radiomics software. Furthermore, a 3 dimensional autoencoder was used to extract DFs. To predict the binary progression-free-survival-outcome, first, an end-to-end CNN algorithm was employed. Subsequently, we applied conventional and tensor DFs vs. RFs as extracted from each image to three sole classifiers, namely multilayer perceptron (MLP), random-forest, and logistic regression (LR), linked with dimension reduction algorithms. DTCWT fusion linked with CNN resulted in accuracies of 75.6 ± 7.0% and 63.4 ± 6.7% in five-fold cross-validation and external-nested-testing, respectively. For the tensor RF-framework, polynomial transform algorithms + analysis of variance feature selector (ANOVA) + LR enabled 76.67 ± 3.3% and 70.6 ± 6.7% in the mentioned tests. For the tensor DF framework, PCA + ANOVA + MLP arrived at 87.0 ± 3.5% and 85.3 ± 5.2% in both tests. This study showed that tensor DF combined with proper machine learning approaches enhanced survival prediction performance compared to conventional DF, tensor and conventional RF, and end-to-end CNN frameworks.

Optimal 18F-FDG PET/CT radiomics model development for predicting EGFR mutation status and prognosis in lung adenocarcinoma: a multicentric study.

To develop and interpret optimal predictive models to identify epidermal growth factor receptor (EGFR) mutation status and subtypes in patients with lung adenocarcinoma based on multicentric 18F-FDG PET/CT data, and further construct a prognostic model to predict their clinical outcome. The 18F-FDG PET/CT imaging and clinical characters of 767 patients with lung adenocarcinoma from 4 cohorts were collected. Seventy-six radiomics candidates using cross-combination method to identity EGFR mutation status and subtypes were built. Further, Shapley additive explanations and local interpretable model-agnostic explanations were used for optimal models' interpretation. Moreover, in order to predict the overall survival, a multivariate Cox proportional hazard model based on handcrafted radiomics features and clinical characteristics was constructed. The predictive performance and clinical net benefit of the models were evaluated via area under receiver operating characteristic (AUC), C-index and decision curve analysis. Among the 76 radiomics candidates, light gradient boosting machine classifier (LGBM) combined with recursive feature elimination wrapped LGBM feature selection method achieved best performance in predicting EGFR mutation status (AUC reached 0.80, 0.61, 0.71 in the internal test cohort and two external test cohorts, respectively). And extreme gradient boosting classifier combined with support vector machine feature selection method achieved best performance in predicting EGFR subtypes (AUC reached 0.76, 0.63, 0.61 in the internal test cohort and two external test cohorts, respectively). The C-index of the Cox proportional hazard model achieved 0.863. The integration of cross-combination method and the external validation from multi-center data achieved a good prediction and generalization performance in predicting EGFR mutation status and its subtypes. The combination of handcrafted radiomics features and clinical factors achieved good performance in predicting prognosis. With the urgent needs of multicentric 18F-FDG PET/CT trails, robust and explainable radiomics models have great potential in decision making and prognosis prediction of lung adenocarcinoma.

The Role of Radiomics in Fibrosis Crohn's Disease: A Review.

Inflammatory bowel disease (IBD) is a global health concern that has been on the rise in recent years. In addition, imaging is the established method of care for detecting, diagnosing, planning treatment, and monitoring the progression of IBD. While conventional imaging techniques are limited in their ability to provide comprehensive information, cross-sectional imaging plays a crucial role in the clinical management of IBD. However, accurately characterizing, detecting, and monitoring fibrosis in Crohn's disease remains a challenging task for clinicians. Recent advances in artificial intelligence technology, machine learning, computational power, and radiomic emergence have enabled the automated evaluation of medical images to generate prognostic biomarkers and quantitative diagnostics. Radiomics analysis can be achieved via deep learning algorithms or by extracting handcrafted radiomics features. As radiomic features capture pathophysiological and biological data, these quantitative radiomic features have been shown to offer accurate and rapid non-invasive tools for IBD diagnostics, treatment response monitoring, and prognosis. For these reasons, the present review aims to provide a comprehensive review of the emerging radiomics methods in intestinal fibrosis research that are highlighted and discussed in terms of challenges and advantages.

A multi-stage fusion framework to classify breast lesions using deep learning and radiomics features computed from four-view mammograms.

Developing computer aided diagnosis (CAD) schemes of mammograms to classify between malignant and benign breast lesions has attracted a lot of research attention over the last several decades. However, unlike radiologists who make diagnostic decisions based on the fusion of image features extracted from multi-view mammograms, most CAD schemes are single-view-based schemes, which limit CAD performance and clinical utility. This study aims to develop and test a novel CAD framework that optimally fuses information extracted from ipsilateral views of bilateral mammograms using both deep transfer learning (DTL) and radiomics feature extraction methods. An image dataset containing 353 benign and 611 malignant cases is assembled. Each case contains four images: the craniocaudal (CC) and mediolateral oblique (MLO) view of the left and right breast. First, we extract four matching regions of interest (ROIs) from images that surround centers of two suspicious lesion regions seen in CC and MLO views, as well as matching ROIs in the contralateral breasts. Next, the handcrafted radiomics (HCRs) features and VGG16 model-generated automated features are extracted from each ROI resulting in eight feature vectors. Then, after reducing feature dimensionality and quantifying the bilateral and ipsilateral asymmetry of four ROIs to yield four new feature vectors, we test four fusion methods to build three support vector machine (SVM) classifiers by an optimal fusion of asymmetrical image features extracted from four view images. Using a 10-fold cross-validation method, results show that a SVM classifier trained using an optimal fusion of four view images yields the highest classification performance (AUC=0.876±0.031), which significantly outperforms SVM classifiers trained using one projection view alone, AUC=0.817±0.026 and 0.792±0.026 for the CC and MLO view of bilateral mammograms, respectively (p<0.001). The study demonstrates that the shift from single-view CAD to four-view CAD and the inclusion of both DTL and radiomics features significantly increases CAD performance in distinguishing between malignant and benign breast lesions.

Differentiation of acute and chronic vertebral compression fractures using conventional CT based on deep transfer learning features and hand-crafted radiomics features

BackgroundWe evaluated the diagnostic efficacy of deep learning radiomics (DLR) and hand-crafted radiomics (HCR) features in differentiating acute and chronic vertebral compression fractures (VCFs).MethodsA total of 365 patients with VCFs were retrospectively analysed based on their computed tomography (CT) scan data. All patients completed MRI examination within 2 weeks. There were 315 acute VCFs and 205 chronic VCFs. Deep transfer learning (DTL) features and HCR features were extracted from CT images of patients with VCFs using DLR and traditional radiomics, respectively, and feature fusion was performed to establish the least absolute shrinkage and selection operator. The MRI display of vertebral bone marrow oedema was used as the gold standard for acute VCF, and the model performance was evaluated using the receiver operating characteristic (ROC).To separately evaluate the effectiveness of DLR, traditional radiomics and feature fusion in the differential diagnosis of acute and chronic VCFs, we constructed a nomogram based on the clinical baseline data to visualize the classification evaluation. The predictive power of each model was compared using the Delong test, and the clinical value of the nomogram was evaluated using decision curve analysis (DCA).ResultsFifty DTL features were obtained from DLR, 41 HCR features were obtained from traditional radiomics, and 77 features fusion were obtained after feature screening and fusion of the two. The area under the curve (AUC) of the DLR model in the training cohort and test cohort were 0.992 (95% confidence interval (CI), 0.983-0.999) and 0.871 (95% CI, 0.805-0.938), respectively. While the AUCs of the conventional radiomics model in the training cohort and test cohort were 0.973 (95% CI, 0.955-0.990) and 0.854 (95% CI, 0.773-0.934), respectively. The AUCs of the features fusion model in the training cohort and test cohort were 0.997 (95% CI, 0.994-0.999) and 0.915 (95% CI, 0.855-0.974), respectively. The AUCs of nomogram constructed by the features fusion in combination with clinical baseline data were 0.998 (95% CI, 0.996–0.999) and 0.946 (95% CI, 0.906–0.987) in the training cohort and test cohort, respectively. The Delong test showed that the differences between the features fusion model and the nomogram in the training cohort and the test cohort were not statistically significant (P values were 0.794 and 0.668, respectively), and the differences in the other prediction models in the training cohort and the test cohort were statistically significant (P < 0.05). DCA showed that the nomogram had high clinical value.ConclusionThe features fusion model can be used for the differential diagnosis of acute and chronic VCFs, and its differential diagnosis ability is improved when compared with that when either radiomics is used alone. At the same time, the nomogram has a high predictive value for acute and chronic VCFs and can be a potential decision-making tool to assist clinicians, especially when a patient is unable to undergo spinal MRI examination.

Radiomics-Guided Global-Local Transformer for Weakly Supervised Pathology Localization in Chest X-Rays.

Before the recent success of deep learning methods for automated medical image analysis, practitioners used handcrafted radiomic features to quantitatively describe local patches of medical images. However, extracting discriminative radiomic features relies on accurate pathology localization, which is difficult to acquire in real-world settings. Despite advances in disease classification and localization from chest X-rays, many approaches fail to incorporate clinically-informed domainspecific radiomic features. For these reasons, we propose a Radiomics-Guided Transformer (RGT) that fuses global image information with local radiomics-guided auxiliary information to provide accurate cardiopulmonary pathology localization and classification without any bounding box annotations. RGT consists of an image Transformer branch, a radiomics Transformer branch, and fusion layers that aggregate image and radiomics information. Using the learned self-attention of its image branch, RGT extracts a bounding box for which to compute radiomic features, which are further processed by the radiomics branch; learned image and radiomic features are then fused and mutually interact via cross-attention layers. Thus, RGT utilizes a novel end-to-end feedback loop that can bootstrap accurate pathology localization only using image-level disease labels. Experiments on the NIH ChestXRay dataset demonstrate that RGT outperforms prior works in weakly supervised disease localization (by an average margin of 3.6% over various intersection-over-union thresholds) and classification (by 1.1% in average area under the receiver operating characteristic curve). We publicly release our codes and pre-trained models at https://github.com/VITAGroup/chext.

Radiogenomic association of deep MR imaging features with genomic profiles and clinical characteristics in breast cancer

BackgroundIt has been believed that traditional handcrafted radiomic features extracted from magnetic resonance imaging (MRI) of tumors are normally shallow and low-ordered. Recent advancement in deep learning technology shows that the high-order deep radiomic features extracted automatically from tumor images can capture tumor heterogeneity in a more efficient way. We hypothesize that MRI-based deep radiomic phenotypes have significant associations with molecular profiles of breast cancer tumors. We aim to identify deep radiomic features (DRFs) from MRI, evaluate their significance in predicting breast cancer (BC) clinical characteristics and explore their associations with multi-level genomic factors.MethodsA denoising autoencoder was built to retrospectively extract 4,096 DRFs from 110 BC patients’ MRI. Visualization and clustering were applied to these DRFs. Linear Mixed Effect models were used to test their associations with multi-level genomic features (GFs) (risk genes, gene signatures, and biological pathway activities) extracted from the same patients’ mRNA expression profile. A Least Absolute Shrinkage and Selection Operator model was used to identify the most predictive DRFs for each clinical characteristic (tumor size (T), lymph node metastasis (N), estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2) status).ResultsThirty-six conventional radiomic features (CRFs) for 87 of the 110 BC patients provided by a previous study were used for comparison. More than 1,000 DRFs were associated with the risk genes, gene signatures, and biological pathways activities (adjusted P-value < 0.05). DRFs produced better performance in predicting T, N, ER, PR, and HER2 status (AUC > 0.9) using DRFs. These DRFs showed significant powers of stratifying patients, linking to relevant biological and clinical characteristics. As a contrast, only eight risk genes were associated with CRFs. The RFs performed worse in predicting clinical characteristics than DRFs.ConclusionsThe deep learning-based auto MRI features perform better in predicting BC clinical characteristics, which are more significantly associated with GFs than traditional semi-auto MRI features. Our radiogenomic approach for identifying MRI-based imaging signatures may pave potential pathways for the discovery of genetic mechanisms regulating specific tumor phenotypes and may enable a more rapid innovation of novel imaging modalities, hence accelerating their translation to personalized medicine.

Brain metastasis magnetic resonance imaging-based deep learning for predicting epidermal growth factor receptor (EGFR) mutation and subtypes in metastatic non-small cell lung cancer.

The preoperative identification of epidermal growth factor receptor (EGFR) mutations and subtypes based on magnetic resonance imaging (MRI) of brain metastases (BM) is necessary to facilitate individualized therapy. This study aimed to develop a deep learning model to preoperatively detect EGFR mutations and identify the location of EGFR mutations in patients with non-small cell lung cancer (NSCLC) and BM. We included 160 and 72 patients who underwent contrast-enhanced T1-weighted (T1w-CE) and T2-weighted (T2W) MRI at Liaoning Cancer Hospital and Institute (center 1) and Shengjing Hospital of China Medical University (center 2) to form a training cohort and an external validation cohort, respectively. A multiscale feature fusion network (MSF-Net) was developed by adaptively integrating features based on different stages of residual network (ResNet) 50 and by introducing channel and spatial attention modules. The external validation set from center 2 was used to assess the performance of MSF-Net and to compare it with that of handcrafted radiomics features. Receiver operating characteristic (ROC) curves, accuracy, precision, recall, and F1-score were used to evaluate the effectiveness of the models. Gradient-weighted class activation mapping (Grad-CAM) was used to demonstrate the attention of the MSF-Net model. The developed MSF-Net generated a better diagnostic performance than did the handcrafted radiomics in terms of the microaveraged area under the curve (AUC) (MSF-Net: 0.91; radiomics: 0.80) and macroaveraged AUC (MSF-Net: 0.90; radiomics: 0.81) for predicting EGFR mutations and subtypes. This study provides an end-to-end and noninvasive imaging tool for the preoperative prediction of EGFR mutation status and subtypes based on BM, which may be helpful for facilitating individualized clinical treatment plans.

NIMG-73. PREDICTING PERITUMORAL GLIOBLASTOMA INFILTRATION USING QUANTITATIVE MAGNETIC RESONANCE FINGERPRINTING RADIOMICS

Abstract PURPOSE Standard clinical magnetic resonance (MR) imaging of glioblastoma (GB) cannot identify malignant infiltration into the peritumoral non-enhancing region. This work implements quantitative MR fingerprinting (MRF) based radiomics to predict infiltrative regions of GB. METHODS Pre-operative MRF T1 and T2 maps along with multiparametric MR (mpMR) images (T1w, T2w, T1w-Gd, FLAIR, and ADC) from GB patients (n = 10) were analyzed. All subjects had histologically confirmed sites of non-enhancing peritumoral infiltration as identified by intra-operative 5-ALA fluorescence-guided tissue resection. These locations (n = 40) were manually annotated on pre-operative FLAIR by a board-certified neuroradiologist and labeled as “infiltration” (INF). For each patient, another FLAIR region ( &amp;gt; 3 cm from the enhancing tumor margin) was identified and labeled as “edema” (ED). Following image co-registration, 693 handcrafted radiomic voxel-based features were extracted from confirmed infiltration (INF) and edema (ED) voxels using a 3D 5x5x5 voxel sliding kernel. Feature selection was performed using the minimum redundancy maximal relevance (MRMR) algorithm, then INF and ED voxel features were used to train two cross-validated binary support vector machine (SVM) models for voxel-wise infiltration prediction: 1) MRF only model and 2) combined MRF and mpMRI model. Following MRMR selection, the combined model included features from T1, T1w, T2w, T1w-Gd, FLAIR, and ADC images. Model performance was evaluated using one withheld subject with multiple known infiltration sites. Balanced test accuracy and receiver operating characteristic (ROC) curve analysis was used to evaluate classification performance. RESULTS The MRF only and combined model achieved balanced test accuracies of 77.87% and 73.21% and AUCs of 0.91 and 0.97, respectively. CONCLUSIONS This study demonstrates that MRF-based radiomic features can predict GB tumor infiltration with high accuracy and provide complementary value to standard clinical MRI features. Our results indicate the high potential of employing MRF to guide personalized GB treatment strategies.

Comparison of Traditional Radiomics, Deep Learning Radiomics and Fusion Methods for Axillary Lymph Node Metastasis Prediction in Breast Cancer

Accurate identification of axillary lymph node (ALN) status in breast cancer patients is important for determining treatment options and avoiding axillary overtreatments. Our study aims to comprehensively compare the performance of the traditional radiomics model, deep learning radiomics model, and the fusion models in evaluating breast cancer ALN status based on dynamic contrast-enhanced-magnetic resonance imaging (DCE-MRI) images. The handcrafted radiomics features and deep features were extracted from 3062 DCE-MRI images. The feature selection was performed by applying mutual information and feature recursive elimination algorithms. The traditional radiomics model and deep learning radiomics model were built using the optimal features and machine learning classifiers, respectively. The fusion models for distinguishing axillary lymph node status were constructed using two fusion strategies. The performance of the models with MRI-reported lymphadenopathy or suspicious nodes to evaluate axillary lymph node status was also compared. The decision fusion model, with the integration of the radiomics features and deep learning features at the decision level, achieved an area under the curve (AUC) of 0.91 (95% confidence interval (CI): 0.879-0.937), which was higher than that of the traditional radiomics model and deep learning radiomics model. The results of the decision fusion model with clinical characteristic yielded an AUC of 0.93 (95% CI: 0.899-0.951), which was also superior to other models incorporating clinical characteristic. This study demonstrates the effectiveness of the fusion models for predicting axillary lymph node metastasis in breast cancer.

Deterministic small-scale undulations of image-based risk predictions from the deep learning of lung tumors in motion.

Deep learning (DL) models that use medical images to predict clinical outcomes are poised for clinical translation. For tumors that reside in organs that move, however, the impact of motion (i.e., degenerated object appearance or blur) on DL model accuracy remains unclear. We examine the impact of tumor motion on an image-based DL framework that predicts local failure risk after lung stereotactic body radiotherapy (SBRT). We input pre-therapy free breathing (FB) computed tomography (CT) images from 849 patients treated with lung SBRT into a multitask deep neural network to generate an image fingerprint signature (or DL score) that predicts time-to-event local failure outcomes. The network includes a convolutional neural network encoder for extracting imaging features and building a task-specific fingerprint, a decoder for estimating handcrafted radiomic features, and a task-specific network for generating image signature for radiotherapy outcome prediction. The impact of tumor motion on the DL scores was then examined for a holdout set of 468 images from 39 patients comprising: (1) FB CT, (2) four-dimensional (4D) CT, and (3) maximum-intensity projection (MIP) images. Tumor motion was estimated using a 3D vector of the maximum distance traveled, and its association with DL score variance was assessed by linear regression. The variance and amplitude in 4D CT image-derived DL scores were associated with tumor motion (R2 =0.48 and 0.46, respectively). Specifically, DL score variance was deterministic and represented by sinusoidal undulations in phase with the respiratory cycle. DL scores, but not tumor volumes, peaked near end-exhalation. The mean of the scores derived from 4D CT images and the score obtained from FB CT images were highly associated (Pearson r=0.99). MIP-derived DL scores were significantly higher than 4D- or FB-derived risk scores (p<0.0001). An image-based DL risk score derived from a series of 4D CT images varies in a deterministic, sinusoidal trajectory in a phase with the respiratory cycle. These results indicate that DL models of tumors in motion can be robust to fluctuations in object appearance due to movement and can guide standardization processes in the clinical translation of DL models for patients with lung cancer.

Radiomics Analysis of Lymph Nodes with Esophageal Squamous Cell Carcinoma Based on Deep Learning.

Purpose To assess the role of multiple radiomic features of lymph nodes in the preoperative prediction of lymph node metastasis (LNM) in patients with esophageal squamous cell carcinoma (ESCC). Methods Three hundred eight patients with pathologically confirmed ESCC were retrospectively enrolled (training cohort, n = 216; test cohort, n = 92). We extracted 207 handcrafted radiomic features and 1000 deep radiomic features of lymph nodes from their computed tomography (CT) images. The t-test and least absolute shrinkage and selection operator (LASSO) were used to reduce the dimensions and select key features. Handcrafted radiomics, deep radiomics, and clinical features were combined to construct models. Models I (handcrafted radiomic features), II (Model I plus deep radiomic features), and III (Model II plus clinical features) were built using three machine learning methods: support vector machine (SVM), adaptive boosting (AdaBoost), and random forest (RF). The best model was compared with the results of two radiologists, and its performance was evaluated in terms of sensitivity, specificity, accuracy, area under the curve (AUC), and receiver operating characteristic (ROC) curve analysis. Results No significant differences were observed between cohorts. Ten handcrafted and 12 deep radiomic features were selected from the extracted features (p < 0.05). Model III could discriminate between patients with and without LNM better than the diagnostic results of the two radiologists. Conclusion The combination of handcrafted radiomic features, deep radiomic features, and clinical features could be used clinically to assess lymph node status in patients with ESCC.

1139P A CT-based radiomic readout on the differential diagnosis of pneumonitis in stage IV non-small cell lung cancer patients treated with immune checkpoint inhibitors

About 4-10% patients who receive immunotherapy are subjected to Immunotherapy-Induced Pneumonitis (IIP) which is a lethal complication that requires immediate interruption of immunotherapy as well as corticosteroid administration. Less than 10% of the patients also develop other types of pneumonitis (OP), (e.g. infection or radiation-induced). Making the differential diagnosis of pneumonitis (IIP vs. OP) is a challenging task due to similar radiological patterns and low incidence rates. This study was performed to develop and evaluate a prediction model using handcrafted radiomic features and their associated clinical metadata to estimate the risk of IIP in patients with stage IV Non-Small Cell Lung Cancer (NSCLC) who developed pneumonitis after immunotherapy.

Computer-aided diagnostic models to classify lymph node metastasis and lymphoma involvement in enlarged cervical lymph nodes using PET/CT.

It is a clinical problem to identify histological component in enlarged cervical lymph nodes, particularly in differentiation between lymph node metastasis and lymphoma involvement. To construct two kinds of deep learning (DL)-based computer-aided diagnosis (CAD) systems including DL-convolutional neural networks (DL-CNN) and DL-machine learning for pathological diagnosis of cervical lymph nodes by positron emission tomography (PET)/computed tomography (CT) images. We collected CT, PET, and PET/CT images series from 165 patients with enlarged cervical lymph nodes receiving examinations from January 2014 to June 2018. Six CNNs pretrained on ImageNet as DL architectures were used for two kinds of DL-based CAD models, including DL-CNN and DL-machine learning models. The DL-CNN models were constructed via transfer learning for classification of lymphomatous and metastatic lymph nodes. The DL-machine learning models were developed by DL-based features extractors and support vector machine (SVM) classifier. As for DL-SVM models, we also evaluate the effect of handcrafted radiomics features in combination of DL-based features. The DL-CNN model with ResNet50 architecture on PET/CT images had the best diagnostic performance among all six algorithms with an area under the receiver operating characteristic curve (AUC) of 0.845 and accuracy of 78.13% in the testing cohort. The DL-SVM model on ResNet50 extractor showed great performance for the testing cohort with an AUC of 0.901, accuracy of 86.96%, sensitivity of 76.09%, and specificity of 94.20%. The combination of DL-based and handcrafted features yielded the improvement of diagnostic performance. Our DL-based CAD systems on PET/CT images were developed for classifying metastatic and lymphomatous involvement with favorable diagnostic performance in enlarged cervical lymph nodes. Further clinical practice of our systems may improve quality of the following therapeutic interventions and optimize patients' outcomes.

A deep learning-based radiomics approach to predict head and neck tumor regression for adaptive radiotherapy

Early regression—the regression in tumor volume during the initial phase of radiotherapy (approximately 2 weeks after treatment initiation)—is a common occurrence during radiotherapy. This rapid radiation-induced tumor regression may alter target coordinates, necessitating adaptive radiotherapy (ART). We developed a deep learning-based radiomics (DLR) approach to predict early head and neck tumor regression and thereby facilitate ART. Primary gross tumor volume (GTVp) was monitored in 96 patients and nodal GTV (GTVn) in 79 patients during treatment. All patients underwent two computed tomography (CT) scans: one before the start of radiotherapy for initial planning and one during radiotherapy for boost planning. Patients were assigned to regression and nonregression groups according to their median tumor regression rate (ΔGTV/treatment day from initial to boost CT scan). We input a GTV image into the convolutional neural network model, which was pretrained using natural image datasets, via transfer learning. The deep features were extracted from the last fully connected layer. To clarify the prognostic power of the deep features, machine learning models were trained. The models then predicted the regression and nonregression of GTVp and GTVn and evaluated the predictive performance by 0.632 + bootstrap area under the curve (AUC). Predictive performance for GTVp regression was highest using the InceptionResNetv2 model (mean AUC = 0.75) and that for GTVn was highest using NASNetLarge (mean AUC = 0.73). Both models outperformed the handcrafted radiomics features (mean AUC = 0.63 for GTVp and 0.61 for GTVn) or clinical factors (0.64 and 0.67, respectively). DLR may facilitate ART for improved radiation side-effects and target coverage.

CT Reconstruction Kernels and the Effect of Pre- and Post-Processing on the Reproducibility of Handcrafted Radiomic Features.

Handcrafted radiomics features (HRFs) are quantitative features extracted from medical images to decode biological information to improve clinical decision making. Despite the potential of the field, limitations have been identified. The most important identified limitation, currently, is the sensitivity of HRF to variations in image acquisition and reconstruction parameters. In this study, we investigated the use of Reconstruction Kernel Normalization (RKN) and ComBat harmonization to improve the reproducibility of HRFs across scans acquired with different reconstruction kernels. A set of phantom scans (n = 28) acquired on five different scanner models was analyzed. HRFs were extracted from the original scans, and scans were harmonized using the RKN method. ComBat harmonization was applied on both sets of HRFs. The reproducibility of HRFs was assessed using the concordance correlation coefficient. The difference in the number of reproducible HRFs in each scenario was assessed using McNemar’s test. The majority of HRFs were found to be sensitive to variations in the reconstruction kernels, and only six HRFs were found to be robust with respect to variations in reconstruction kernels. The use of RKN resulted in a significant increment in the number of reproducible HRFs in 19 out of the 67 investigated scenarios (28.4%), while the ComBat technique resulted in a significant increment in 36 (53.7%) scenarios. The combination of methods resulted in a significant increment in 53 (79.1%) scenarios compared to the HRFs extracted from original images. Since the benefit of applying the harmonization methods depended on the data being harmonized, reproducibility analysis is recommended before performing radiomics analysis. For future radiomics studies incorporating images acquired with similar image acquisition and reconstruction parameters, except for the reconstruction kernels, we recommend the systematic use of the pre- and post-processing approaches (respectively, RKN and ComBat).