Integrative Model of CT Imaging and Clinical Features Using Attentional Multi-view Convolutional Neural Network (AM-CNN) for Prediction of Esophageal Fistula in Esophageal Cancer

Abstract

The purpose of this study was to propose an integrative risk model to predict esophageal fistula in patients with esophageal cancer (EC) using both clinical data and computerized tomography (CT) radiomic features. Radiomic features and clinical features of 558 EC patients who didn’t received esophageal surgery between July 2014 and August 2019 were extracted and analyzed. Of these, 186 patients (cases) who developed esophageal fistula were enrolled and compared with 372 controls (1:2 matched with the diagnosis time of EC, sex, marriage, and race). All 558 patients were divided into training set (n = 390) and validation set (n = 168) randomly. Data augmentation was performed including shifting of [-10, -5, 0, +5, +10] pixels along x and y axis and rotations of [-10,0, +10] degrees, resulting in 130×25×3×3 = 29250 samples. In the training set (130 cases and 260 controls), the model consists of three components: a) radiomic, multi-scale radiographic descriptor, extraction by AM-CNN. 2D slices from nine views of planes, where there were three patches of contextual CT, segmented tumor and neighboring information in each view were fed into AM-CNN. b) 33 clinical factors including general, diagnostic, pathologic, therapeutic and hematological parameters were fed into neural network for high-level latent representation. c) the radiographic descriptor and latent clinical factor representations are associated by a fully connected layer for patient level risk prediction using SoftMax classifier which were validated in validation set (56 cases and 112 controls). The integrative model was found to be more predictive, achieving (C-index, sensitivity, specification) of (0.901, 0.835, 0.918) in external validation, which was better than using clinical data alone by logistic regression (C-index 0.799) and using radiomic features alone by AM-CNN (C-index 0.855). The integration of radiomic descriptors from CT Imaging and clinical data improved the prediction performance of esophageal fistula in EC patients. We suggest that this model has the potential to support individualized stratification and treatment planning for EC patients.