Convolutional Neural Network Classification of Pretreatment Biopsies of Rectal Adenocarcinoma Predicts Complete Clinical Response to Short-Course Radiation and Consolidation Chemotherapy

Abstract

<h3>Purpose/Objective(s)</h3> Nonoperative treatment of rectal cancer (RCa) with short course radiation (SCRT) (25 Gy in 5 fractions) followed by chemotherapy results in high treatment efficacy while maintaining rectal function. Identifying patients likely to experience a complete response to nonoperative management could prevent overtreatment with chemotherapy and reduce the risk of metastatic progression for incomplete responders. We constructed a convolutional neural network (CNN) classifier of pre-treatment histopathological images to identify patients who will experience a complete response to SCRT and consolidation chemotherapy. <h3>Materials/Methods</h3> Image files of hematoxylin and eosin-stained pre-treatment rectal adenocarcinoma biopsies (n=15) were acquired using the Zeiss microscopy software suite. Files were segmented using grid tools in the ZEN software package. Grid images comprised largely of whitespace were manually removed. Grid images were exported to .jpeg format, preserving micron scale pixel resolution. This analysis utilized 32 .jpeg images belonging to complete responders and 37 .jpeg images belonging to partial responders as the basis for data augmentation to generate a more robust training cohort. We employed the architecture and weights of the well-known VGG16 model with fine tuning of the later layers of the model during training. Regularization, dropout, and early stopping were implemented to prevent overfitting. <h3>Results</h3> Utilizing the VGG16 model with frozen nodal weights resulted in poor classification accuracy and failure to improve via monitoring loss function (training accuracy: 0.3152, validation accuracy: 0.33). Unfreezing, or pruning, after the 7th layer of the VGG16 model and allowing fine tuning of the transferred architecture improved this approach. Later network architectural layers are hypothesized to represent the most sophisticated image features. When nodal weights in VGG16's later layers were allowed to be updated based on training from histopathological slide images the classifiers accuracy improved (training accuracy: 0.9457, validation accuracy: 0.944). Further, loss function was reliably minimized (Epoch 10 validation set binary cross-entropy loss: 0.610, Epoch 50 validation set binary cross-entropy loss: 0.0648). <h3>Conclusion</h3> This is the first study to utilize imaging features extracted by a CNN to predict biological behavior or responsiveness to therapy in rectal cancer. We demonstrated that a transfer learning and layer pruning approach outperforms classical transfer learning utilizing previously deeply trained architectures. While future studies to refine and validate this CNN classifier are warranted, deep learning methods can play a role in preventing overtreatment vs metastatic progression in patients receiving SCRT for RCa.