Whole-slide Histopathology Images Research Articles

Improved breast cancer histological grading using deep learning

The Nottingham histological grade (NHG) is a well-established prognostic factor for breast cancer that is broadly used in clinical decision making. However, ∼50% of patients are classified as grade 2, an intermediate risk group with low clinical value. To improve risk stratification of NHG 2 breast cancer patients, we developed and validated a novel histological grade model (DeepGrade) based on digital whole-slide histopathology images (WSIs) and deep learning. In this observational retrospective study, routine WSIs stained with haematoxylin and eosin from 1567 patients were utilised for model optimisation and validation. Model generalisability was further evaluated in an external test set with 1262 patients. NHG 2 cases were stratified into two groups, DG2-high and DG2-low, and the prognostic value was assessed. The main outcome was recurrence-free survival. DeepGrade provides independent prognostic information for stratification of NHG 2 cases in the internal test set, where DG2-high showed an increased risk for recurrence (hazard ratio [HR] 2.94, 95% confidence interval [CI] 1.24-6.97, P= 0.015) compared with the DG2-low group after adjusting for established risk factors (independent test data). DG2-low also shared phenotypic similarities with NHG 1, and DG2-high with NHG 3, suggesting that the model identifies morphological patterns in NHG 2 that are associated with more aggressive tumours. The prognostic value of DeepGrade was further assessed in the external test set, confirming an increased risk for recurrence in DG2-high (HR 1.91, 95% CI 1.11-3.29, P= 0.019). The proposed model-based stratification of patients with NHG 2 tumours is prognostic and adds clinically relevant information over routine histological grading. The methodology offers a cost-effective alternative to molecular profiling to extract information relevant for clinical decisions.

Integrative Analysis of Histopathological Images and Genomic Data in Colon Adenocarcinoma.

BackgroundColon adenocarcinoma (COAD) is one of the most common malignant tumors in the world. The histopathological features are crucial for the diagnosis, prognosis, and therapy of COAD.MethodsWe downloaded 719 whole-slide histopathological images from TCIA, and 459 corresponding HTSeq-counts mRNA expression and clinical data were obtained from TCGA. Histopathological image features were extracted by CellProfiler. Prognostic image features were selected by the least absolute shrinkage and selection operator (LASSO) and support vector machine (SVM) algorithms. The co-expression gene module correlated with prognostic image features was identified by weighted gene co-expression network analysis (WGCNA). Random forest was employed to construct an integrative prognostic model and calculate the histopathological-genomic prognosis factor (HGPF).ResultsThere were five prognostic image features and one co-expression gene module involved in the model construction. The time-dependent receiver operating curve showed that the prognostic model had a significant prognostic value. Patients were divided into high-risk group and low-risk group based on the HGPF. Kaplan-Meier analysis indicated that the overall survival of the low-risk group was significantly better than the high-risk group.ConclusionsThese results suggested that the histopathological image features had a certain ability to predict the survival of COAD patients. The integrative prognostic model based on the histopathological images and genomic features could further improve the prognosis prediction in COAD, which may assist the clinical decision in the future.

Prediction of BRCA Gene Mutation in Breast Cancer Based on Deep Learning and Histopathology Images.

BackgroundBreast cancer is one of the most common cancers and the leading cause of death from cancer among women worldwide. The genetic predisposition to breast cancer may be associated with a mutation in particular genes such as gene BRCA1/2. Patients who carry a germline pathogenic mutation in BRCA1/2 genes have a significantly increased risk of developing breast cancer and might benefit from targeted therapy. However, genetic testing is time consuming and costly. This study aims to predict the risk of gBRCA mutation by using the whole-slide pathology features of breast cancer H&E stains and the patients’ gBRCA mutation status.MethodsIn this study, we trained a deep convolutional neural network (CNN) of ResNet on whole-slide images (WSIs) to predict the gBRCA mutation in breast cancer. Since the dimensions are too large for slide-based training, we divided WSI into smaller tiles with the original resolution. The tile-based classification was then combined by adding the positive classification result to generate the combined slide-based accuracy. Models were trained based on the annotated tumor location and gBRCA mutation status labeled by a designated breast cancer pathologist. Four models were trained on tiles cropped at 5×, 10×, 20×, and 40× magnification, assuming that low magnification and high magnification may provide different levels of information for classification.ResultsA trained model was validated through an external dataset that contains 17 mutants and 47 wilds. In the external validation dataset, AUCs (95% CI) of DL models that used 40×, 20×, 10×, and 5× magnification tiles among all cases were 0.766 (0.763–0.769), 0.763 (0.758–0.769), 0.750 (0.738–0.761), and 0.551 (0.526–0.575), respectively, while the corresponding magnification slides among all cases were 0.774 (0.642–0.905), 0.804 (0.676–0.931), 0.828 (0.691–0.966), and 0.635 (0.471–0.798), respectively. The study also identified the influence of histological grade to the accuracy of the prediction.ConclusionIn this paper, the combination of pathology and molecular omics was used to establish the gBRCA mutation risk prediction model, revealing the correlation between the whole-slide histopathological images and gRCA mutation risk. The results indicated that the prediction accuracy is likely to improve as the training data expand. The findings demonstrated that deep CNNs could be used to assist pathologists in the detection of gene mutation in breast cancer.

Second-order multi-instance learning model for whole slide image classification

Whole slide histopathology images (WSIs) play a crucial role in diagnosing lymph node metastasis of breast cancer, which usually lack fine-grade annotations of tumor regions and have large resolutions (typically 105 × 105 pixels). Multi-instance learning has gradually become a dominant weakly supervised learning framework for WSI classification when only slide-level labels are available. In this paper, we develop a novel second-order multiple instances learning method (SoMIL) with an adaptive aggregator stacked by the attention mechanism and recurrent neural network (RNN) for histopathological image classification. To be specific, the proposed method applies a second-order pooling module (matrix power normalization covariance) for instance-level feature extraction of weakly supervised learning framework, attempting to explore second-order statistics of deep features for histopathological images. Additionally, we utilize an efficient channel attention mechanism to adaptively highlight the most discriminative instance features, followed by an RNN to update the final bag-level representation for the slide classification. Experimental results on the lymph node metastasis dataset of 2016 Camelyon grand challenge demonstrate the significant improvement of our proposed SoMIL framework compared with other state-of-the-art multi-instance learning methods. Moreover, in the external validation on 130 WSIs, SoMIL also achieves an impressive area under the curve performance that competitive to the fully-supervised framework.

Deep Learning for the Classification of Non-Hodgkin Lymphoma on Histopathological Images.

Simple SummaryHistopathological examination of lymph node (LN) specimens allows the detection of hematological diseases. The identification and the classification of lymphoma, a blood cancer with a manifestation in LNs, are difficult and require many years of training, as well as additional expensive investigations. Today, artificial intelligence (AI) can be used to support the pathologist in identifying abnormalities in LN specimens. In this article, we trained and optimized an AI algorithm to automatically detect two common lymphoma subtypes that require different therapies using normal LN parenchyma as a control. The balanced accuracy in an independent test cohort was above 95%, which means that the vast majority of cases were classified correctly and only a few cases were misclassified. We applied specific methods to explain which parts of the image were important for the AI algorithm and to ensure a reliable result. Our study shows that classifications of lymphoma subtypes is possible with high accuracy. We think that routine histopathological applications for AI should be pursued.The diagnosis and the subtyping of non-Hodgkin lymphoma (NHL) are challenging and require expert knowledge, great experience, thorough morphological analysis, and often additional expensive immunohistological and molecular methods. As these requirements are not always available, supplemental methods supporting morphological-based decision making and potentially entity subtyping are required. Deep learning methods have been shown to classify histopathological images with high accuracy, but data on NHL subtyping are limited. After annotation of histopathological whole-slide images and image patch extraction, we trained and optimized an EfficientNet convolutional neuronal network algorithm on 84,139 image patches from 629 patients and evaluated its potential to classify tumor-free reference lymph nodes, nodal small lymphocytic lymphoma/chronic lymphocytic leukemia, and nodal diffuse large B-cell lymphoma. The optimized algorithm achieved an accuracy of 95.56% on an independent test set including 16,960 image patches from 125 patients after the application of quality controls. Automatic classification of NHL is possible with high accuracy using deep learning on histopathological images and routine diagnostic applications should be pursued.

TSDLPP: A Novel Two-Stage Deep Learning Framework For Prognosis Prediction Based on Whole Slide Histopathological Images.

Recently, digital pathology image-based prognosis prediction has become a hot topic in healthcare research to make early decisions on therapy and improve the treatment quality of patients. Therefore, there has been a recent surge of interest in designing deep learning method solving the problem of prognosis prediction with digital pathology images. However, whole slide histopathological images (WSIs) based prognosis prediction is still a challenge due to the large size of pathological images, the heterogeneity of tumors and the high cost of region of interests (ROIs) labeling. In this study, we design a novel two-stage deep learning framework for prognosis prediction (TSDLPP) based on WSIs. Our proposed framework consists of two-stage paradigms: 1) training tissue decomposition network (TDNet) to divide WSIs into cancerous and non-cancerous regions, 2) integrating general prognosis-related densely connected CNN (GPR-DCCNN) and morphology-specific prognosis-related densely connected CNNs (MSPR-DCCNNs) to extract different level features of pathological images. In the end, we apply TSDLPP to the prognosis prediction of breast cancer using The Cancer Genome Atlas (TCGA) datasets. Experiment results demonstrate that TSDLPP obtains superior performance of prognosis prediction compared with the existing state-of-arts methods.

A fast and effective detection framework for whole-slide histopathology image analysis.

Pathologists generally pan, focus, zoom and scan tissue biopsies either under microscopes or on digital images for diagnosis. With the rapid development of whole-slide digital scanners for histopathology, computer-assisted digital pathology image analysis has attracted increasing clinical attention. Thus, the working style of pathologists is also beginning to change. Computer-assisted image analysis systems have been developed to help pathologists perform basic examinations. This paper presents a novel lightweight detection framework for automatic tumor detection in whole-slide histopathology images. We develop the Double Magnification Combination (DMC) classifier, which is a modified DenseNet-40 to make patch-level predictions with only 0.3 million parameters. To improve the detection performance of multiple instances, we propose an improved adaptive sampling method with superpixel segmentation and introduce a new heuristic factor, local sampling density, as the convergence condition of iterations. In postprocessing, we use a CNN model with 4 convolutional layers to regulate the patch-level predictions based on the predictions of adjacent sampling points and use linear interpolation to generate a tumor probability heatmap. The entire framework was trained and validated using the dataset from the Camelyon16 Grand Challenge and Hubei Cancer Hospital. In our experiments, the average AUC was 0.95 in the test set for pixel-level detection.

A Multi-Channel and Multi-Spatial Attention Convolutional Neural Network for Prostate Cancer ISUP Grading

Prostate cancer (PCa) is one of the most prevalent cancers worldwide. As the demand for prostate biopsies increases, a worldwide shortage and an uneven geographical distribution of proficient pathologists place a strain on the efficacy of pathological diagnosis. Deep learning (DL) is able to automatically extract features from whole-slide images of prostate biopsies annotated by skilled pathologists and to classify the severity of PCa. A whole-slide image of biopsies has many irrelevant features that weaken the performance of DL models. To enable DL models to focus more on cancerous tissues, we propose a Multi-Channel and Multi-Spatial (MCMS) Attention module that can be easily plugged into any backbone CNN to enhance feature extraction. Specifically, MCMS learns a channel attention vector to assign weights to channels in the feature map by pooling from multiple attention branches with different reduction ratios; similarly, it also learns a spatial attention matrix to focus on more relevant areas of the image, by pooling from multiple convolutional layers with different kernel sizes. The model is verified on the most extensive multi-center PCa dataset that consists of 11,000 H&E-stained histopathology whole-slide images. Experimental results demonstrate that an MCMS-assisted CNN can effectively boost prediction performance in accuracy (ACC) and quadratic weighted kappa (QWK), compared with prior studies. The proposed model and results can serve as a credible benchmark for future research in automated PCa grading.

Independent real-world application of a clinical-grade automated prostate cancer detection system.

Artificial intelligence (AI)‐based systems applied to histopathology whole‐slide images have the potential to improve patient care through mitigation of challenges posed by diagnostic variability, histopathology caseload, and shortage of pathologists. We sought to define the performance of an AI‐based automated prostate cancer detection system, Paige Prostate, when applied to independent real‐world data. The algorithm was employed to classify slides into two categories: benign (no further review needed) or suspicious (additional histologic and/or immunohistochemical analysis required). We assessed the sensitivity, specificity, positive predictive values (PPVs), and negative predictive values (NPVs) of a local pathologist, two central pathologists, and Paige Prostate in the diagnosis of 600 transrectal ultrasound‐guided prostate needle core biopsy regions (‘part‐specimens’) from 100 consecutive patients, and to ascertain the impact of Paige Prostate on diagnostic accuracy and efficiency. Paige Prostate displayed high sensitivity (0.99; CI 0.96–1.0), NPV (1.0; CI 0.98–1.0), and specificity (0.93; CI 0.90–0.96) at the part‐specimen level. At the patient level, Paige Prostate displayed optimal sensitivity (1.0; CI 0.93–1.0) and NPV (1.0; CI 0.91–1.0) at a specificity of 0.78 (CI 0.64–0.89). The 27 part‐specimens considered by Paige Prostate as suspicious, whose final diagnosis was benign, were found to comprise atrophy (n = 14), atrophy and apical prostate tissue (n = 1), apical/benign prostate tissue (n = 9), adenosis (n = 2), and post‐atrophic hyperplasia (n = 1). Paige Prostate resulted in the identification of four additional patients whose diagnoses were upgraded from benign/suspicious to malignant. Additionally, this AI‐based test provided an estimated 65.5% reduction of the diagnostic time for the material analyzed. Given its optimal sensitivity and NPV, Paige Prostate has the potential to be employed for the automated identification of patients whose histologic slides could forgo full histopathologic review. In addition to providing incremental improvements in diagnostic accuracy and efficiency, this AI‐based system identified patients whose prostate cancers were not initially diagnosed by three experienced histopathologists. © 2021 The Authors. The Journal of Pathology published by John Wiley & Sons, Ltd. on behalf of The Pathological Society of Great Britain and Ireland.

A deep learning model to detect pancreatic ductal adenocarcinoma on endoscopic ultrasound-guided fine-needle biopsy

Histopathological diagnosis of pancreatic ductal adenocarcinoma (PDAC) on endoscopic ultrasonography-guided fine-needle biopsy (EUS-FNB) specimens has become the mainstay of preoperative pathological diagnosis. However, on EUS-FNB specimens, accurate histopathological evaluation is difficult due to low specimen volume with isolated cancer cells and high contamination of blood, inflammatory and digestive tract cells. In this study, we performed annotations for training sets by expert pancreatic pathologists and trained a deep learning model to assess PDAC on EUS-FNB of the pancreas in histopathological whole-slide images. We obtained a high receiver operator curve area under the curve of 0.984, accuracy of 0.9417, sensitivity of 0.9302 and specificity of 0.9706. Our model was able to accurately detect difficult cases of isolated and low volume cancer cells. If adopted as a supportive system in routine diagnosis of pancreatic EUS-FNB specimens, our model has the potential to aid pathologists diagnose difficult cases.

Deep learning-based six-type classifier for lung cancer and mimics from histopathological whole slide images: a retrospective study

BackgroundTargeted therapy and immunotherapy put forward higher demands for accurate lung cancer classification, as well as benign versus malignant disease discrimination. Digital whole slide images (WSIs) witnessed the transition from traditional histopathology to computational approaches, arousing a hype of deep learning methods for histopathological analysis. We aimed at exploring the potential of deep learning models in the identification of lung cancer subtypes and cancer mimics from WSIs.MethodsWe initially obtained 741 WSIs from the First Affiliated Hospital of Sun Yat-sen University (SYSUFH) for the deep learning model development, optimization, and verification. Additional 318 WSIs from SYSUFH, 212 from Shenzhen People’s Hospital, and 422 from The Cancer Genome Atlas were further collected for multi-centre verification. EfficientNet-B5- and ResNet-50-based deep learning methods were developed and compared using the metrics of recall, precision, F1-score, and areas under the curve (AUCs). A threshold-based tumour-first aggregation approach was proposed and implemented for the label inferencing of WSIs with complex tissue components. Four pathologists of different levels from SYSUFH reviewed all the testing slides blindly, and the diagnosing results were used for quantitative comparisons with the best performing deep learning model.ResultsWe developed the first deep learning-based six-type classifier for histopathological WSI classification of lung adenocarcinoma, lung squamous cell carcinoma, small cell lung carcinoma, pulmonary tuberculosis, organizing pneumonia, and normal lung. The EfficientNet-B5-based model outperformed ResNet-50 and was selected as the backbone in the classifier. Tested on 1067 slides from four cohorts of different medical centres, AUCs of 0.970, 0.918, 0.963, and 0.978 were achieved, respectively. The classifier achieved high consistence to the ground truth and attending pathologists with high intraclass correlation coefficients over 0.873.ConclusionsMulti-cohort testing demonstrated our six-type classifier achieved consistent and comparable performance to experienced pathologists and gained advantages over other existing computational methods. The visualization of prediction heatmap improved the model interpretability intuitively. The classifier with the threshold-based tumour-first label inferencing method exhibited excellent accuracy and feasibility in classifying lung cancers and confused nonneoplastic tissues, indicating that deep learning can resolve complex multi-class tissue classification that conforms to real-world histopathological scenarios.

Development of a Histopathology Informatics Pipeline for Classification and Prediction of Clinical Outcomes in Subtypes of Renal Cell Carcinoma

Abstract Purpose: Histopathology evaluation is the gold standard for diagnosing clear cell (ccRCC), papillary, and chromophobe renal cell carcinoma (RCC). However, interrater variability has been reported, and the whole-slide histopathology images likely contain underutilized biological signals predictive of genomic profiles. Experimental Design: To address this knowledge gap, we obtained whole-slide histopathology images and demographic, genomic, and clinical data from The Cancer Genome Atlas, the Clinical Proteomic Tumor Analysis Consortium, and Brigham and Women's Hospital (Boston, MA) to develop computational methods for integrating data analyses. Leveraging these large and diverse datasets, we developed fully automated convolutional neural networks to diagnose renal cancers and connect quantitative pathology patterns with patients' genomic profiles and prognoses. Results: Our deep convolutional neural networks successfully detected malignancy (AUC in the independent validation cohort: 0.964–0.985), diagnosed RCC histologic subtypes (independent validation AUCs of the best models: 0.953–0.993), and predicted stage I ccRCC patients' survival outcomes (log-rank test P = 0.02). Our machine learning approaches further identified histopathology image features indicative of copy-number alterations (AUC > 0.7 in multiple genes in patients with ccRCC) and tumor mutation burden. Conclusions: Our results suggest that convolutional neural networks can extract histologic signals predictive of patients' diagnoses, prognoses, and genomic variations of clinical importance. Our approaches can systematically identify previously unknown relations among diverse data modalities.

Human-interpretable image features derived from densely mapped cancer pathology slides predict diverse molecular phenotypes

Computational methods have made substantial progress in improving the accuracy and throughput of pathology workflows for diagnostic, prognostic, and genomic prediction. Still, lack of interpretability remains a significant barrier to clinical integration. We present an approach for predicting clinically-relevant molecular phenotypes from whole-slide histopathology images using human-interpretable image features (HIFs). Our method leverages >1.6 million annotations from board-certified pathologists across >5700 samples to train deep learning models for cell and tissue classification that can exhaustively map whole-slide images at two and four micron-resolution. Cell- and tissue-type model outputs are combined into 607 HIFs that quantify specific and biologically-relevant characteristics across five cancer types. We demonstrate that these HIFs correlate with well-known markers of the tumor microenvironment and can predict diverse molecular signatures (AUROC 0.601–0.864), including expression of four immune checkpoint proteins and homologous recombination deficiency, with performance comparable to ‘black-box’ methods. Our HIF-based approach provides a comprehensive, quantitative, and interpretable window into the composition and spatial architecture of the tumor microenvironment.

Deep learning-based tumor microenvironment analysis in colon adenocarcinoma histopathological whole-slide images

Background and Objective: Colon cancer is a fatal disease, and a comprehensive understanding of the tumor microenvironment (TME) could lead to better risk stratification, prognosis prediction, and therapy management. In this paper, we focused on the automatic evaluation of TME in giga-pixel digital histopathology whole-slide images. Methods: A convolutional neural network is used to recognize nine different content presented in colon cancer whole-slide images. Several implementation details, including the foreground filtering and stain normalization are discussed. Based on the whole-slide segmentation, several TME descriptors are quantified and correlated with the clinical outcome by Kaplan-Meier analysis and Cox regression. Specifically, the stroma, tumor, necrosis, and lymphocyte components are discussed. Results: We validated the method on colon adenocarcinoma cases from The Cancer Genome Atlas project. The result shows that the stroma is an independent predictor of progression-free interval (PFI) after corrected by age and pathological stage, with a hazard ratio of 1.665 (95%CI: 1.110~2.495, p = 0.014). High-level necrosis component and lymphocytes component tend to be correlated with poor PFI, with a hazard ratio of 1.552 (95%CI: 0.943~2.554, p = 0.084) and 1.512 (95%CI: 0.979~2.336, p = 0.062), respectively. Conclusions: The result reveals the complex role of the tumor microenvironment in colon adenocarcinoma, and the quantified descriptors are potential predictors of disease progression. The method could be considered for risk stratification and targeted therapy and extend to other types of cancer, leading to a better understanding of the tumor microenvironment.

Resource-frugal classification and analysis of pathology slides using image entropy

Pathology slides of lung malignancies are classified using resource-frugal convolution neural networks (CNNs) that may be deployed on mobile devices. In particular, the challenging task of distinguishing adenocarcinoma (LUAD) and squamous-cell carcinoma (LUSC) lung cancer subtypes is approached in two stages. First, whole-slide histopathology images are downsampled to a size too large for CNN analysis but large enough to retain key anatomic detail. The downsampled images are decomposed into smaller square tiles, which are sifted based on their image entropies. A lightweight CNN produces tile-level classifications that are aggregated to classify the slide. The resulting accuracies are comparable to those obtained with much more complex CNNs and larger training sets. To allow clinicians to visually assess the basis for the classification — that is, to see the image regions that underlie it — color-coded probability maps are created by overlapping tiles and averaging the tile-level probabilities at a pixel level.

Machine Learning Based Classification from Whole-Slide Histopathological Images Enables Reliable and Interpretable Diagnosis of Inverted Urothelial Papilloma

Inverted urothelial papilloma (IUP) is a benign neoplasm of the urinary tract that accounts for less than 1% of urothelial tumors. It is diagnostically challenging for pathologists to distinguish histological features of IUP from other subtypes of non-invasive urothelial carcinoma, such as inverted Ta urothelial carcinoma (UCInv) and low-grade Ta urothelial carcinoma (UCLG). Using a machine learning approach, we analyzed the H&E-stained whole-slide histopathological images of 64 IUP (the largest cohort to date), 69 UCInv, and 92 UCLG samples, and propose a reliable, reproducible, and interpretable machine learning pipeline to classify IUP from other non-invasive urothelial carcinomas. The results showed that our method could achieve area under the ROC of 0.913 and 0.920 for classifying IUP from UCInv and UCLG, respectively, which is superior to competing methods, including deep learning-based methods. Testing of the classification models on an external validation dataset confirmed that our model can effectively identify IUP with high accuracy. Our results suggest that the proposed machine learning pipeline can robustly and accurately capture histopathological differences between IUP and other urothelial carcinoma subtypes, which can be extended to identify other rare cancer subtypes with limited samples and has the potential to expand the clinician’s armamentarium for accurate diagnosis. Funding: This work was supported in part by Indiana Precision Health Initiative to Kun Huang and Jie Zhang. Declaration of Interest: None to declare. Ethical Approval: This research was approved by the Institutional Review Boards of participating institutions.

HookNet: Multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide images.

We propose HookNet, a semantic segmentation model for histopathology whole-slide images, which combines context and details via multiple branches of encoder-decoder convolutional neural networks. Concentric patches at multiple resolutions with different fields of view, feed different branches of HookNet, and intermediate representations are combined via a hooking mechanism. We describe a framework to design and train HookNet for achieving high-resolution semantic segmentation and introduce constraints to guarantee pixel-wise alignment in feature maps during hooking. We show the advantages of using HookNet in two histopathology image segmentation tasks where tissue type prediction accuracy strongly depends on contextual information, namely (1) multi-class tissue segmentation in breast cancer and, (2) segmentation of tertiary lymphoid structures and germinal centers in lung cancer. We show the superiority of HookNet when compared with single-resolution U-Net models working at different resolutions as well as with a recently published multi-resolution model for histopathology image segmentation. We have made HookNet publicly available by releasing the source code1 as well as in the form of web-based applications2,3 based on the grand-challenge.org platform.

Multifaceted fused-CNN based scoring of breast cancer whole-slide histopathology images

Automating the scoring of Whole-Slide Images (WSIs) is a challenging task because the search space for selecting region of interest (ROI) is huge due to the very large sizes of WSIs. A Multifaceted Fused-CNN (MF-CNN) and a Hybrid-Descriptor are proposed to develop an integrated scoring system for Breast Cancer histopathology WSIs. Suitable color and textural features are identified to help mitotic count based selection of ROIs at lower resolution. To recognize complex patterns, the MF-CNN considers multiple facets of the input image. It counts mitoses, extracts handcrafted features from ROIs and utilizes global texture of the images to form a Hybrid-Descriptor for training a classifier assigning scores to WSIs. The proposed system is evaluated on a publicly available benchmark (TUPAC16) and produced the highest score of 0.582 in terms of Cohen’s Kappa. It surpassed human experts’ level accuracy of ROI selection and can therefore reduce the burden of manual ROI selection for WSIs.

Deciphering serous ovarian carcinoma histopathology and platinum response by convolutional neural networks

BackgroundOvarian cancer causes 151,900 deaths per year worldwide. Treatment and prognosis are primarily determined by the histopathologic interpretation in combination with molecular diagnosis. However, the relationship between histopathology patterns and molecular alterations is not fully understood, and it is difficult to predict patients’ chemotherapy response using the known clinical and histological variables.MethodsWe analyzed the whole-slide histopathology images, RNA-Seq, and proteomics data from 587 primary serous ovarian adenocarcinoma patients and developed a systematic algorithm to integrate histopathology and functional omics findings and to predict patients’ response to platinum-based chemotherapy.ResultsOur convolutional neural networks identified the cancerous regions with areas under the receiver operating characteristic curve (AUCs) > 0.95 and classified tumor grade with AUCs > 0.80. Functional omics analysis revealed that expression levels of proteins participated in innate immune responses and catabolic pathways are associated with tumor grade. Quantitative histopathology analysis successfully stratified patients with different response to platinum-based chemotherapy (P = 0.003).ConclusionsThese results indicated the potential clinical utility of quantitative histopathology evaluation in tumor cell detection and chemotherapy response prediction. The developed algorithm is easily extensible to other tumor types and treatment modalities.

Abstract 2096: A deep learning system to predict disease-specific survival in stage II and stage III colorectal cancer

Abstract Accurate prognosis in colorectal cancer can have important implications for clinical management. Here, we develop a deep learning system (DLS) to first identify invasive cancer and then directly predict disease specific survival (DSS) for stage II and stage III colorectal cancer using only digitized histopathology whole-slide images. The DLS was trained using slides from 1173 stage II and 1266 stage III cases (18,304 total slides) and was evaluated on a held-out test set of 601 stage II and 638 stage III cases (9,340 total slides). The area under the receiver operating characteristic curve (AUC) for 5-year DSS prediction was 68.0 for stage II (95% CI 62.2-73.1) and 65.5 for stage III (95% CI 61.1-70.0). For stage II, 5-year DSS was 64% for DLS-predicted high-risk cases versus 89% for DLS-predicted low-risk cases (upper and lower risk quartiles; p<0.001, log rank test). For stage III, 5-year DSS was 35% for DLS-predicted high-risk cases versus 66% for DLS-predicted low-risk cases (upper and lower risk quartiles; p<0.001, log rank test). In a multivariable Cox model, the DLS prediction remained significantly associated with DSS after adjusting for T-category, N-category, age, gender, tumor grade, and lymphovascular invasion (stage II: adjusted hazard ratio 1.55, 95% CI 1.33-1.81, p<0.0001; stage III: adjusted hazard ratio 1.35, 95% CI (1.21-1.51), p<0.0001). Finally, a combined proportional-hazards model using the DLS along with baseline clinicopathologic information provided better risk prediction than the DLS or baseline information alone, increasing 5-year AUC over the baseline-only model by 8.9 points (95% CI 3.9-13.6) and 5.3 points (95% CI 2.3-8.4) for stages II and III, respectively. Taken together, these findings demonstrate that the DLS provides significant prognostic value and risk stratification in both stage II and stage III colorectal cancer, and can be combined with known risk features to further improve prognostic accuracy. This represents novel work to train a DLS to directly predict patient outcomes using whole-slide images and weakly supervised learning. The ability to use non-annotated slides as input has important implications for possible clinical applications and the features learned by the model may also help to identify new prognosis-associated morphologic factors in colorectal cancer. Additional work is ongoing to confirm the utility of these findings, such as validation in additional datasets and interpretability experiments to better understand the features learned by the DLS for these predictions. Citation Format: Ellery Wulczyn, David F. Steiner, Melissa Moran, Markus Plass, Robert Reihs, Heimo Mueller, Apaar Sadhwani, Yuannan Cai, Isabelle Flament, Po-Hsuan Cameron Chen, Yun Liu, Martin C. Stumpe, Zhaoyang Xu, Kurt Zatloukal, Craig H. Mermel. A deep learning system to predict disease-specific survival in stage II and stage III colorectal cancer [abstract]. In: Proceedings of the Annual Meeting of the American Association for Cancer Research 2020; 2020 Apr 27-28 and Jun 22-24. Philadelphia (PA): AACR; Cancer Res 2020;80(16 Suppl):Abstract nr 2096.