Digital Pathology Images Research Articles

Glioma Grading via Analysis of Digital Pathology Images Using Machine Learning.

Cancer pathology reflects disease progression (or regression) and associated molecular characteristics, and provides rich phenotypic information that is predictive of cancer grade and has potential implications in treatment planning and prognosis. According to the remarkable performance of computational approaches in the digital pathology domain, we hypothesized that machine learning can help to distinguish low-grade gliomas (LGG) from high-grade gliomas (HGG) by exploiting the rich phenotypic information that reflects the microvascular proliferation level, mitotic activity, presence of necrosis, and nuclear atypia present in digital pathology images. A set of 735 whole-slide digital pathology images of glioma patients (median age: 49.65 years, male: 427, female: 308, median survival: 761.26 days) were obtained from TCGA. Sub-images that contained a viable tumor area, showing sufficient histologic characteristics, and that did not have any staining artifact were extracted. Several clinical measures and imaging features, including conventional (intensity, morphology) and advanced textures features (gray-level co-occurrence matrix and gray-level run-length matrix), extracted from the sub-images were further used for training the support vector machine model with linear configuration. We sought to evaluate the combined effect of conventional imaging, clinical, and texture features by assessing the predictive value of each feature type and their combinations through a predictive classifier. The texture features were successfully validated on the glioma patients in 10-fold cross-validation (accuracy = 75.12%, AUC = 0.652). The addition of texture features to clinical and conventional imaging features improved grade prediction compared to the models trained on clinical and conventional imaging features alone (p = 0.045 and p = 0.032 for conventional imaging features and texture features, respectively). The integration of imaging, texture, and clinical features yielded a significant improvement in accuracy, supporting the synergistic value of these features in the predictive model. The findings suggest that the texture features, when combined with conventional imaging and clinical markers, may provide an objective, accurate, and integrated prediction of glioma grades. The proposed digital pathology imaging-based marker may help to (i) stratify patients into clinical trials, (ii) select patients for targeted therapies, and (iii) personalize treatment planning on an individual person basis.

Abstract P5-02-02: Artificial intelligence powered predictive analysis of atypical ductal hyperplasia from digitized pathology images

Abstract Background: Approximately 15-25% of patients with atypical ductal hyperplasia (ADH) diagnosed on breast core needle biopsy (CNB) are upgraded to ductal carcinoma in situ (DCIS) or invasive carcinoma (IC) on surgical excision. The reproducible identification of patients with ADH on CNB who are more likely to have upgrades at excision remains elusive. We hypothesized that a machine learning approach could be utilized to train models to recognize ADH on digitized pathology images and to identify cases of ADH more likely to be upgraded to DCIS or IC at excision. The purpose of this study was to determine the accuracy of the machine learning approach to identify ADH. Methods: 726 digitized images of CNB slides derived from 306 cases with a diagnosis of ADH between 11/2004-3/2018 were included in this study. Independent histologic review by two breast pathologists identified slides with and without ADH from each case. 39 board certified pathologists with experience in evaluation of breast biopsies were employed for tissue region annotation on the PathAI research platform (not intended for diagnostic purposes), yielding 14,118 tissue region annotations. Region annotations included ADH, ADH stroma, flat epithelial atypia (FEA), lobular neoplasia (LN), calcifications (Ca), columnar cell change/hyperplasia, sclerosing adenosis, papilloma, normal terminal duct lobular units and other non-atypical breast tissue regions. These annotations were used to train a convolutional neural network (CNN) with 35 layers and approximately 9 million parameters to identify ADH. The data were split into training and testing sets, representing 61.1% and 38.9% of the data respectively. The distribution of cases, images with ADH and cases with upgrade were balanced between the training and testing sets. Results: CNB specimens were assigned labels of “ADH” or “No ADH” based on histologic assessment. AI models were able to predict the diagnosis of ADH with 85% sensitivity (144 of 168 images within the test set) and 69% specificity (78 of 113 images within the test set). The slide-level area under the receiver operator curve (ROC) for this model was 0.84. Conclusions: A deep learning-based classifier showed strong performance for the identification of ADH from whole slide images of H&E stained breast CNBs. With further development, this approach may improve the reproducibility and standardization of the diagnosis of ADH. Future analyses will focus on determining if morphologic features of ADH extracted by the deep learning system can be used to predict upgrade to DCIS and IC. This approach may help stratify patients with ADH on CNB into those who require surgical excision and those who can be followed with active surveillance. Citation Format: Jennifer K. Kerner, Allison Cleary, Suyog Jain, Harsha Pokkalla, Benjamin Glass, Sam Grossmith, Maya Harary, Elizabeth Mittendorf, Andrew H. Beck, Aditya Khosla, Stuart J. Schnitt, Ilan Wapinski, Tari King. Artificial intelligence powered predictive analysis of atypical ductal hyperplasia from digitized pathology images [abstract]. In: Proceedings of the 2019 San Antonio Breast Cancer Symposium; 2019 Dec 10-14; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2020;80(4 Suppl):Abstract nr P5-02-02.

Development of Intraoperative Rapid and Double-checking Support Tools for Pathological Diagnosis of Rare Cancers by Pathological Image Recognition using Digital Pathological Images and Artificial Intelligence Technology

Development of Intraoperative Rapid and Double-checking Support Tools for Pathological Diagnosis of Rare Cancers by Pathological Image Recognition using Digital Pathological Images and Artificial Intelligence Technology

A deep-learning-based prognostic nomogram integrating microscopic digital pathology and macroscopic magnetic resonance images in nasopharyngeal carcinoma: a multi-cohort study.

Background:To explore the prognostic value of radiomics-based and digital pathology-based imaging biomarkers from macroscopic magnetic resonance imaging (MRI) and microscopic whole-slide images for patients with nasopharyngeal carcinoma (NPC).Methods:We recruited 220 NPC patients and divided them into training (n = 132), internal test (n = 44), and external test (n = 44) cohorts. The primary endpoint was failure-free survival (FFS). Radiomic features were extracted from pretreatment MRI and selected and integrated into a radiomic signature. The histopathological signature was extracted from whole-slide images of biopsy specimens using an end-to-end deep-learning method. Incorporating two signatures and independent clinical factors, a multi-scale nomogram was constructed. We also tested the correlation between the key imaging features and genetic alternations in an independent cohort of 16 patients (biological test cohort).Results:Both radiomic and histopathologic signatures presented significant associations with treatment failure in the three cohorts (C-index: 0.689–0.779, all p < 0.050). The multi-scale nomogram showed a consistent significant improvement for predicting treatment failure compared with the clinical model in the training (C-index: 0.817 versus 0.730, p < 0.050), internal test (C-index: 0.828 versus 0.602, p < 0.050) and external test (C-index: 0.834 versus 0.679, p < 0.050) cohorts. Furthermore, patients were stratified successfully into two groups with distinguishable prognosis (log-rank p < 0.0010) using our nomogram. We also found that two texture features were related to the genetic alternations of chromatin remodeling pathways in another independent cohort.Conclusion:The multi-scale imaging features showed a complementary value in prognostic prediction and may improve individualized treatment in NPC.

Evolution of Telepathology: A Comprehensive Analysis of Global Telepathology Literature Between 1986 and 2017.

Telepathology is an application of telemedicine providing remote evaluation and consultation of digital pathology images and can be used for educational or experimental purposes. Bibliometrics is a statistical discipline investigating publication patterns and trends in a certain academic field. Although bibliometric and scientometric studies are becoming increasingly popular, the relevant literature contains only one limited article related to telepathology. The aim of our study was to perform a holistic bibliometric analysis of the telepathology literature. Since the first article on telepathology was published in 1986, we included all indexed articles retrieved from Web of Science databases between 1986 and 2017. We found that the USA covering 43.01% of all literature was the leading country in the telepathology field and was followed by Germany, Italy and the UK (n=120, 90 and 83, respectively). The countries with the most contributions were located in the continents of Europe and North America. The most productive source titles were Human Pathology, Journal of Telemedicine and Telecare, and Modern Pathology. Harvard University ranked first with 59 articles. The most commonly used keywords of the telepathology literature were "telepathology", "telemedicine", "digital pathology", "virtual microscopy" and "telecytology". We noted that all of the ten countries with the most contributions were in the developed category of UN classification and all twenty of the most productive institutions were from developed countries. We suggest that researchers from developing and least developed countries should be encouraged to carry out novel studies since telemedicine is a required and promising technology for rural developing or least developed areas in which access to health care is difficult.

Inside the USCAP Journals

Inside the USCAP Journals

NIMG-76. RADIOPATHOMICS: INTEGRATION OF RADIOGRAPHIC AND HISTOLOGIC CHARACTERISTICS FOR PROGNOSTICATION IN GLIOBLASTOMA

Abstract INTRODUCTION Large number of diverse imaging [e.g., multi-parametric MRI (mpMRI), and digital pathology images] and non-imaging (e.g., clinical) biomedical data streams are being routinely acquired as part of the standard clinical workflow for glioblastoma patients. However, under the current clinical practice, these data streams are not collectively used for diagnosis. We sought to assess the synergies between pathologic, and radiomic features by evaluating the predictive value of each group of features and their combinations through a prognostic classifier. METHODS The mpMRI (T1,T1-Gd,T2,T2-FLAIR) and corresponding digital pathology images for 135 de novo glioblastoma was acquired from TCIA. An extensive panel of handcrafted features, including shape, volume, intensity distributions, gray-level co-occurrence matrix based texture, was extracted from delineated tumor regions of mpMRI scans. A set of 100 region-of-interest each comprising 1024x1024 that contained viable tumor with descriptive histologic characteristics and that were free of artifacts were extracted from digital pathology images, and were quantified in terms of nuclear texture features, and nuclear intensity and gradient statistics. A support vector regression multivariately integrated these features towards a marker of overall-survival. The accuracy of the predictive model for each group of features, and their combinations, was determined via a 10-fold cross-validation scheme. RESULTS The Pearson correlation coefficient between the survival scores predicted by SVR and the actual survival scores was estimated to be 0.75 and 0.77 for radiographic and pathologic data, however, the integration of these data yielded a clear improvement in correlation (0.81), supporting the synergistic value of these features in the prognostic model. CONCLUSION Radiomic features extracted from preoperative mpMRI, when used together with digital pathology features, offer synergistic value in assessment of prognosis in individual patients. The proposed radiopathomics marker may contribute to (i) stratification of patients into clinical trials, (ii) patient selection for targeted therapy, and (iii) personalized treatment planning.

Segmentation and Grade Prediction of Colon Cancer Digital Pathology Images Across Multiple Institutions.

Distinguishing benign from malignant disease is a primary challenge for colon histopathologists. Current clinical methods rely on qualitative visual analysis of features such as glandular architecture and size that exist on a continuum from benign to malignant. Consequently, discordance between histopathologists is common. To provide more reliable analysis of colon specimens, we propose an end-to-end computational pathology pipeline that encompasses gland segmentation, cancer detection, and then further breaking down the malignant samples into different cancer grades. We propose a multi-step gland segmentation method, which models tissue components as ellipsoids. For cancer detection/grading, we encode cellular morphology, spatial architectural patterns of glands, and texture by extracting multi-scale features: (i) Gland-based: extracted from individual glands, (ii) local-patch-based: computed from randomly-selected image patches, and (iii) image-based: extracted from images, and employ a hierarchical ensemble-classification method. Using two datasets (Rawalpindi Medical College (RMC), n = 174 and gland segmentation (GlaS), n = 165) with three cancer grades, our method reliably delineated gland regions (RMC = 87.5%, GlaS = 88.4%), detected the presence of malignancy (RMC = 97.6%, GlaS = 98.3%), and predicted tumor grade (RMC = 98.6%, GlaS = 98.6%). Training the model using one dataset and testing it on the other showed strong concordance in cancer detection (Train RMC – Test GlaS = 94.5%, Train GlaS – Test RMC = 93.7%) and grading (Train RMC – Test GlaS = 95%, Train GlaS – Test RMC = 95%) suggesting that the model will be applicable across institutions. With further prospective validation, the techniques demonstrated here may provide a reproducible and easily accessible method to standardize analysis of colon cancer specimens.

IHC Color Histograms for Unsupervised Ki67 Proliferation Index Calculation.

Automated image analysis tools for Ki67 breast cancer digital pathology images would have significant value if integrated into diagnostic pathology workflows. Such tools would reduce the workload of pathologists, while improving efficiency, and accuracy. Developing tools that are robust and reliable to multicentre data is challenging, however, differences in staining protocols, digitization equipment, staining compounds, and slide preparation can create variabilities in image quality and color across digital pathology datasets. In this work, a novel unsupervised color separation framework based on the IHC color histogram (IHCCH) is proposed for the robust analysis of Ki67 and hematoxylin stained images in multicentre datasets. An “overstaining” threshold is implemented to adjust for background overstaining, and an automated nuclei radius estimator is designed to improve nuclei detection. Proliferation index and F1 scores were compared between the proposed method and manually labeled ground truth data for 30 TMA cores that have ground truths for Ki67+ and Ki67− nuclei. The method accurately quantified the PI over the dataset, with an average proliferation index difference of 3.25%. To ensure the method generalizes to new, diverse datasets, 50 Ki67 TMAs from the Protein Atlas were used to test the validated approach. As the ground truth for this dataset is PI ranges, the automated result was compared to the PI range. The proposed method correctly classified 74 out of 80 TMA images, resulting in a 92.5% accuracy. In addition to these validations experiments, performance was compared to two color-deconvolution based methods, and to six machine learning classifiers. In all cases, the proposed work maintained more consistent (reproducible) results, and higher PI quantification accuracy.

Towards pixel-to-pixel deep nucleus detection in microscopy images

BackgroundNucleus or cell detection is a fundamental task in microscopy image analysis and supports many other quantitative studies such as object counting, segmentation, tracking, etc. Deep neural networks are emerging as a powerful tool for biomedical image computing; in particular, convolutional neural networks have been widely applied to nucleus/cell detection in microscopy images. However, almost all models are tailored for specific datasets and their applicability to other microscopy image data remains unknown. Some existing studies casually learn and evaluate deep neural networks on multiple microscopy datasets, but there are still several critical, open questions to be addressed.ResultsWe analyze the applicability of deep models specifically for nucleus detection across a wide variety of microscopy image data. More specifically, we present a fully convolutional network-based regression model and extensively evaluate it on large-scale digital pathology and microscopy image datasets, which consist of 23 organs (or cancer diseases) and come from multiple institutions. We demonstrate that for a specific target dataset, training with images from the same types of organs might be usually necessary for nucleus detection. Although the images can be visually similar due to the same staining technique and imaging protocol, deep models learned with images from different organs might not deliver desirable results and would require model fine-tuning to be on a par with those trained with target data. We also observe that training with a mixture of target and other/non-target data does not always mean a higher accuracy of nucleus detection, and it might require proper data manipulation during model training to achieve good performance.ConclusionsWe conduct a systematic case study on deep models for nucleus detection in a wide variety of microscopy images, aiming to address several important but previously understudied questions. We present and extensively evaluate an end-to-end, pixel-to-pixel fully convolutional regression network and report a few significant findings, some of which might have not been reported in previous studies. The model performance analysis and observations would be helpful to nucleus detection in microscopy images.

Proliferation Tumour Marker Network (PTM-NET) for the identification of tumour region in Ki67 stained breast cancer whole slide images

Uncontrolled proliferation is a hallmark of cancer and can be assessed by labelling breast tissue using immunohistochemistry for Ki67, a protein associated with cell proliferation. Accurate measurement of Ki67-positive tumour nuclei is of critical importance, but requires annotation of the tumour regions by a pathologist. This manual annotation process is highly subjective, time-consuming and subject to inter- and intra-annotator experience. To address this challenge, we have developed Proliferation Tumour Marker Network (PTM-NET), a deep learning model that objectively annotates the tumour regions in Ki67-labelled breast cancer digital pathology images using a convolution neural network. Our custom designed deep learning model was trained on 45 immunohistochemical Ki67-labelled whole slide images to classify tumour and non-tumour regions and was validated on 45 whole slide images from two different sources that were stained using different protocols. Our results show a Dice coefficient of 0.74, positive predictive value of 70% and negative predictive value of 88.3% against the manual ground truth annotation for the combined dataset. There were minimal differences between the images from different sources and the model was further tested in oestrogen receptor and progesterone receptor-labelled images. Finally, using an extension of the model, we could identify possible hotspot regions of high proliferation within the tumour. In the future, this approach could be useful in identifying tumour regions in biopsy samples and tissue microarray images.

Canadian Association of Neuropathologists L’Association Canadienne des Neuropathologistes

Abstracts and unknown cases presented at the 58th annual meetingThe Canadian Association of Neuropathologists (CANP) held their 58th annual meeting at the Halifax Harbourfront Marriott from October 3rd to 6th, 2018, under the leadership of Dr Marc Del Bigio, CANP president, and Dr. Sidney Croul handled the local arrangements. The annual banquet was held at Chives restaurant in Halifax.The academic program of 20 scientific abstracts and 8 unknown cases was compiled by the CANP Secretary-Treasurer Dr Julia Keith into 5 sessions; I. Traumatic and Neurodegenerative Neuropathology, II. Pediatric, Epilepsy and Miscellaneous Neuropathology, III. Tumour Neuropathology, IV. Infectious/Immune Mediated Neuropathology, and V. Quality Assurance in Neuropathology. The Quality Assurance session included a guided interactive forum led by Dr Marc Del Bigio, and the Neurodegenerative session included an invited talk on Butyrylcholinesterase as a Biomarker for Alzheimer’s Disease by Dr Sultan Darvesh (Faculty of Medicine, Dalhousie University). The digital pathology images from the 8 unknown cases are available for viewing online (www.canp.ca) thanks to the CANP webmaster, Dr Jason Karamchandani.The Presidential Symposium explored the theme of Brain Inflammation. Dr Sean Pittock (Dept of Neurology, Mayo Clinic, Rochester, MN) delivered the Dr Jerzy Olszewski Guest Lecture entitled Neural autoantibody discovery: the new era of autoimmune neurology – out with the old and in with the new! Dr Christian Pagnoux (Vasculitis Clinic, Mount Sinai Hospital, Toronto ON) was an invited speaker who spoke on Vasculitis – effects on the nervous system. Dr Alex Easton (Dept of Pathology, Dalhousie University, Halifax NS) was the Dr Gordon Mathieson invited lecturer speaking about The blood brain barrier and inflammation, and Dr Serge Rivest (Directeur du Centre de Recherche du CHU de Québec, Université Laval, Quebec QC) delivered the Dr David Robertson invited lecture on Microglial roles in human neuropathology. The symposium concluded with a panel discussion and group Question and Answer session with the speakers moderated by Dr Marc Del Bigio.Several trainee awards were handed out. The Dr Mary Tom Award for best clinical science paper was awarded to Dr Maryam Abdollahi (supervisor Dr David Munoz) and the Dr Morrison H. Finlayson Award for best basic science/research paper went to Dr Maliha Khara (Drs Zhou, Wong, Renwick).The following abstracts were presented at the 58th annual meeting of the Canadian Association of Neuropathologists (CANP) in October 2018.

A BAYESIAN MARK INTERACTION MODEL FOR ANALYSIS OF TUMOR PATHOLOGY IMAGES.

With the advance of imaging technology, digital pathology imaging of tumor tissue slides is becoming a routine clinical procedure for cancer diagnosis. This process produces massive imaging data that capture histological details in high resolution. Recent developments in deep-learning methods have enabled us to identify and classify individual cells from digital pathology images at large scale. Reliable statistical approaches to model the spatial pattern of cells can provide new insight into tumor progression and shed light on the biological mechanisms of cancer. We consider the problem of modeling spatial correlations among three commonly seen cells observed in tumor pathology images. A novel geostatistical marking model with interpretable underlying parameters is proposed in a Bayesian framework. We use auxiliary variable MCMC algorithms to sample from the posterior distribution with an intractable normalizing constant. We demonstrate how this model-based analysis can lead to sharper inferences than ordinary exploratory analyses, by means of application to three benchmark datasets and a case study on the pathology images of 188 lung cancer patients. The case study shows that the spatial correlation between tumor and stromal cells predicts patient prognosis. This statistical methodology not only presents a new model for characterizing spatial correlations in a multitype spatial point pattern conditioning on the locations of the points, but also provides a new perspective for understanding the role of cell-cell interactions in cancer progression.

Tumor Infiltrating Lymphocytes in Occult Primary HPV+ Oropharyngeal Squamous Cell Carcinoma (OPSCC): Comparison of the Primary Tumor and Regional Lymph Node Metastases

A subset of patients with HPV-associated OPSCC present with small radiographically undetectable primary tumors and relatively bulky regional nodal disease. We hypothesized that this pattern of presentation may reflect differential immune infiltration into the primary tumor and regional nodes. This analysis sought to characterize the tumor infiltrating lymphocytes (TILs) in paired primary tumors and regional nodes for patients presenting with occult primary HPV+ OPSCC managed with primary site resection and neck dissection. We analyzed 14 patients presenting with initially occult primary HPV+ OPSCC managed with trans-oral robotic surgery (TORS) and neck dissection, with a final TNM stage of pT1 N2b by AJCC 7th edition. Immunohistochemistry (IHC) was performed on paired specimens from the primary site and involved nodes for CD3, CD8, FOXP3, PD-L1, and CTLA-4. Quantification of TILs was performed using QuPath, a standard open source software for digital pathology image analysis. IHC results are reported as the percentage of positive cells divided by total cells in the tumor area. Statistical analysis of primary and nodal TILs was performed using the paired t-test. Occult primary tumors had a higher percentage of CD3+ and CD8+ TILs compared to tumor in lymph nodes: 61% versus 42% for CD3+ (p = 0.006), and 36% versus 23% for CD8+ T cells, respectively (p = 0.01). There was a higher concentration of FOXP3+ TILs in primary tumors compared to nodal tumor: 8% versus 5% (p = 0.01), and a trend towards a higher percentage of CTLA-4+ cells (52% versus 34%, p = 0.09). Of note, in many specimens we observed strong expression of CTLA-4 on tumor cells and not only on TILs, explaining the high percentage of cells staining positive for CTLA-4. There was no detectable difference in PD-L1 expression (15% versus 11%, NS). Occult primary HPV+ OPSCC tumors have a higher concentration of CD3+ and CD8+ TILs compared to their associated regional lymph node metastases, potentially accounting for the clinical presentation of these patients. The increased concentration of CD8+ T cells, in conjunction with the observed increase in FOXP3+ regulatory T cells and CTLA-4+ cells, suggests a T-cell inflamed phenotype at the primary site. By contrast, the lymph nodes appear to be relatively immunologically “cold.” Efforts are underway to characterize the interferon signature at the primary site and involved nodes, and identify potential mechanisms of immune escape in the nodes. Differences in the tumor microenvironment between the primary site and metastatic nodes may explain the somewhat unique clinical presentation of HPV+ HNSCC, and may have implications for mechanisms of distant metastatic spread and treatment with immunotherapy.

Complete Digital Pathology for Routine Histopathology Diagnosis in a Multicenter Hospital Network.

Complete digital pathology and whole slide imaging for routine histopathology diagnosis is currently in use in few laboratories worldwide. Granada University Hospitals, Spain, which comprises 4 hospitals, adopted full digital pathology for primary histopathology diagnosis in 2016. To describe the methodology adopted and the resulting experience at Granada University Hospitals in transitioning to full digital diagnosis. All histopathology glass slides generated for routine diagnosis were digitized at ×40 using the Philips IntelliSite Pathology Solution, which includes an ultrafast scanner and an image management system. All hematoxylin-eosin-stained preparations and immunohistochemistry and histochemistry slides were digitized. The existing sample-tracking software and image management system were integrated to allow data interchange through the Health Level 7 protocol. Circa 160 000 specimens have been signed out using digital pathology for primary diagnosis. This comprises more than 800 000 digitized glass slides. The scanning error rate during the implementation phase was below 1.5%, and subsequent workflow optimization rendered this rate negligible. Since implementation, Granada University Hospitals pathologists have signed out 21% more cases per year on average. Digital pathology is an adequate medium for primary histopathology diagnosis. Successful digitization relies on existing sample tracking and integration of the information technology infrastructure. Rapid and reliable scanning at ×40 equivalent was key to the transition to a fully digital workflow. Digital pathology resulted in efficiency gains in the preanalytical and analytical phases, and created the basis for computational pathology: the use of computer-assisted tools to aid diagnosis.

A Machine Learning Based Risk Stratification Model Could Predict Recurrence in Early-Stage Colon Carcinoma via Digital Pathology Images: A Double-Center Study

A Machine Learning Based Risk Stratification Model Could Predict Recurrence in Early-Stage Colon Carcinoma via Digital Pathology Images: A Double-Center Study

Abstract 1648: Deep learning tissue "fingerprints" to identify patients and predict clinical subtypes of breast cancer from digital pathology images

Abstract Inspired by facial recognition, we define an analogous problem in digital pathology called tissue matching. The objective is to match histology image patches to the patient. Unlike many classification problems in pathology, tissue matching comes with its own ground truth, and does not require subjective annotation. Using tissue-matching via deep neural networks, we developed strategies to overcome common sources of noise in histology data, including H&amp;E and slide scanner variability. We demonstrate that the typical approach of increasing the training set size, in the current study up to 12,000 tissue cores, yields only moderate gains in test accuracy, while including a loss term promoting style-invariance cooperatively improves the impact of scaling training set size. Our best neural network learned features that matched patients from the test set of H&amp;E image patches with 93% accuracy (n=104 patients, baseline accuracy &amp;lt;1%). Leveraging this network’s identification ability, we applied its internal representation as a feature vector (tissue fingerprint) to apply to other problems in pathology. We found that a fingerprint-based classifier could predict estrogen receptor status from breast cancer whole-slides on two separate cohorts with high accuracy (AUCROC=0.89), a sizeable improvement over previous approaches. This work presents a novel, scalable, and transparent approach to train neural networks to identify distinctive features of pathology images. These features have demonstrated utility in classifying biomarkers from morphology and have the potential to assist in a number of difficult tasks in pathology, including tissue subtype classification and patient tumor similarity assessments based on morphometry. Citation Format: Rishi Rawat, Fei Sha, Darryl Shibata, Daniel Ruderman, David Agus. Deep learning tissue "fingerprints" to identify patients and predict clinical subtypes of breast cancer from digital pathology images [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 1648.

Abstract 1184: Comprehensive analysis with interactive exploration of immune response signatures in 10,000 tumor samples

Abstract In recent years, analysis of cancer genomics data using methods of immunogenomics has yielded valuable insight into how cancer cells interact with immune cells in the tumor microenvironment. A recent analysis of the multiple molecular platforms by The Cancer Genome Atlas (TCGA) of over 10,000 tumors comprising 33 cancer types identified six immune subtypes, spanning multiple tumor types, that are characterized by differences in: macrophage vs. lymphocyte signatures; Th1:Th2 cell ratio; extent of intratumoral heterogeneity; aneuploidy; extent of neoantigen load; signatures of cell proliferation; expression of immunomodulatory genes; and disease outcome [1]. Particular driver mutations correlate with variation in leukocyte levels across all cancers or with the fraction of individual immune cell types. Intracellular and extracellular networks (involving transcription, microRNAs, copy number and epigenetic processes) are predicted to play a role in establishing the observed tumor-immune cell interactions, both across and within immune subtypes. Additionally, machine learning methods have been applied to H&amp;E images to extract information on which tissue regions contain tumor infiltrating lymphocytes (TILs), yielding TIL maps of whole slide images from digital pathology[2]. Spatial patterns of TILs are associated with a variety of genomic alterations, including cancer subtypes. The CRI iAtlas (www.cri-iatlas.org) is a cloud-based platform for data exploration and discovery, allowing researchers to study TCGA immune response characterizations, and the relationships among them in individual tumor types, tumor subtypes, and immune subtypes. iAtlas supports the adaptive exploration of correlations within the cellularity of the tumor microenvironment, immune expression signatures, tumor mutation burden, cancer driver mutations, adaptive cell clonality, patient survival, and expression of key immunomodulators. iAtlas was launched in April 2018, and has since been expanded to include new capabilities such as (1) user-defined loading of cohorts, (2) a tool for classifying expression data into immune subtypes, (3) integration of TIL mapping from digital pathology images, and (4) addition of annotated genomics datasets from immunotherapy clinical trials as comparative data sources. As the resource evolves, we expect that the CRI iAtlas will help to accelerate discovery and improve patient outcomes by providing researchers greater access to immunogenomics data to better understand the immunological characteristics of the tumor microenvironment and its potential impact on patient responses to immunotherapy. [1] Thorsson, V, et al., The Immune Landscape of Cancer; Immunity 48, p812 - 830.e14 (2018) [2] Saltz, J et al. Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images; Cell Reports 23 pp.181-193.e7 (2018) Citation Format: Vesteinn Thorsson, David L. Gibbs, Mary L. Disis, Elizabeth G. Demicco, Alexander J. Lazar, Jonathan S. Serody, James A. Eddy, Ilya Shmulevich, Justin Guinney, Benjamin G. Vincent. Comprehensive analysis with interactive exploration of immune response signatures in 10,000 tumor samples [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 1184.

Abstract 3130: Using digital pathology based “IO Scorecards” to describe relationships between PD-L1 expression and CD8 positive immune cell infiltration

Abstract Several studies have shown that the location and expression of infiltrating immune cells in patient tumors can better identify which patients are more likely to respond to anti- PD-1/PD-L1 therapy. In particular, immunohistochemistry-based studies have shown that the spatial location of PD-L1 expression has particular biological relevance, as PD-L1 expression in the tumor cells or immune cells in the tumor syncytium, tumor microenvionment (TME), or tumor-stroma boundary all describe differential PD-L1 biology. Here, we use Flagship’s digital pathology platform (cTA®) to investigate IHC based PD-L1 and CD8 staining patterns in Non-Small Cell Lung (NSCLC) and Urothelial Carcinoma (UC) tissue biopsies. The cTA platform creates thousands of per-cell Biofeatures™ derived from the digital pathology images of the IHC stained tissue, and applies Artificial Intelligence (AI) to the data to summary score endpoints for patient and cohort classification. In this approach, each tissue’s IO landscape is represented using an “IO Scorecard“, which summarizes the IHC biomarker data in a summary score which captures a comprehensive analysis of the tissue sample. The AI-determined scorecard models can be used to monitor changes before and after drug treatment and/or create predictive models for patient response outcomes. In this study, NSCLC and Urothelial Carcinoma samples were sectioned and stained using either the FDA-approved Dako 22C3 or SP263 PD-L1 IHC assays. Serial sections of each tissue specimen were also stained for CD8 expression. The cTA process detected all cells, assigned them to the tumor or TME compartments, and recorded the Biofeatures™ data which characterized PD-L1 or CD8 staining in the Tumor, Tumor/TME margin, or TME compartments. The method was validated by its ability to reproduce pathologist scoring for PD-L1 and CD8. Using the accepted data, we used the AI system to create novel Scorecard based patient classifications which describe differential PD-L1 status, CD8 status, and PD-L1/CD8 status combined. The AI Scorecard approach demonstrated that certain PD-L1 staining Biofeatures™ may also predict the CD8 status of a tumor, suggesting that additional CD8 staining may not be necessary to understand important expression patterns pertaining to cytotoxic T-cells. In summary, we demonstrated how the “IO Scorecards” are able to classify patients into differential immune status cohorts using a novel AI based scoring system, which relies only on PD-L1 IHC staining, by creating a comprehensive, contextual profile of PD-L1 staining that does not require additional CD8 IHC staining to characterize the impact of cytotoxic T-cells in a tissue sample. Citation Format: Charles Caldwell, Will Paces, Jeni Caldara, Bharathi Vennapusa, Joseph S. Krueger. Using digital pathology based “IO Scorecards” to describe relationships between PD-L1 expression and CD8 positive immune cell infiltration [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 3130.

Cancer Image Quantification With And Without, Expensive Whole Slide Imaging Scanners.

Automated analysis of digitized pathology images in tele-health applications can have a transformative impact on under-served communities in the developing world. However, the vast majority of existing image analysis algorithms are trained on slide images acquired via expensive Whole-Slide-Imaging (WSI) scanners. High scanner cost is a key bottleneck preventing large-scale adoption of digital pathology in developing countries. In this work, we investigate the viability of automated analysis of slide images captured from the eyepiece of a microscope via a smart phone. The mitosis detection application is considered as a use case.Results indicate performance degradation when using (lower-quality) smartphone images; as expected. However, the performance gap is not too wide (F1-score smartphone=0.65, F1-score WSI=0.70) demonstrating that smartphones could potentially be employed as image acquisition devices for digital pathology at locations where expensive scanners are not available.