Histopathological Slide Images Research Articles

Abstract PO4-07-05: Clinical evaluation of image-based risk profiling in breast cancer histopathology and comparison to an established gene expression assay

Abstract Background: Among oestrogen receptor (ER)-positive and human epidermal growth factor receptor 2 (HER2)-negative early breast cancer, a significant proportion of patients are categorised as intermediate risk, based on classic clinico-pathological variables, thus providing limited information to guide adjuvant chemotherapy decisions. Prognostic risk profiling is an integrated part of modern breast cancer diagnostics to provide additional risk information for this patient group. Among the established prognostic assays based on gene expression, the Prosigna assay is widely used and provides an individual risk of recurrence (ROR) score and identifies intrinsic subtypes with associated survival outcomes. Pioneering artificial intelligence-based precision diagnostics, Stratipath Breast, is a deep learning-based image analysis tool that utilises digitised histopathological whole slide images to stratify intermediate risk patients in terms of risk of recurrence. Materials and methods: The study included 234 invasive breast tumours from patients with early ER-positive HER2-negative breast cancer, clinically assessed as intermediate risk tumours and eligible for chemotherapy. All tumours had therefore previously been analysed by the Prosigna assay in clinical routine at point of diagnosis between 2020 and 2022 at the Karolinska University Hospital and Södersjukhuset, Stockholm, Sweden. Clinicopathological data including Prosigna results (ROR score, risk group and intrinsic subtype) were extracted from medical records, along with the corresponding archived haematoxylin and eosin (HE)-stained formalin-fixed paraffin-embedded tissue slides. The HE slides were subsequently digitised and analysed by the Stratipath Breast tool. The agreement between the two tests for risk stratification was evaluated in this real-world breast cancer case series. Results: The Prosigna assay classified 76 (32.5%), 110 (47.0%) and 48 (20.5%) tumours as low, intermediate and high risk, respectively. The Stratipath Breast analysis identified 116 (49.6%) tumours as low risk and 118 (50.4%) as high risk. Among Prosigna intermediate risk tumours, 52 (47.3%) were stratified as low risk and 58 (52.7%) as high risk by Stratipath Breast. The overall agreement between the two tests for low risk and high risk groups was 71.0%, with a Cohen’s linear kappa of 0.42. Twelve of the 48 Prosigna high risk cases were classified as Stratipath low risk. ROR scores were higher in the Stratipath high risk group compared to the low risk group (p < 0.001), across all cases as well as in the Prosigna intermediate group. Among the 176 histological grade (NHG) 2 tumours, 97 (55.1%) and 79 (44.9%) were stratified as Stratipath low risk and high risk, respectively, whereas 66 (37.5%), 83 (47.2%) and 27 (15.3%) were stratified as Prosigna low, intermediate and high risk, respectively. The majority of NHG1 (10 of 12) and NHG3 (37 of 46) tumours were stratified as Stratipath low risk and high risk, respectively. For both risk profiling tests, NHG and Ki67 proliferation index differed between risk groups. Conclusions: In this study of clinically assessed intermediate risk ER-positive HER2-negative breast cancer, we observed a moderate agreement between Prosigna and Stratipath Breast for low risk and high risk groups. In addition, image-based risk profiling stratified more of the NHG2 tumours as high risk. Citation Format: Stephanie Robertson, Yinxi Wang, Wenwen Sun, Emelie Karlsson, Sandy Kang Lövgren, Mattias Rantalainen, Johan Hartman. Clinical evaluation of image-based risk profiling in breast cancer histopathology and comparison to an established gene expression assay [abstract]. In: Proceedings of the 2023 San Antonio Breast Cancer Symposium; 2023 Dec 5-9; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2024;84(9 Suppl):Abstract nr PO4-07-05.

Abstract 2584: Decoding colon cancer recurrence: Unveiling accurate predictions with attention-guided deep neural networks on histopathological whole slide images

Abstract Background: Up to 40% of colon cancer patients are at high risk of cancer recurrence, yet accurate and timely prediction tools are lacking. Leveraging whole slide images (WSIs) and deep learning models, we aimed to develop precise algorithm for prediction of colon cancer recurrence. Thus, enables risk stratification for optimized therapeutic interventions and improved health outcomes. Design: We designed an attention-based deep learning model for predicting colon cancer recurrence on paraffin-embedded, hematoxylin and eosin-stained colon tissue biopsies of digital slides. The WSIs were downloaded from the TCGA dataset, and preprocessed for color normalization, tissue segmentation, tilling, and histopathological features extraction. The entire dataset was then labeled based on cancer recurrence status, and divided into training (70%), validation (15%), and testing sets (15%). The model's performance was then evaluated by standard evaluation metrics, including receiver operating characteristic Area Under the Curve (AUC) and accuracy, on the training, validation, and testing datasets, where interpretability attention heatmaps were applied to gain insights into specific histological features and patterns involved in the model's decision-making process. The study is supported by the NIH-NCI-T32 for Next Generation Pathologists Program at our institution. Results: A total of 350 WSIs were included and labeled into two classes: post-therapeutic colon cancer recurrence and no recurrence. The tissue segmentation process involved converting RGB images to HSV color space, applying a median blur filter (kernel size: 7), and performing thresholding using Otsu's method. The extracted features were utilized to train and construct a Clustering-constrained Attention Multiple Instance Learning (CLAM) model. The model demonstrated consistent performance across the validation and testing datasets, achieving an AUC of 0.85 with an accuracy of 0.83 on the validation set, and an identical AUC of 0.85 with an accuracy of 0.83 on the testing set, indicating the model's robust ability to identify patients at risk of cancer recurrence. Conclusion: The study demonstrates the strong performance of our deep learning model in accurately identifying patients at risk for colon cancer recurrence based on H&E WSIs. This capability paves the way for optimizing therapeutic interventions and implementing effective surveillance strategies. Consequently, highlighting the crucial role of pathologists in collaborating with the oncologist for optimizing the management of colon cancer care in the era of personalized medicine. Citation Format: Mohammad K. Alexanderani, Mohamed Omar, Matthew Greenblatt, Ethel Cesarman, Maria T. Diaz-Meco, Jorge Moscat, Luigi Marchionni. Decoding colon cancer recurrence: Unveiling accurate predictions with attention-guided deep neural networks on histopathological whole slide images [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 2584.

Abstract 886: Integrated deep learning model for predicting DNA methylation and tumor types from histopathology in central nervous system tumors

Abstract Precision in diagnosis of diverse central nervous system (CNS) tumor types is crucial for optimal patient treatment. DNA methylation profiles, which capture the methylation status of thousands of individual CpG sites, are data-driven means to enhance diagnostic accuracy, but this technique is expensive, time-consuming, and not yet routinely available. To address this, we developed DEPLOY, a deep-learning model that predicts 10 major categories of CNS tumors from histopathology. DEPLOY integrates three distinct components: the first classifies CNS tumors directly from histopathology slide images (‘direct model’); the second initially generates predictions for DNA methylation beta values, which are subsequently used for tumor classification (‘indirect model’); and the third classifies tumor types directly from routinely available patient demographics. First, we find that DEPLOY accurately predicted beta values from histopathology images, suggesting that application of inferred methylation data is a promising approach for deep learning histopathology-based classifiers. Using a 10-class model trained on an internal dataset, we apply this model on two independent external test datasets of 1,522 and 348 cases, achieving top-1 accuracies of 96% and 94%, respectively on samples which are predicted with high confidence. Taken together, DEPLOY could assist pathologists in diagnosing CNS tumors, in an equitable very low-cost manner, within a clinically relevant short time frame. Citation Format: Eldad David Shulman, Danh-Tai Hoang, Rust Turakulov, Zied Abdullaev, Omkar Singh, Emma M. Campagnolo, Eric A. Stone, MacLean P. Nasrallah, Eytan Ruppin, Kenneth Aldape. Integrated deep learning model for predicting DNA methylation and tumor types from histopathology in central nervous system tumors [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 886.

A multi-class brain tumor grading system based on histopathological images using a hybrid YOLO and RESNET networks

Gliomas are primary brain tumors caused by glial cells. These cancers’ classification and grading are crucial for prognosis and treatment planning. Deep learning (DL) can potentially improve the digital pathology investigation of brain tumors. In this paper, we developed a technique for visualizing a predictive tumor grading model on histopathology pictures to help guide doctors by emphasizing characteristics and heterogeneity in forecasts. The proposed technique is a hybrid model based on YOLOv5 and ResNet50. The function of YOLOv5 is to localize and classify the tumor in large histopathological whole slide images (WSIs). The suggested technique incorporates ResNet into the feature extraction of the YOLOv5 framework, and the detection results show that our hybrid network is effective for identifying brain tumors from histopathological images. Next, we estimate the glioma grades using the extreme gradient boosting classifier. The high-dimensional characteristics and nonlinear interactions present in histopathology images are well-handled by this classifier. DL techniques have been used in previous computer-aided diagnosis systems for brain tumor diagnosis. However, by combining the YOLOv5 and ResNet50 architectures into a hybrid model specifically designed for accurate tumor localization and predictive grading within histopathological WSIs, our study presents a new approach that advances the field. By utilizing the advantages of both models, this creative integration goes beyond traditional techniques to produce improved tumor localization accuracy and thorough feature extraction. Additionally, our method ensures stable training dynamics and strong model performance by integrating ResNet50 into the YOLOv5 framework, addressing concerns about gradient explosion. The proposed technique is tested using the cancer genome atlas dataset. During the experiments, our model outperforms the other standard ways on the same dataset. Our results indicate that the proposed hybrid model substantially impacts tumor subtype discrimination between low-grade glioma (LGG) II and LGG III. With 97.2% of accuracy, 97.8% of precision, 98.6% of sensitivity, and the Dice similarity coefficient of 97%, the proposed model performs well in classifying four grades. These results outperform current approaches for identifying LGG from high-grade glioma and provide competitive performance in classifying four categories of glioma in the literature.

Deep learning model to differentiate Crohn’s disease from intestinal tuberculosis using histopathological whole slide images from intestinal specimens

Deep learning model to differentiate Crohn’s disease from intestinal tuberculosis using histopathological whole slide images from intestinal specimens

Deep Multi-Dictionary Learning for Survival Prediction With Multi-Zoom Histopathological Whole Slide Images.

Survival prediction based on histopathological whole slide images (WSIs) is of great significance for risk-benefit assessment and clinical decision. However, complex microenvironments and heterogeneous tissue structures in WSIs bring challenges to learning informative prognosis-related representations. Additionally, previous studies mainly focus on modeling using mono-scale WSIs, which commonly ignore useful subtle differences existed in multi-zoom WSIs. To this end, we propose a deep multi-dictionary learning framework for cancer survival prediction with multi-zoom histopathological WSIs. The framework can recognize and learn discriminative clusters (i.e., microenvironments) based on multi-scale deep representations for survival analysis. Specifically, we learn multi-scale features based on multi-zoom tiles from WSIs via stacked deep autoencoders network followed by grouping different microenvironments by cluster algorithm. Based on multi-scale deep features of clusters, a multi-dictionary learning method with a post-pruning strategy is devised to learn discriminative representations from selected prognosis-related clusters in a task-driven manner. Finally, a survival model (i.e., EN-Cox) is constructed to estimate the risk index of an individual patient. The proposed model is evaluated on three datasets derived from The Cancer Genome Atlas (TCGA), and the experimental results demonstrate that it outperforms several state-of-the-art survival analysis approaches.

Deep synergetic spiking neural P systems for the overall survival time prediction of glioblastoma patients

Histopathological whole slide images (WSIs) are the gold standard for cancer diagnosis. In prognosis, WSIs can also help predict the overall survival (OS) time of cancer (such as glioblastoma (GBM) patients). However, current region of interest (ROI)-based supervised learning methods for GBM patients are unsatisfactory in accuracy and efficiency. To mitigate the issues, we propose a novel deep synergetic spiking neural (DSSN) P system to perform ROI-free prediction of OS time in GBM patients. Specifically, a dual-task attention network is designed by different synergistic firing mechanisms on new types of neurons to predict OS time and classify genotype simultaneously. The network can not only capture useful cell features but also enrich and share features between the two tasks to improve performance. Four triggering rules are also proposed to cluster pathological images into different phenotypes related to survival to implement ROI-free prediction. Since P systems are highly parallel, multiple preprocessing and ensemble learning of dual-task attention networks with different initial configurations are conducted to further improve OS time prediction accuracy and reduce time complexity. Experimental results show that the DSSN P system outperforms ten other state-of-the-art models, demonstrating the ability of the DSSN P system to predict OS time in GBM patients.

Explainable AI and its instantiation in Computational Pathology for a better understanding of Alzheimer’s disease

AbstractBackgroundEXplainable Artificial Intelligence (XAI) has an essential role to accelerate the AI adoption by empowering biomedical experts and catalyzing modern research through the emergence of a new generation of virtual actors, enabling traceable knowledge‐based quantification, as a guided qualification support. XAI instantiation is shown through our study, hypothesizing that rapid Alzheimer’s Disease (rAD) brains display subtle histological changes that would be undercovered by high‐throughput automated microscopic analysis. We study the topography and morphology of tau lesions’ aggregates to better understand the morphological substratum of Alzheimer’s Disease (AD) heterogeneity.MethodTo address this question at a large scale, we designed, tested and implemented a software for automatic segmentation, annotation and quantitation of brain lesions in histopathological whole slide images (WSI). A dataset of 15 whole slide images of postmortem human brain tissue is used in this study. These are fully annotated by neuropathologists yielding a set of more than 30,000 annotated plaques and tangle objects. A deep learning pipeline consisting of an attention‐UNet model with explainability features is trained for segmentation of tau aggregates in WSIs. The attention maps corresponding to segmentation predictions from the model are used by pathologists as a comparable reference to assist their decision making process.ResultThe DL‐based WSI analysis framework allows detecting a majority of the tau aggregates and refining tau aggregate boundaries with higher accuracy compared to manual annotations. Besides, with such a DL model based tau identification, the pathologist needs to annotate only a few samples for the DL model training. Overall, DL assisted analysis of tau aggregates could ease the effort of pathologists and facilitate analysis of hundreds of WSIs which are vital in AD research areas such as identifying different AD forms and AD patient stratification.ConclusionXAI is not only providing traceability by opening the way to an effective adoption, but also allows optimizing the ML/DL design, by keeping high performances. The green footprint is, therefore, considered, as well as the GDPR (European General Data Protection Regulation), towards Responsible Artificial Intelligence.

Revolutionizing Oral Cancer Detection: An Approach Using Aquila and Gorilla Algorithms Optimized Transfer Learning-Based CNNs.

The early detection of oral cancer is pivotal for improving patient survival rates. However, the high cost of manual initial screenings poses a challenge, especially in resource-limited settings. Deep learning offers an enticing solution by enabling automated and cost-effective screening. This study introduces a groundbreaking empirical framework designed to revolutionize the accurate and automatic classification of oral cancer using microscopic histopathology slide images. This innovative system capitalizes on the power of convolutional neural networks (CNNs), strengthened by the synergy of transfer learning (TL), and further fine-tuned using the novel Aquila Optimizer (AO) and Gorilla Troops Optimizer (GTO), two cutting-edge metaheuristic optimization algorithms. This integration is a novel approach, addressing bias and unpredictability issues commonly encountered in the preprocessing and optimization phases. In the experiments, the capabilities of well-established pre-trained TL models, including VGG19, VGG16, MobileNet, MobileNetV3Small, MobileNetV2, MobileNetV3Large, NASNetMobile, and DenseNet201, all initialized with 'ImageNet' weights, were harnessed. The experimental dataset consisted of the Histopathologic Oral Cancer Detection dataset, which includes a 'normal' class with 2494 images and an 'OSCC' (oral squamous cell carcinoma) class with 2698 images. The results reveal a remarkable performance distinction between the AO and GTO, with the AO consistently outperforming the GTO across all models except for the Xception model. The DenseNet201 model stands out as the most accurate, achieving an astounding average accuracy rate of 99.25% with the AO and 97.27% with the GTO. This innovative framework signifies a significant leap forward in automating oral cancer detection, showcasing the tremendous potential of applying optimized deep learning models in the realm of healthcare diagnostics. The integration of the AO and GTO in our CNN-based system not only pushes the boundaries of classification accuracy but also underscores the transformative impact of metaheuristic optimization techniques in the field of medical image analysis.

Histopathologic brain age estimation via multiple instance learning

Understanding age acceleration, the discordance between biological and chronological age, in the brain can reveal mechanistic insights into normal physiology as well as elucidate pathological determinants of age-related functional decline and identify early disease changes in the context of Alzheimer’s and other disorders. Histopathological whole slide images provide a wealth of pathologic data on the cellular level that can be leveraged to build deep learning models to assess age acceleration. Here, we used a collection of digitized human post-mortem hippocampal sections to develop a histological brain age estimation model. Our model predicted brain age within a mean absolute error of 5.45 ± 0.22 years, with attention weights corresponding to neuroanatomical regions vulnerable to age-related changes. We found that histopathologic brain age acceleration had significant associations with clinical and pathologic outcomes that were not found with epigenetic based measures. Our results indicate that histopathologic brain age is a powerful, independent metric for understanding factors that contribute to brain aging.

High-Order Correlation-Guided Slide-Level Histology Retrieval With Self-Supervised Hashing.

Histopathological Whole Slide Images (WSIs) play a crucial role in cancer diagnosis. It is of significant importance for pathologists to search for images sharing similar content with the query WSI, especially in the case-based diagnosis. While slide-level retrieval could be more intuitive and practical in clinical applications, most methods are designed for patch-level retrieval. A few recently unsupervised slide-level methods only focus on integrating patch features directly, without perceiving slide-level information, and thus severely limits the performance of WSI retrieval. To tackle the issue, we propose a High-Order Correlation-Guided Self-Supervised Hashing-Encoding Retrieval (HSHR) method. Specifically, we train an attention-based hash encoder with slide-level representation in a self-supervised manner, enabling it to generate more representative slide-level hash codes of cluster centers and assign weights for each. These optimized and weighted codes are leveraged to establish a similarity-based hypergraph, in which a hypergraph-guided retrieval module is adopted to explore high-order correlations in the multi-pairwise manifold to conduct WSI retrieval. Extensive experiments on multiple TCGA datasets with over 24,000 WSIs spanning 30 cancer subtypes demonstrate that HSHR achieves state-of-the-art performance compared with other unsupervised histology WSI retrieval methods.

Valuing vicinity: Memory attention framework for context-based semantic segmentation in histopathology.

The segmentation of histopathological whole slide images into tumourous and non-tumourous types of tissue is a challenging task that requires the consideration of both local and global spatial contexts to classify tumourous regions precisely. The identification of subtypes of tumour tissue complicates the issue as the sharpness of separation decreases and the pathologist's reasoning is even more guided by spatial context. However, the identification of detailed tissue types is crucial for providing personalized cancer therapies. Due to the high resolution of whole slide images, existing semantic segmentation methods, restricted to isolated image sections, are incapable of processing context information beyond. To take a step towards better context comprehension, we propose a patch neighbour attention mechanism to query the neighbouring tissue context from a patch embedding memory bank and infuse context embeddings into bottleneck hidden feature maps. Our memory attention framework (MAF) mimics a pathologist's annotation procedure - zooming out and considering surrounding tissue context. The framework can be integrated into any encoder-decoder segmentation method. We evaluate the MAF on two public breast cancer and liver cancer data sets and an internal kidney cancer data set using famous segmentation models (U-Net, DeeplabV3) and demonstrate the superiority over other context-integrating algorithms - achieving a substantial improvement of up to 17% on Dice score. The code is publicly available at https://github.com/tio-ikim/valuing-vicinity.

Cancer Survival Prediction From Whole Slide Images With Self-Supervised Learning and Slide Consistency

Histopathological Whole Slide Images (WSIs) at giga-pixel resolution are the gold standard for cancer analysis and prognosis. Due to the scarcity of pixel- or patch-level annotations of WSIs, many existing methods attempt to predict survival outcomes based on a three-stage strategy that includes patch selection, patch-level feature extraction and aggregation. However, the patch features are usually extracted by using truncated models (e.g. ResNet) pretrained on ImageNet without fine-tuning on WSI tasks, and the aggregation stage does not consider the many-to-one relationship between multiple WSIs and the patient. In this paper, we propose a novel survival prediction framework that consists of patch sampling, feature extraction and patient-level survival prediction. Specifically, we employ two kinds of self-supervised learning methods, i.e. colorization and cross-channel, as pretext tasks to train convnet-based models that are tailored for extracting features from WSIs. Then, at the patient-level survival prediction we explicitly aggregate features from multiple WSIs, using consistency and contrastive losses to normalize slide-level features at the patient level. We conduct extensive experiments on three large-scale datasets: TCGA-GBM, TCGA-LUSC and NLST. Experimental results demonstrate the effectiveness of our proposed framework, as it achieves state-of-the-art performance in comparison with previous studies, with concordance index of 0.670, 0.679 and 0.711 on TCGA-GBM, TCGA-LUSC and NLST, respectively.

Histopathologic Brain Age Estimation via Deep Multiple Instance Learning (P3-6.008)

<h3>Objective:</h3> To identify features of histologic brain aging and clinical correlates of brain age acceleration. <h3>Background:</h3> The discordance between cellular and chronologic aging is useful for understanding diseases in the brain and biology at large. One established method for analyzing the factors that contribute to brain aging is to train machine learning models that predict an individual’s age based on an MRI image of their brain. While this approach has yielded important insights, it is inherently constrained by the information provided by an MRI. However, age-dependent pathologic change has the potential to be assessed at greater detail histologically. Histopathological whole slide images provide more granular information regarding cellular structure, injury, dysfunction, and morphology. Recent technological advances in whole slide image digitization has paved the way for large scale analysis of histologic data via artificially intelligent based computer vision techniques. <h3>Design/Methods:</h3> We leveraged a large novel collection of uniformly processed digitized human post-mortem brain tissue sections to create a histological brain age estimation model. We further investigated the effect of cognitive impairment and exogenous stress on the model. This was accomplished by developing a context-aware attention-based deep multiple instance learning model on 702 human brain tissues sections (ages 50–110 yr) from the hippocampus stained with Luxol Fast Blue counterstained with hematoxylin and eosin on a brain age estimation task. <h3>Results:</h3> This model estimated brain age within a mean absolute error of 5.2 years. Learned attention weights corresponded to neuroanatomical regions vulnerable to age-related change. We found that histopathologic brain age acceleration had significant associations with cognitive impairment, MMSE, p-tau burden, chronic traumatic encephalopathy, and cerebrovascular disease. These associations were not found when using epigenetic-based measures of age acceleration. <h3>Conclusions:</h3> These data indicate that estimated histopathologic brain age can be used as a reliable pathologic correlate to identify factors that contribute to accelerated or decelerated brain aging. <b>Disclosure:</b> Mr. Marx has nothing to disclose. Dr. McKenzie has nothing to disclose. Mr. Kauffman has nothing to disclose. Mr. Koenigsberg has nothing to disclose. The institution of Dr. Farrell has received research support from NIH. Dr. Crary has nothing to disclose.

Computer Aided Classifier of Colorectal Cancer on Histopatological Whole Slide Images Analyzing Deep Learning Architecture Parameters

The diagnosis of different pathologies and stages of cancer using whole histopathology slide images (WSI) is the gold standard for determining the degree of tissue metastasis. The use of deep learning systems in the field of medical images, especially histopathology images, is becoming increasingly important. The training and optimization of deep neural network models involve fine-tuning parameters and hyperparameters such as learning rate, batch size (BS), and boost to improve the performance of the model in task-specific applications. Tuning hyperparameters is a major challenge in designing deep neural network models, having a large impact on the performance. This paper analyzes how the parameters and hyperparameters of a deep learning architecture affect the classification of colorectal cancer (CRC) histopathology images using the well-known VGG19 model. This paper also discusses the pre-processing of these images, such as the use of color normalization and stretching transformations on the data set. Among these hyperparameters, the most important neural network hyperparameter is the learning rate (LR). In this paper, different strategies for the optimization of LR are analyzed (both static and dynamic) and a new experiment based on the variation of LR is proposed (the relevance of dynamic strategies over fixed LR is highlighted), after each layer of the neural network together with decreasing variations according to the epochs. The results obtained are very remarkable, obtaining in the simulation an accurate system that achieves 96.4% accuracy on test images (for nine different tissue classes) using the triangular-cyclic learning rate.

Abstract 3177: Prediction of pathway-omics signature of histopathology images via attention-based deep learning in lung adenocarcinoma and squamous cell carcinoma

Abstract Introduction: Unraveling the underlying signaling pathways and multi-omics expression is vital for cancer research. We aim to construct an analytic deep learning (DL) pipeline toward the discovery of pathways and multi-omics signatures with high confidence from histopathological whole slide images (WSIs) for lung cancer. Methods: A total of 458 and 442 diagnostic WSIs from TCGA-LUAD and TCGA-LUSC cohorts, respectively, were preprocessed via steps including patch generation, color normalization, blurry removal, K-means clustering, and pre-trained Convolutional Neural Network (CNN) based feature extraction. The attention-based model was trained on the CNN-extracted features that provided the importance score for each cluster which was utilized for the final prediction of (1) pathway activities, (2) gene expression, and (3) protein expression. The model was first used to predict 1387 pathway activity scores using PARADIGM as the ground truth. The thresholds for Spearman’s correlation R &amp;gt;= 0.5 and R &amp;gt;= 0.45at p-value &amp;lt;0.05 were set for LUAD and LUSC, respectively, for selecting the well-predicted pathways. Later, multiple models were trained to predict the expression value of the genes and proteins involved in these well-predicted pathways. Moreover, leveraging the attention mechanism, heat maps were generated to visualize the high and low-activated areas on the WSIs thus providing a better interpretation of the predictions. Results: The models demonstrated good performance in predicting pathway scores, genes, and proteins’ expression using the hold-out testing sets. For LUAD, the model successfully predicted important pathways such as interleukin-, uPAR-, and Sema4D-related and PIP3-activated AKT signaling pathways, which have been proven to be associated with LUAD. At the gene- and protein expression level, the model predicted IRF4, EREG, CCNA2, IL2RB, GRB2, CCNB1, STAT5A genes, and AKT protein, respectively, with high and significant correlation. Similarly, for LUSC, the model predicted highly correlated pathways such as the regulation of telomerase and inflammasomes as well as important genes including AIM2, SIN3A, EGFR, and ATM with significant correlation. Furthermore, the highly activated hot spots from the heat map could assist in decoding the model prediction for crucial tissue locations, eventually contributing to a better interpretation of the final output. Conclusion: The proposed attention-based DL pipeline not only provides high-resolution results at the pathways, and multi-omics levels, but offers a cost-effective approach that enables high throughput screening for potential targets rapidly and robustly via a single WSI. Citation Format: Han-Ru Chen, Nam Nhut Phan, Tzu-Pin Lu, Liang-Chuan Lai, Mong-Hsun Tsai, Amrita Chattopadhyay, Eric Y. Chuang. Prediction of pathway-omics signature of histopathology images via attention-based deep learning in lung adenocarcinoma and squamous cell carcinoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 3177.

Prediction of drug-induced hepatotoxicity based on histopathological whole slide images.

Liver is an important metabolic organ in human body and is sensitive to toxic chemicals or drugs. Adverse reactions caused by drug hepatotoxicity will damage the liver and hepatotoxicity is the leading cause of removal of approved drugs from the market. Therefore, it is of great significance to identify liver toxicity as early as possible in the drug development process. In this study, we developed a predictive model for drug hepatotoxicity based on histopathological whole silde images (WSI) which are the by-product of drug experiments and have received little attention. To better represent the WSIs, we constructed a graph representation for each WSI by dividing it into small patches, taking sampled patches as nodes and calculating the correlation coefficients between node features as the edges of the graph structure. Then a WSI-level graph convolutional network (GCN) was built to effectively extract the node information of the graph and predict the toxicity. In addition, we introduced a gated attention global context vector (gaGCV) to combine the global context to make node features to contain more comprehensive information. The results validated on rat liver in vivo data from the Open TG-GATES show that the use of WSI for the prediction of toxicity is feasible and effective.

Improving the robustness and stability of a machine learning model for breast cancer prognosis through the use of multi-modal classifiers

Breast cancer is a deadly disease with a high mortality rate among PAN cancers. The advancements in biomedical information retrieval techniques have been beneficial in developing early prognosis and diagnosis systems for cancer patients. These systems provide the oncologist with plenty of information from several modalities to make the correct and feasible treatment plan for breast cancer patients and protect them from unnecessary therapies and their toxic side effects. The cancer patient’s related information can be collected using various modalities like clinical, copy number variation, DNA-methylation, microRNA sequencing, gene expression, and histopathological whole slide images. High dimensionality and heterogeneity in these modalities demand the development of some intelligent systems to understand related features to the prognosis and diagnosis of diseases and make correct predictions. In this work, we have studied some end-to-end systems having two main components : (a) dimensionality reduction techniques applied to original features from different modalities and (b) classification techniques applied to the fusion of reduced feature vectors from different modalities for automatic predictions of breast cancer patients into two categories: short-time and long-time survivors. Principal component analysis (PCA) and variational auto-encoders (VAEs) are used as the dimensionality reduction techniques, followed by support vector machines (SVM) or random forest as the machine learning classifiers. The study utilizes raw, PCA, and VAE extracted features of the TCGA-BRCA dataset from six different modalities as input to the machine learning classifiers. We conclude this study by suggesting that adding more modalities to the classifiers provides complementary information to the classifier and increases the stability and robustness of the classifiers. In this study, the multimodal classifiers have not been validated on primary data prospectively.

A pathomic approach for tumor-infiltrating lymphocytes classification on breast cancer digital pathology images

Background and objectivesThe detection of tumor-infiltrating lymphocytes (TILs) could aid in the development of objective measures of the infiltration grade and can support decision-making in breast cancer (BC). However, manual quantification of TILs in BC histopathological whole slide images (WSI) is currently based on a visual assessment, thus resulting not standardized, not reproducible, and time-consuming for pathologists. In this work, a novel pathomic approach, aimed to apply high-throughput image feature extraction techniques to analyze the microscopic patterns in WSI, is proposed. In fact, pathomic features provide additional information concerning the underlying biological processes compared to the WSI visual interpretation, thus providing more easily interpretable and explainable results than the most frequently investigated Deep Learning based methods in the literature. MethodsA dataset containing 1037 regions of interest with tissue compartments and TILs annotated on 195 TNBC and HER2+ BC hematoxylin and eosin (H&E)-stained WSI was used. After segmenting nuclei within tumor-associated stroma using a watershed-based approach, 71 pathomic features were extracted from each nucleus and reduced using a Spearman's correlation filter followed by a nonparametric Wilcoxon rank-sum test and least absolute shrinkage and selection operator. The relevant features were used to classify each candidate nucleus as either TILs or non-TILs using 5 multivariable machine learning classification models trained using 5-fold cross-validation (1) without resampling, (2) with the synthetic minority over-sampling technique and (3) with downsampling. The prediction performance of the models was assessed using ROC curves. Results21 features were selected, with most of them related to the well-known TILs properties of having regular shape, clearer margins, high peak intensity, more homogeneous enhancement and different textural pattern than other cells. The best performance was obtained by Random-Forest with ROC AUC of 0.86, regardless of resampling technique. ConclusionsThe presented approach holds promise for the classification of TILs in BC H&E-stained WSI and could provide support to pathologists for a reliable, rapid and interpretable clinical assessment of TILs in BC.

Feature Selection and Comparative Analysis of Breast Cancer Prediction Using Clinical Data and Histopathological Whole Slide Images

Feature Selection and Comparative Analysis of Breast Cancer Prediction Using Clinical Data and Histopathological Whole Slide Images