Pathology Whole-slide Images Research Articles

Improved DeTraC Binary Coyote Net-Based Multiple Instance Learning for Predicting Lymph Node Metastasis of Breast Cancer From Whole-Slide Pathological Images.

Early detection of lymph node metastasis in breast cancer is vital for improving treatment outcomes and prognosis. This study introduces an Improved Decompose, Transfer, and Compose Binary Coyote Net-based Multiple Instance Learning (ImDeTraC-BCNet-MIL) method for predicting lymph node metastasis from Whole Slide Images (WSIs) using multiple instance learning. The method involves segmenting WSIs into patches using Otsu and double-dimensional clustering techniques. The developed multiple instance learning approach introduces a paradigm into computational pathology by shaping pathological data and constructing features. ImDeTraC-BCNet-MIL was utilised for feature generation during both training and testing to differentiate lymph node metastasis in WSIs. The proposed model achieves the highest accuracy of 95.3% and 99.8%, precision values of 98% and 99.8%, and recall rates of 92.9% and 99.8% on the Camelyon16 and Camelyon17 datasets. These findings underscore the effectiveness of ImDeTraC-BCNet-MIL in enhancing the early detection of lymph node metastasis in breast cancer.

Dual-Attention Multiple Instance Learning Framework for Pathology Whole-Slide Image Classification

Conventional methods for tumor diagnosis suffer from two inherent limitations: they are time-consuming and subjective. Computer-aided diagnosis (CAD) is an important approach for addressing these limitations. Pathology whole-slide images (WSIs) are high-resolution tissue images that have made significant contributions to cancer diagnosis and prognosis assessment. Due to the complexity of WSIs and the availability of only slide-level labels, multiple instance learning (MIL) has become the primary framework for WSI classification. However, most MIL methods fail to capture the interdependence among image patches within a WSI, which is crucial for accurate classification prediction. Moreover, due to the weak supervision of slide-level labels, overfitting may occur during the training process. To address these issues, this paper proposes a dual-attention-based multiple instance learning framework (DAMIL). DAMIL leverages the spatial relationships and channel information between WSI patches for classification prediction, without detailed pixel-level tumor annotations. The output of the model preserves the semantic variations in the latent space, enhances semantic disturbance invariance, and provides reliable class identification for the final slide-level representation. We validate the effectiveness of DAMIL on the most commonly used public dataset, Camelyon16. The results demonstrate that DAMIL outperforms the state-of-the-art methods in terms of classification accuracy (ACC), area under the curve (AUC), and F1-Score. Our model also allows for the examination of its interpretability by visualizing the dual-attention weights. To the best of our knowledge, this is the first attempt to use a dual-attention mechanism, considering both spatial and channel information, for whole-slide image classification.

Multicenter evaluation of CT deep radiomics model in predicting Leibovich score risk groups for non-metastatic clear cell renal cell carcinoma

BackgroundNon-metastatic clear cell renal cell carcinoma (nccRCC) poses a significant risk of postoperative recurrence and metastasis, underscoring the importance of accurate preoperative risk assessment. While the Leibovich score is effective, it relies on postoperative histopathological data. This study aims to evaluate the efficacy of CT radiomics and deep learning models in predicting Leibovich score risk groups in nccRCC, and to explore the interrelationship between CT and pathological features. Patients and MethodsThis research analyzed 600 nccRCC patients from four datasets, dividing them into low (Leibovich scores of 0–2) and intermediate to high risk (Leibovichscores exceeding 3) groups. Radiological models were developed from CT subjective features, and radiomics and deep learning models were constructed from CT images. Additionally, a deep radiomics model and a fusion model integrating radiological, radiomics, and deep learning features were introduced. Model performance was assessed by AUC values, while survival differences across predicted groups were analyzed using survival curves and the DeLong test. Moreover, the research investigated the connection between CT and pathological features derived from whole-slide pathological images in the second validation dataset. ResultsWithin the training dataset, four radiological, three radiomics, and thirteen deep learning features were selected to develop models predicting nccRCC Leibovich score risk groups. The deep radiomics model demonstrated superior predictive accuracy, evidenced by AUC values of 0.881, 0.829, and 0.819 in external validation datasets. Notably, significant differences in overall survival were observed among patients classified by this model (DeLong test p < 0.05 across all datasets). Furthermore, a correlation and complementarity were observed between deep radiomics features and pathological deep learning features. ConclusionsThe CT deep radiomics model precisely predicts nccRCC Leibovich score risk groups preoperatively and highlights the synergistic effect between CT and pathological data.

Computer-aided diagnosis system for grading brain tumor using histopathology images based on color and texture features

BackgroundCancer pathology shows disease development and associated molecular features. It provides extensive phenotypic information that is cancer-predictive and has potential implications for planning treatment. Based on the exceptional performance of computational approaches in the field of digital pathogenic, the use of rich phenotypic information in digital pathology images has enabled us to identify low-level gliomas (LGG) from high-grade gliomas (HGG). Because the differences between the textures are so slight, utilizing just one feature or a small number of features produces poor categorization results.MethodsIn this work, multiple feature extraction methods that can extract distinct features from the texture of histopathology image data are used to compare the classification outcomes. The successful feature extraction algorithms GLCM, LBP, multi-LBGLCM, GLRLM, color moment features, and RSHD have been chosen in this paper. LBP and GLCM algorithms are combined to create LBGLCM. The LBGLCM feature extraction approach is extended in this study to multiple scales using an image pyramid, which is defined by sampling the image both in space and scale. The preprocessing stage is first used to enhance the contrast of the images and remove noise and illumination effects. The feature extraction stage is then carried out to extract several important features (texture and color) from histopathology images. Third, the feature fusion and reduction step is put into practice to decrease the number of features that are processed, reducing the computation time of the suggested system. The classification stage is created at the end to categorize various brain cancer grades. We performed our analysis on the 821 whole-slide pathology images from glioma patients in the Cancer Genome Atlas (TCGA) dataset. Two types of brain cancer are included in the dataset: GBM and LGG (grades II and III). 506 GBM images and 315 LGG images are included in our analysis, guaranteeing representation of various tumor grades and histopathological features.ResultsThe fusion of textural and color characteristics was validated in the glioma patients using the 10-fold cross-validation technique with an accuracy equals to 95.8%, sensitivity equals to 96.4%, DSC equals to 96.7%, and specificity equals to 97.1%. The combination of the color and texture characteristics produced significantly better accuracy, which supported their synergistic significance in the predictive model. The result indicates that the textural characteristics can be an objective, accurate, and comprehensive glioma prediction when paired with conventional imagery.ConclusionThe results outperform current approaches for identifying LGG from HGG and provide competitive performance in classifying four categories of glioma in the literature. The proposed model can help stratify patients in clinical studies, choose patients for targeted therapy, and customize specific treatment schedules.

Deep Learning Analysis for Predicting Tumor Spread through Air Space in Early-Stage Lung Adenocarcinoma Pathology Images.

The presence of spread through air spaces (STASs) in early-stage lung adenocarcinoma is a significant prognostic factor associated with disease recurrence and poor outcomes. Although current STAS detection methods rely on pathological examinations, the advent of artificial intelligence (AI) offers opportunities for automated histopathological image analysis. This study developed a deep learning (DL) model for STAS prediction and investigated the correlation between the prediction results and patient outcomes. To develop the DL-based STAS prediction model, 1053 digital pathology whole-slide images (WSIs) from the competition dataset were enrolled in the training set, and 227 WSIs from the National Taiwan University Hospital were enrolled for external validation. A YOLOv5-based framework comprising preprocessing, candidate detection, false-positive reduction, and patient-based prediction was proposed for STAS prediction. The model achieved an area under the curve (AUC) of 0.83 in predicting STAS presence, with 72% accuracy, 81% sensitivity, and 63% specificity. Additionally, the DL model demonstrated a prognostic value in disease-free survival compared to that of pathological evaluation. These findings suggest that DL-based STAS prediction could serve as an adjunctive screening tool and facilitate clinical decision-making in patients with early-stage lung adenocarcinoma.

Multi-Instance Multi-Task Learning for Joint Clinical Outcome and Genomic Profile Predictions From the Histopathological Images.

With the remarkable success of digital histopathology and the deep learning technology, many whole-slide pathological images (WSIs) based deep learning models are designed to help pathologists diagnose human cancers. Recently, rather than predicting categorical variables as in cancer diagnosis, several deep learning studies are also proposed to estimate the continuous variables such as the patients' survival or their transcriptional profile. However, most of the existing studies focus on conducting these predicting tasks separately, which overlooks the useful intrinsic correlation among them that can boost the prediction performance of each individual task. In addition, it is sill challenge to design the WSI-based deep learning models, since a WSI is with huge size but annotated with coarse label. In this study, we propose a general multi-instance multi-task learning framework (HistMIMT) for multi-purpose prediction from WSIs. Specifically, we firstly propose a novel multi-instance learning module (TMICS) considering both common and specific task information across different tasks to generate bag representation for each individual task. Then, a soft-mask based fusion module with channel attention (SFCA) is developed to leverage useful information from the related tasks to help improve the prediction performance on target task. We evaluate our method on three cancer cohorts derived from the Cancer Genome Atlas (TCGA). For each cohort, our multi-purpose prediction tasks range from cancer diagnosis, survival prediction and estimating the transcriptional profile of gene TP53. The experimental results demonstrated that HistMIMT can yield better outcome on all clinical prediction tasks than its competitors.

Improving diagnosis and outcome prediction of gastric cancer via multimodal learning using whole slide pathological images and gene expression

For the diagnosis and outcome prediction of gastric cancer (GC), machine learning methods based on whole slide pathological images (WSIs) have shown promising performance and reduced the cost of manual analysis. Nevertheless, accurate prediction of GC outcome may rely on multiple modalities with complementary information, particularly gene expression data. Thus, there is a need to develop multimodal learning methods to enhance prediction performance. In this paper, we collect a dataset from Ruijin Hospital and propose a multimodal learning method for GC diagnosis and outcome prediction, called GaCaMML, which is featured by a cross-modal attention mechanism and Per-Slide training scheme. Additionally, we perform feature attribution analysis via integrated gradient (IG) to identify important input features. The proposed method improves prediction accuracy over the single-modal learning method on three tasks, i.e., survival prediction (by 4.9% on C-index), pathological stage classification (by 11.6% on accuracy), and lymph node classification (by 12.0% on accuracy). Especially, the Per-Slide strategy addresses the issue of a high WSI-to-patient ratio and leads to much better results compared with the Per-Person training scheme. For the interpretable analysis, we find that although WSIs dominate the prediction for most samples, there is still a substantial portion of samples whose prediction highly relies on gene expression information. This study demonstrates the great potential of multimodal learning in GC-related prediction tasks and investigates the contribution of WSIs and gene expression, respectively, which not only shows how the model makes a decision but also provides insights into the association between macroscopic pathological phenotypes and microscopic molecular features.

Pancreatic Cancer Action Network's SPARK: A Cloud-Based Patient Health Data and Analytics Platform for Pancreatic Cancer.

Pancreatic cancer currently holds the position of third deadliest cancer in the United States and the 5-year survival rate is among the lowest for major cancers at just 12%. Thus, continued research efforts to better understand the clinical and molecular underpinnings of pancreatic cancer are critical to developing both early detection methodologies as well as improved therapeutic options. This study introduces Pancreatic Cancer Action Network's (PanCAN's) SPARK, a cloud-based data and analytics platform that integrates patient health data from the PanCAN's research initiatives and aims to accelerate pancreatic cancer research by making real-world patient health data and analysis tools easier to access and use. The SPARK platform integrates clinical, molecular, multiomic, imaging, and patient-reported data generated from PanCAN's research initiatives. The platform is built on a cloud-based infrastructure powered by Velsera. Cohort exploration and browser capabilities are built using Velsera ARIA, a specialized product for leveraging clinicogenomic data to build cohorts, query variant information, and drive downstream association analyses. Data science and analytic capabilities are also built into the platform allowing researchers to perform simple to complex analysis. Version 1 of the SPARK platform was released to pilot users, who represented diverse end users, including molecular biologists, clinicians, and bioinformaticians. Included in the pilot release of SPARK are deidentified clinical (including treatment and outcomes data), molecular, multiomic, and whole-slide pathology images for over 600 patients enrolled in PanCAN's Know Your Tumor molecular profiling service. The pilot release of the SPARK platform introduces qualified researchers to PanCAN real-world patient health data and analytical resources in a centralized location.

Artifact detection of low-magnification pathology whole-slide images

Artifact detection of low-magnification pathology whole-slide images

Scaling the Inference of Digital Pathology Deep Learning Models Using CPU-Based High-Performance Computing

Digital pathology whole-slide images (WSIs) are large-size gigapixel images and image analysis based on deep learning artificial intelligence (AI) technology often involves pixel-wise testing of a trained deep learning neural network (DLNN) on hundreds of WSI images, which is time consuming. We take advantage of High-Performance Computing (HPC) facilities to parallelize this procedure into multiple independent (and hence delightfully parallel) tasks. However, traditional software parallelization techniques and regular file formats can have significant scaling problems on HPC clusters. In this work, a useful computational strategy is designed to localize and extract relevant patches in WSI files and group them in HDF5 files well suited for parallel I/O. HPC's array job facilities are adapted for hierarchical scaling and parallelization of WSI pre-processing and testing of trained algorithms. Applying these techniques to testing a trained DLNN on the CAMELYON datasets with 399 WSIs reduced the theoretical processing time of 18 years on a single CPU or 30 days on a single GPU to less than 45 hours on an HPC cluster of 4,000 CPU cores. The efficiency-accuracy trade-off we demonstrated on this dataset further reinforced the importance of efficient computation techniques, without which the accuracy may be sacrificed. The framework developed here for testing DLNNs does not rely on any specific neural network architecture and HPC cluster setup and can be utilized for any large-scale image processing and big-data analysis.

Yolov5 AI Deep Learning model driven Nuclear Pleomorphism Grading on Breast Cancer Pathology WSI for Nottingham Cancer Grading

Breast cancer is the second largest cancer caused in the world due to the uncontrollable growth in breast cells. Nottingham Grading is the internationally acceptable system to grade breast cancer. Nuclear pleomorphism is one of the breast cancer biomarkers for computing Nottingham grading. Pathologists grade nuclear pleomorphism on breast cancer glass tissue slides using a conventional microscope which is time consuming and has considerable inter-observer variability between pathologists. The paper proposed an Artificial Intelligence (AI) deep learning model to grade grade1, grade2, grade3 nuclear pleomorphism on breast cancer whole slide images (WSI). The proposed Yolov5 model is trained and tested on 1,30,000 WSI tiles having around two lakh annotations. The accuracy of the model is mAP 0.89. The proposed model saves the time and reduces the workload of the pathologist and also helps them to produce accurate results.

Neuropathologist-level integrated classification of adult-type diffuse gliomas using deep learning from whole-slide pathological images

Current diagnosis of glioma types requires combining both histological features and molecular characteristics, which is an expensive and time-consuming procedure. Determining the tumor types directly from whole-slide images (WSIs) is of great value for glioma diagnosis. This study presents an integrated diagnosis model for automatic classification of diffuse gliomas from annotation-free standard WSIs. Our model is developed on a training cohort (n = 1362) and a validation cohort (n = 340), and tested on an internal testing cohort (n = 289) and two external cohorts (n = 305 and 328, respectively). The model can learn imaging features containing both pathological morphology and underlying biological clues to achieve the integrated diagnosis. Our model achieves high performance with area under receiver operator curve all above 0.90 in classifying major tumor types, in identifying tumor grades within type, and especially in distinguishing tumor genotypes with shared histological features. This integrated diagnosis model has the potential to be used in clinical scenarios for automated and unbiased classification of adult-type diffuse gliomas.

Characterizing the Survival-Associated Interactions between Tumor-infiltrating Lymphocytes and Tumors from Pathological Images and Multi-omics Data.

The tumor-infiltrating lymphocytes (TILs) and its correlation with tumors have shown significant values in the development of cancers. Many observations indicated that the combination of the whole-slide pathological images (WSIs) and genomic data can better characterize the immunological mechanisms of TILs. However, the existing image-genomic studies evaluated the TILs by the combination of pathological image and single-type of omics data (e.g., mRNA), which is difficulty in assessing the underlying molecular processes of TILs holistically. Additionally, it is still very challenging to characterize the intersections between TILs and tumor regions in WSIs and the high dimensional genomic data also brings difficulty for the integrative analysis with WSIs. Based on the above considerations, we proposed an end-to-end deep learning framework i.e., IMO-TILs that can integrate pathological image with multi-omics data (i.e., mRNA and miRNA) to analyze TILs and explore the survival-associated interactions between TILs and tumors. Specifically, we firstly apply the graph attention network to describe the spatial interactions between TILs and tumor regions in WSIs. As to genomic data, the Concrete AutoEncoder (i.e., CAE) is adopted to select survival-associated Eigengenes from the high-dimensional multi-omics data. Finally, the deep generalized canonical correlation analysis (DGCCA) accompanied with the attention layer is implemented to fuse the image and multi-omics data for prognosis prediction of human cancers. The experimental results on three cancer cohorts derived from the Cancer Genome Atlas (TCGA) indicated that our method can both achieve higher prognosis results and identify consistent imaging and multi-omics bio-markers correlated strongly with the prognosis of human cancers.

SGCL: Spatial guided contrastive learning on whole-slide pathological images.

Self-supervised representation learning (SSL) has achieved remarkable success in its application to natural images while falling behind in performance when applied to whole-slide pathological images (WSIs). This is because the inherent characteristics of WSIs in terms of gigapixel resolution and multiple objects in training patches are fundamentally different from natural images. Directly transferring the state-of-the-art (SOTA) SSL methods designed for natural images to WSIs will inevitably compromise their performance. We present a novel scheme SGCL: Spatial Guided Contrastive Learning, to fully explore the inherent properties of WSIs, leveraging the spatial proximity and multi-object priors for stable self-supervision. Beyond the self-invariance of instance discrimination, we expand and propagate the spatial proximity for the intra-invariance from the same WSI and inter-invariance from different WSIs, as well as propose the spatial-guided multi-cropping for inner-invariance within patches. To adaptively explore such spatial information without supervision, we propose a new loss function and conduct a theoretical analysis to validate it. This novel scheme of SGCL is able to achieve additional improvements over the SOTA pre-training methods on diverse downstream tasks across multiple datasets. Extensive ablation studies have been carried out and visualizations of these results have been presented to aid understanding of the proposed SGCL scheme. As open science, all codes and pre-trained models are available at https://github.com/HHHedo/SGCL.

TCGA RNA-Seq and Tumor-Infiltrating Lymphocyte Imaging Data Reveal Cold Tumor Signatures of Invasive Ductal Carcinomas and Estrogen Receptor-Positive Human Breast Tumors.

Comparative studies of immune-active hot and immune-deserted cold tumors are critical for identifying therapeutic targets and strategies to improve immunotherapy outcomes in cancer patients. Tumors with high tumor-infiltrating lymphocytes (TILs) are likely to respond to immunotherapy. We used the human breast cancer RNA-seq data from the cancer genome atlas (TCGA) and classified them into hot and cold tumors based on their lymphocyte infiltration scores. We compared the immune profiles of hot and cold tumors, their corresponding normal tissue adjacent to the tumor (NAT), and normal breast tissues from healthy individuals from the Genotype-Tissue Expression (GTEx) database. Cold tumors showed a significantly lower effector T cells, lower levels of antigen presentation, higher pro-tumorigenic M2 macrophages, and higher expression of extracellular matrix (ECM) stiffness-associated genes. Hot/cold dichotomy was further tested using TIL maps and H&E whole-slide pathology images from the cancer imaging archive (TCIA). Analysis of both datasets revealed that infiltrating ductal carcinoma and estrogen receptor ER-positive tumors were significantly associated with cold features. However, only TIL map analysis indicated lobular carcinomas as cold tumors and triple-negative breast cancers (TNBC) as hot tumors. Thus, RNA-seq data may be clinically relevant to tumor immune signatures when the results are supported by pathological evidence.

Prototypical multiple instance learning for predicting lymph node metastasis of breast cancer from whole-slide pathological images

Computerized identification of lymph node metastasis of breast cancer (BCLNM) from whole-slide pathological images (WSIs) can largely benefit therapy decision and prognosis analysis. Besides the general challenges of computational pathology, like extra-high resolution, very expensive fine-grained annotation, etc., two particular difficulties with this task lie in (1) modeling the significant inter-tumoral heterogeneity in BCLNM pathological images, and (2) identifying micro-metastases, i.e., metastasized tumors with tiny foci. Towards this end, this paper presents a novel weakly supervised method, termed as Prototypical Multiple Instance Learning (PMIL), to learn to predict BCLNM from WSIs with slide-level class labels only. PMIL introduces the well-established vocabulary-based multiple instance learning (MIL) paradigm into computational pathology, which is characterized by utilizing the so-called prototypes to model pathological data and construct WSI features. PMIL mainly consists of two innovatively designed modules, i.e., the prototype discovery module which acquires prototypes from training data by unsupervised clustering, and the prototype-based slide embedding module which builds WSI features by matching constitutive patches against the prototypes. Relative to existing MIL methods for WSI classification, PMIL has two substantial merits: (1) being more explicit and interpretable in modeling the inter-tumoral heterogeneity in BCLNM pathological images, and (2) being more effective in identifying micro-metastases. Evaluation is conducted on two datasets, i.e., the public Camelyon16 dataset and the Zbraln dataset created by ourselves. PMIL achieves an AUC of 88.2% on Camelyon16 and 98.4% on Zbraln (at 40x magnification factor), which consistently outperforms other compared methods. Comprehensive analysis will also be carried out to further reveal the effectiveness and merits of the proposed method.

Classification of whole slide images of breast histopathology based on spatial correlation characteristics

目的 病理学检查是明确乳腺癌诊断及肿瘤类型的金标准。深度神经网络广泛应用于乳腺病理全切片的诊断工作并取得了明显进展，但是现有大多数工作只是将全切片切割成小图像块，对每个图像块进行单独处理，没有考虑它们之间的空间信息。为此，提出了一种融合空间相关性特征的乳腺组织病理全切片分类方法。方法 首先基于卷积神经网络对病理图像块进行预测，并提取每个图像块有代表性的深层特征，然后利用特征融合将图像块及其周围图像的特征进行聚合，以形成具有空间相关性的块描述符，最后将全切片图像中最可疑的块描述符传递给循环神经网络，以预测最终的全切片级别的分类。结果 本文构建了一个经过详细标注的乳腺病理全切片数据集，并在此数据集上进行良性/恶性二分类实验。在自建的数据集中与3种全切片分类方法进行了比较。结果表明，本文方法的分类精度达到96.3%，比未考虑空间相关性的方法高出了1.9%，与基于热力图特征和基于空间性和随机森林的方法相比，分类精度分别高出8.8%和1.3%。结论 本文提出的乳腺组织病理全切片识别方法将空间相关性特征融合和RNN分类集成到一个统一模型，有助于提高图像识别准确率，为病理图像诊断工作提供了高效的辅助诊断工具。;Objective Pathological examination can be as the“gold standard”to interpret breast cancer diagnosis and its tumor types. To improve the treatment effect of breast cancer，accurate interpretation is beneficial to clarify the type of the disease early and effectively. However，it is time-consuming to check the pathological whole slide images（WSIs）manually，and the diagnosis result is easily affected by personal experience. Due to computer vision-based convolutional neural networks（CNNs）are applied to the classification task of histopathological images computer-aided diagnostic（CAD）techniques for digital images of histopathology has been developing intensively. However，it is still challenged to split largesized WSIs into small patches and each patch is processed individually without the spatial information between them. Also， the information-involved of the patches learned is not utilized in the small size patch feature aggregation process. To resolve these problems，we develop a spatial correlation-based classification method for breast histopathology images，which can re-identify the WSIs classification problem through deep learning feature fusion and recurrent neural network（RNN）based classification. Method Our demonstration consists of four aspects：1）pre-processing operation of WSIs，2）CNN-based breast patch prediction，3）image patch feature fusion，and 4）RNN-based WSIs classification. First，it is focused on a preprocessing operation on whole slide images of breast pathology，using a sliding window to cut the WSIs into patches of a suitable size in terms of a CNN. Second，CNN-based patch prediction is used to predict the possibility of cancer in each patch and each patch is encoded as a fixed-length feature. To reduce unpredictable class of cancerous patch being classified as non-cancerous patch，the ResNet34 is used as the patch classification model and penalty factors is added in Focal loss. Third，the feature fusion of image blocks can be recognized as a feature fusion niche that takes the entire grid of patches as input and the spatial correlation of patches and their surrounding patches are as feature fusion of all patches within the grid，thus block descriptors are formed. Feature fusion methods are involved in，including 1）Weight，2）Max， 3）Avg，4）Norm3，and 5）WeightNorm3. Finally，the RNN-based WSIs classification is used to pass the block feature descriptors with high probability of cancer in each WSIs to the RNN，and the RNN model is used to learn the sequence relationship between the image block feature sequences. It can expand the processing field of view of the classification model to multiple Blocks and optimize the classification accuracy of breast pathology WSIs. Additionally，we design and develop an online recognition system for assistance. Result A detailed annotated whole slide images dataset of breast pathology is constructed，and benign/malignant dichotomous classification experiments are carried out on this dataset. The results are compared to three WSIs classification methods in the self-constructed dataset，and it shows that the classification accuracy of the method can be reached to 96. 3%，which is 1. 9% higher than non-spatial correlation method. The classification accuracy of each is 8. 8% and 1. 3% higher compared to heat map features-based methods and spatial correlation and random forest-related methods. Conclusion The proposed breast pathology recognition method can improve image recognition accuracy and provide an efficient diagnostic aid for pathology image diagnosis work，which integrates feature fusion and RNN classification into a synthesized model.

Analysis of Spatial Molecular Data.

Digital analysis of pathology whole-slide images has been recently gaining interest in the context of cancer diagnosis and treatment. In particular, deep learning methods have demonstrated significant potential in supporting pathology analysis, recently detecting molecular traits never before recognized in pathology H&E whole-slide images (WSIs). Alongside these advancements in the digital analysis of WSIs, it is becoming increasingly evident that both spatial and overall tumor heterogeneity may be significant determinants of cancer prognosis and treatment outcome. In this chapter, we describe methods that aim to support these two elements. We describe both an end-to-end deep learning pipeline for producing limited spatial transcriptomics from WSIs with associated bulk gene expression data, as well as an algorithm for quantifying spatial tumor heterogeneity based on the results of this pipeline.

Deep learning-based histotype diagnosis of ovarian carcinoma whole-slide pathology images

Ovarian carcinoma has the highest mortality of all female reproductive cancers and current treatment has become histotype-specific. Pathologists diagnose five common histotypes by microscopic examination, however, histotype determination is not straightforward, with only moderate interobserver agreement between general pathologists (Cohen's kappa 0.54–0.67). We hypothesized that machine learning (ML)-based image classification models may be able to recognize ovarian carcinoma histotype sufficiently well that they could aid pathologists in diagnosis. We trained four different artificial intelligence (AI) algorithms based on deep convolutional neural networks to automatically classify hematoxylin and eosin-stained whole slide images. Performance was assessed through cross-validation on the training set (948 slides corresponding to 485 patients), and on an independent test set of 60 patients from another institution. The best-performing model achieved a diagnostic concordance of 81.38% (Cohen's kappa of 0.7378) in our training set, and 80.97% concordance (Cohen's kappa 0.7547) on the external dataset. Eight cases misclassified by ML in the external set were reviewed by two subspecialty pathologists blinded to the diagnoses, molecular and immunophenotype data, and ML-based predictions. Interestingly, in 4 of 8 cases from the external dataset, the expert review pathologists rendered diagnoses, based on blind review of the whole section slides classified by AI, that were in agreement with AI rather than the integrated reference diagnosis. The performance characteristics of our classifiers indicate potential for improved diagnostic performance if used as an adjunct to conventional histopathology.

Interpretable classification of pathology whole-slide images using attention based context-aware graph convolutional neural network

Background and ObjectiveWhole slide image (WSI) classification and lesion localization within giga-pixel slide are challenging tasks in computational pathology that requires context-aware representations of histological features to adequately infer nidus. The existing weakly supervised learning methods mainly treat different locations in the slide as independent regions and cannot learn potential nonlinear interactions between instances based on i.i.d assumption, resulting in the model unable to effectively utilize context-ware information to predict the labels of WSIs and locate the region of interest (ROI). MethodsHere, we propose an interpretable classification model named bidirectional Attention-based Multiple Instance Learning Graph Convolutional Network (ABMIL-GCN), which hierarchically aggregates context-aware features of instances into a global representation in a topology fashion to predict the slide labels and localize the region of lymph node metastasis in WSIs. ResultsWe verified the superiority of this method on the Camelyon16 dataset, and the results show that the average predicted ACC and AUC of the proposed model after flooding optimization can reach 90.89% and 0.9149, respectively. The average accuracy and ACC score are improved by more than 7% and 4% compared with the existing state-of-the-art algorithms. ConclusionsThe results demonstrate that context-aware GCN outperforms existing weakly supervised learning methods by introducing spatial correlations between the neighbor image patches, which also addresses the ‘accuracy-interpretability trade-off’ problem. The framework provides a novel paradigm for the clinical application of computer-aided diagnosis and intelligent systems.