Classification Of Whole-slide Images Research Articles

Benchmarking common uncertainty estimation methods with histopathological images under domain shift and label noise.

In the past years, deep learning has seen an increase in usage in the domain of histopathological applications. However, while these approaches have shown great potential, in high-risk environments deep learning models need to be able to judge their uncertainty and be able to reject inputs when there is a significant chance of misclassification. In this work, we conduct a rigorous evaluation of the most commonly used uncertainty and robustness methods for the classification of Whole Slide Images, with a focus on the task of selective classification, where the model should reject the classification in situations in which it is uncertain. We conduct our experiments on tile-level under the aspects of domain shift and label noise, as well as on slide-level. In our experiments, we compare Deep Ensembles, Monte-Carlo Dropout, Stochastic Variational Inference, Test-Time Data Augmentation as well as ensembles of the latter approaches. We observe that ensembles of methods generally lead to better uncertainty estimates as well as an increased robustness towards domain shifts and label noise, while contrary to results from classical computer vision benchmarks no systematic gain of the other methods can be shown. Across methods, a rejection of the most uncertain samples reliably leads to a significant increase in classification accuracy on both in-distribution as well as out-of-distribution data. Furthermore, we conduct experiments comparing these methods under varying conditions of label noise. Lastly, we publish our code framework to facilitate further research on uncertainty estimation on histopathological data.

SMILE: Cost-sensitive multi-task learning for nuclear segmentation and classification with imbalanced annotations.

High throughput nuclear segmentation and classification of whole slide images (WSIs) is crucial to biological analysis, clinical diagnosis and precision medicine. With the advances of CNN algorithms and the continuously growing datasets, considerable progress has been made in nuclear segmentation and classification. However, few works consider how to reasonably deal with nuclear heterogeneity in the following two aspects: imbalanced data distribution and diversified morphology characteristics. The minority classes might be dominated by the majority classes due to the imbalanced data distribution and the diversified morphology characteristics may lead to fragile segmentation results. In this study, a cost-Sensitive MultI-task LEarning (SMILE) framework is conducted to tackle the data heterogeneity problem. Based on the most popular multi-task learning backbone in nuclei segmentation and classification, we propose a multi-task correlation attention (MTCA) to perform feature interaction of multiple high relevant tasks to learn better feature representation. A cost-sensitive learning strategy is proposed to solve the imbalanced data distribution by increasing the penalization for the error classification of the minority classes. Furthermore, we propose a novel post-processing step based on the coarse-to-fine marker-controlled watershed scheme to alleviate fragile segmentation when nuclei are with large size and unclear contour. Extensive experiments show that the proposed method achieves state-of-the-art performances on CoNSeP and MoNuSAC 2020 datasets. The code is available at: https://github.com/panxipeng/nuclear_segandcls.

A color-based deep-learning approach for tissue slide lung cancer classification

Non-Small Cell Lung Cancer is the most common type of lung cancer, accounting for more than 80% of all cases. The analysis of histopathological images is the appropriate way to detect lung cancer in its initial stages. However, due to the heterogeneous patterns of molecular structures and the high magnification factor, the classification of whole slide images is difficult. Furthermore, these large-pixel images cannot be considered directly due to the high computational requirements. Several deep-learning models have been developed for the classification of histopathological images, but only in the default RGB format. Existing models are complex and rely on pathologist annotations which increases the computational cost and time. The present paper proposes a deep-learning framework, a Color-based Dilated Convolutional Neural Network (CD-CNN) for the multi-class classification of patch-based whole slide images of lung cancer. The proposed CD-CNN framework includes data preparation, color space transformation, and the classification stages. It thoroughly examines the impact of five color spaces on classification performance. The proposed model was also compared to three pre-trained models and the Convolutional Neural Network model on three different datasets namely: The Cancer Genome Atlas dataset, the Clinical Proteomic Tumor Analysis Consortium cohorts, and the LC25000 dataset. The analysis of all the models reveals that the HSV color space outperforms the other transformations for all of the datasets tested. The results depict that the proposed CD-CNN achieved the best classification results in HSV color space for all three datasets i.e. accuracy of 0.97–0.99, precision, recall, and F1 score 0.97–0.98 and AUC value of 0.970–0.984. In addition, the highest value of the kappa score 0.986 has been achieved by the proposed model. These findings demonstrate the efficacy of the proposed method and show an improvement in diagnostic systems.

Multilayer outperforms single-layer slide scanning in AI-based classification of whole slide images with low-burden acid-fast mycobacteria (AFB)

Manual screening of Ziehl-Neelsen (ZN)-stained slides that are negative or contain rare acid-fast mycobacteria (AFB) is labor-intensive and requires repetitive refocusing to visualize AFB candidates under the microscope. Whole slide image (WSI) scanners have enabled implementation of AI to classify digital ZN-stained slides as AFB+ or AFB-. By default, these scanners acquire a single-layer WSI. However, some scanners can acquire a multilayer WSI with a z-stack and an extended focus image layer embedded. We developed a parameterized WSI classification pipeline to assess whether multilayer imaging improves ZN-stained slide classification accuracy. A CNN built into the pipeline classified tiles in each image layer to form an AFB probability score heatmap. Features extracted from the heatmap were then entered into a WSI classifier. 46 AFB+ and 88 AFB- single-layer WSIs were used for the classifier training. 15 AFB+ (with rare microorganisms) and 5 AFB- multilayer WSIs comprised the test set.Parameters in the pipeline included: (a) a WSI representation: z-stack of image layers, middle image layer (a single image layer equivalent) or an extended focus image layer, (b) 4 methods of aggregating AFB probability scores across the z-stack, (c) 3 classifiers, (d) 3 AFB probability thresholds, and (e) 9 feature vector types extracted from the aggregated AFB probability heatmaps. Balanced accuracy (BACC) was used to measure the pipeline performance for all parameter combinations. Analysis of Covariance (ANCOVA) was used to statistically evaluate the effect of each parameter on the BACC.After adjusting for other factors, a significant effect of the WSI representation (p-value < 1.99E-76), classifier type (p-value < 1.73E-21), and AFB threshold (p-value = 0.003) was observed on the BACC. The feature type had no significant effect (p-value = 0.459) on the BACC. WSIs represented by the middle layer, extended focus layer and the z-stack followed by the weighted averaging of AFB probability scores were classified with the average BACC of 58.80%, 68.64%, and 77.28%, respectively. The multilayer WSIs represented by the z-stack with the weighted averaging of AFB probability scores were classified by a Random Forest classifier with the average BACC of 83.32%. Low classification accuracy of WSIs represented by the middle layer suggests that they contain fewer features permitting identification of AFB than the multilayer WSIs. Our results indicate that single-layer acquisition can introduce a bias (sampling error) into the WSI. This bias can be mitigated by the multilayer or the extended focus acquisitions.

Classification of whole slide images of breast histopathology based on spatial correlation characteristics

目的 病理学检查是明确乳腺癌诊断及肿瘤类型的金标准。深度神经网络广泛应用于乳腺病理全切片的诊断工作并取得了明显进展，但是现有大多数工作只是将全切片切割成小图像块，对每个图像块进行单独处理，没有考虑它们之间的空间信息。为此，提出了一种融合空间相关性特征的乳腺组织病理全切片分类方法。方法 首先基于卷积神经网络对病理图像块进行预测，并提取每个图像块有代表性的深层特征，然后利用特征融合将图像块及其周围图像的特征进行聚合，以形成具有空间相关性的块描述符，最后将全切片图像中最可疑的块描述符传递给循环神经网络，以预测最终的全切片级别的分类。结果 本文构建了一个经过详细标注的乳腺病理全切片数据集，并在此数据集上进行良性/恶性二分类实验。在自建的数据集中与3种全切片分类方法进行了比较。结果表明，本文方法的分类精度达到96.3%，比未考虑空间相关性的方法高出了1.9%，与基于热力图特征和基于空间性和随机森林的方法相比，分类精度分别高出8.8%和1.3%。结论 本文提出的乳腺组织病理全切片识别方法将空间相关性特征融合和RNN分类集成到一个统一模型，有助于提高图像识别准确率，为病理图像诊断工作提供了高效的辅助诊断工具。;Objective Pathological examination can be as the“gold standard”to interpret breast cancer diagnosis and its tumor types. To improve the treatment effect of breast cancer，accurate interpretation is beneficial to clarify the type of the disease early and effectively. However，it is time-consuming to check the pathological whole slide images（WSIs）manually，and the diagnosis result is easily affected by personal experience. Due to computer vision-based convolutional neural networks（CNNs）are applied to the classification task of histopathological images computer-aided diagnostic（CAD）techniques for digital images of histopathology has been developing intensively. However，it is still challenged to split largesized WSIs into small patches and each patch is processed individually without the spatial information between them. Also， the information-involved of the patches learned is not utilized in the small size patch feature aggregation process. To resolve these problems，we develop a spatial correlation-based classification method for breast histopathology images，which can re-identify the WSIs classification problem through deep learning feature fusion and recurrent neural network（RNN）based classification. Method Our demonstration consists of four aspects：1）pre-processing operation of WSIs，2）CNN-based breast patch prediction，3）image patch feature fusion，and 4）RNN-based WSIs classification. First，it is focused on a preprocessing operation on whole slide images of breast pathology，using a sliding window to cut the WSIs into patches of a suitable size in terms of a CNN. Second，CNN-based patch prediction is used to predict the possibility of cancer in each patch and each patch is encoded as a fixed-length feature. To reduce unpredictable class of cancerous patch being classified as non-cancerous patch，the ResNet34 is used as the patch classification model and penalty factors is added in Focal loss. Third，the feature fusion of image blocks can be recognized as a feature fusion niche that takes the entire grid of patches as input and the spatial correlation of patches and their surrounding patches are as feature fusion of all patches within the grid，thus block descriptors are formed. Feature fusion methods are involved in，including 1）Weight，2）Max， 3）Avg，4）Norm3，and 5）WeightNorm3. Finally，the RNN-based WSIs classification is used to pass the block feature descriptors with high probability of cancer in each WSIs to the RNN，and the RNN model is used to learn the sequence relationship between the image block feature sequences. It can expand the processing field of view of the classification model to multiple Blocks and optimize the classification accuracy of breast pathology WSIs. Additionally，we design and develop an online recognition system for assistance. Result A detailed annotated whole slide images dataset of breast pathology is constructed，and benign/malignant dichotomous classification experiments are carried out on this dataset. The results are compared to three WSIs classification methods in the self-constructed dataset，and it shows that the classification accuracy of the method can be reached to 96. 3%，which is 1. 9% higher than non-spatial correlation method. The classification accuracy of each is 8. 8% and 1. 3% higher compared to heat map features-based methods and spatial correlation and random forest-related methods. Conclusion The proposed breast pathology recognition method can improve image recognition accuracy and provide an efficient diagnostic aid for pathology image diagnosis work，which integrates feature fusion and RNN classification into a synthesized model.

A loss-based patch label denoising method for improving whole-slide image analysis using a convolutional neural network

This paper proposes a deep learning-based patch label denoising method (LossDiff) for improving the classification of whole-slide images of cancer using a convolutional neural network (CNN). Automated whole-slide image classification is often challenging, requiring a large amount of labeled data. Pathologists annotate the region of interest by marking malignant areas, which pose a high risk of introducing patch-based label noise by involving benign regions that are typically small in size within the malignant annotations, resulting in low classification accuracy with many Type-II errors. To overcome this critical problem, this paper presents a simple yet effective method for noisy patch classification. The proposed method, validated using stomach cancer images, provides a significant improvement compared to other existing methods in patch-based cancer classification, with accuracies of 98.81%, 97.30% and 89.47% for binary, ternary, and quaternary classes, respectively. Moreover, we conduct several experiments at different noise levels using a publicly available dataset to further demonstrate the robustness of the proposed method. Given the high cost of producing explicit annotations for whole-slide images and the unavoidable error-prone nature of the human annotation of medical images, the proposed method has practical implications for whole-slide image annotation and automated cancer diagnosis.

Transfer Learning Approach for Classification of Histopathology Whole Slide Images.

The classification of whole slide images (WSIs) provides physicians with an accurate analysis of diseases and also helps them to treat patients effectively. The classification can be linked to further detailed analysis and diagnosis. Deep learning (DL) has made significant advances in the medical industry, including the use of magnetic resonance imaging (MRI) scans, computerized tomography (CT) scans, and electrocardiograms (ECGs) to detect life-threatening diseases, including heart disease, cancer, and brain tumors. However, more advancement in the field of pathology is needed, but the main hurdle causing the slow progress is the shortage of large-labeled datasets of histopathology images to train the models. The Kimia Path24 dataset was particularly created for the classification and retrieval of histopathology images. It contains 23,916 histopathology patches with 24 tissue texture classes. A transfer learning-based framework is proposed and evaluated on two famous DL models, Inception-V3 and VGG-16. To improve the productivity of Inception-V3 and VGG-16, we used their pre-trained weights and concatenated these with an image vector, which is used as input for the training of the same architecture. Experiments show that the proposed innovation improves the accuracy of both famous models. The patch-to-scan accuracy of VGG-16 is improved from 0.65 to 0.77, and for the Inception-V3, it is improved from 0.74 to 0.79.

Feasibility of fully automated classification of whole slide images based on deep learning

Although microscopic analysis of tissue slides has been the basis for disease diagnosis for decades, intra- and inter-observer variabilities remain issues to be resolved. The recent introduction of digital scanners has allowed for using deep learning in the analysis of tissue images because many whole slide images (WSIs) are accessible to researchers. In the present study, we investigated the possibility of a deep learning-based, fully automated, computer-aided diagnosis system with WSIs from a stomach adenocarcinoma dataset. Three different convolutional neural network architectures were tested to determine the better architecture for tissue classifier. Each network was trained to classify small tissue patches into normal or tumor. Based on the patch-level classification, tumor probability heatmaps can be overlaid on tissue images. We observed three different tissue patterns, including clear normal, clear tumor and ambiguous cases. We suggest that longer inspection time can be assigned to ambiguous cases compared to clear normal cases, increasing the accuracy and efficiency of histopathologic diagnosis by pre-evaluating the status of the WSIs. When the classifier was tested with completely different WSI dataset, the performance was not optimal because of the different tissue preparation quality. By including a small amount of data from the new dataset for training, the performance for the new dataset was much enhanced. These results indicated that WSI dataset should include tissues prepared from many different preparation conditions to construct a generalized tissue classifier. Thus, multi-national/multi-center dataset should be built for the application of deep learning in the real world medical practice.

Automated histological classification of whole slide images of colorectal biopsy specimens.

BackgroundAn automated image analysis system, e-Pathologist, was developed to improve the quality of colorectal biopsy diagnostics in routine pathology practice.ObjectiveThe aim of the study was to evaluate the classification accuracy of the e-Pathologist image analysis software in the setting of routine pathology practice in two institutions.Materials and methodsIn total, 1328 colorectal tissue specimens were consecutively obtained from two hospitals (1077 tissues from Tokyo hospital, and 251 tissues from East hospital) and the stained specimen slides were anonymized and digitized. At least two experienced gastrointestinal pathologists evaluated each slide for pathological diagnosis. We compared the 3-tier classification results (carcinoma or suspicion of carcinoma, adenoma, and lastly negative for a neoplastic lesion) between the human pathologists and that of e-Pathologist.ResultsFor the Tokyo hospital specimens, all carcinoma tissues were correctly classified (n=112), and 9.9% (80/810) of the adenoma tissues were incorrectly classified as negative. For the East hospital specimens, 0 out of the 51 adenoma tissues were incorrectly classified as negative while 9.3% (11/118) of the carcinoma tissues were incorrectly classified as either adenoma, or negative. For the Tokyo and East hospital datasets, the undetected rate of carcinoma, undetected rate of adenoma, and over-detected proportion were 0% and 9.3%, 9.9% and 0%, and 36.1% and 27.1%, respectively.ConclusionsThis image analysis system requires some improvements; however, it has the potential to assist pathologists in quality improvement of routine pathological practice in the not too distant future.