High-resolution Histopathological Images Research Articles

Deep learning-based identification of esophageal cancer subtypes through analysis of high-resolution histopathology images.

Esophageal cancer (EC) remains a significant health challenge globally, with increasing incidence and high mortality rates. Despite advances in treatment, there remains a need for improved diagnostic methods and understanding of disease progression. This study addresses the significant challenges in the automatic classification of EC, particularly in distinguishing its primary subtypes: adenocarcinoma and squamous cell carcinoma, using histopathology images. Traditional histopathological diagnosis, while being the gold standard, is subject to subjectivity and human error and imposes a substantial burden on pathologists. This study proposes a binary class classification system for detecting EC subtypes in response to these challenges. The system leverages deep learning techniques and tissue-level labels for enhanced accuracy. We utilized 59 high-resolution histopathological images from The Cancer Genome Atlas (TCGA) Esophageal Carcinoma dataset (TCGA-ESCA). These images were preprocessed, segmented into patches, and analyzed using a pre-trained ResNet101 model for feature extraction. For classification, we employed five machine learning classifiers: Support Vector Classifier (SVC), Logistic Regression (LR), Decision Tree (DT), AdaBoost (AD), Random Forest (RF), and a Feed-Forward Neural Network (FFNN). The classifiers were evaluated based on their prediction accuracy on the test dataset, yielding results of 0.88 (SVC and LR), 0.64 (DT and AD), 0.82 (RF), and 0.94 (FFNN). Notably, the FFNN classifier achieved the highest Area Under the Curve (AUC) score of 0.92, indicating its superior performance, followed closely by SVC and LR, with a score of 0.87. This suggested approach holds promising potential as a decision-support tool for pathologists, particularly in regions with limited resources and expertise. The timely and precise detection of EC subtypes through this system can substantially enhance the likelihood of successful treatment, ultimately leading to reduced mortality rates in patients with this aggressive cancer.

Image Processing for Automatic Cell Nucleus Segmentation Using Super pixel and Clustering Methods on Histopathological Images

On a daily basis, it appears that the number of cancer cases and cancer-related deaths are increasing. Early detection and treatment of the malignant region are critical for successful treatment. Early detection of sick cells is made possible with the use of computer-assisted programmes, which are then diagnosed by experienced pathologists due to their efforts. Using computer-aided programmes, this research found that cell nuclei could be automatically detected in high-resolution histopathological images using global segmentation methods such as k-Means and Fuzzy C Means and algorithms from superpixel segmentation methods such as SLIC, Quick-shift, Felzenszwalb, Watershed, and ERS. Using high-quality histopathology pictures, the researchers discovered that the k-means and FCM algorithms performed significantly better than the baseline techniques in the study. In terms of precision, the Quickshift and SLIC approaches produced superior outcomes. The K-means and FCM algorithms perform best in the F-M test, and the true negative ratio is more successful in the Quickshift and SLIC methods than in the F-M test.

TIST: Transcriptome and Histopathological Image Integrative Analysis for Spatial Transcriptomics

Sequencing-based spatial transcriptomics (ST) is an emerging technology to study in situ gene expression patterns at the whole-genome scale. Currently, ST data analysis is still complicated by high technical noises and low resolution. In addition to the transcriptomic data, matched histopathological images are usually generated for the same tissue sample along the ST experiment. The matched high-resolution histopathological images provide complementary cellular phenotypical information, providing an opportunity to mitigate the noises in ST data. We present a novel ST data analysis method called transcriptome and histopathological image integrative analysis for ST (TIST), which enables the identification of spatial clusters (SCs) and the enhancement of spatial gene expression patterns by integrative analysis of matched transcriptomic data and images. TIST devises a histopathological feature extraction method based on Markov random field (MRF) to learn the cellular features from histopathological images, and integrates them with the transcriptomic data and location information as a network, termed TIST-net. Based on TIST-net, SCs are identified by a random walk-based strategy, and gene expression patterns are enhanced by neighborhood smoothing. We benchmark TIST on both simulated datasets and 32 real samples against several state-of-the-art methods. Results show that TIST is robust to technical noises on multiple analysis tasks for sequencing-based ST data and can find interesting microstructures in different biological scenarios. TIST is available at http://lifeome.net/software/tist/ and https://ngdc.cncb.ac.cn/biocode/tools/BT007317.

Automatic Cell Nucleus Segmentation Using Superpixels and Clustering Methods in Histopathological Images

It is seen that there is an increase in cancer and cancer-related deaths day by day. Early diagnosis is vital for the early treatment of the cancerous area. Computer-aided programs allow for early diagnosis of unhealthy cells that specialist pathologists diagnose as a result of efforts. In this study, kMeans and Fuzzy C Means methods, which are among the global segmentation methods, and SLIC, Quickshift, Felzenszwalb, Watershed and ERS algorithms, which are among the superpixel segmentation methods, were used for automatic cell nucleus detection in high resolution histopathological images with computer aided programs. As a result of the study, the success performances of the segmentation algorithms were analyzed and evaluated. It is seen that better success is obtained in watershed and FCM algorithms in high resolution histopathological images used. Quickshift and SLIC methods gave better results in terms of precision. It is seen that there are k-Means and FCM algorithms that provide the best performance in F measure (F-M) and the true negative rate (TNR) is more successful in Quickshift, k-Means and SLIC methods.

Fusion of whole and part features for the classification of histopathological image of breast tissue.

Nowadays Computer-Aided Diagnosis (CAD) models, particularly those based on deep learning, have been widely used to analyze histopathological images in breast cancer diagnosis. However, due to the limited availability of such images, it is always tedious to train deep learning models that require a huge amount of training data. In this paper, we propose a new deep learning-based CAD framework that can work with less amount of training data. We use pre-trained models to extract image features that can then be used with any classifier. Our proposed features are extracted by the fusion of two different types of features (foreground and background) at two levels (whole-level and part-level). Foreground and background feature to capture information about different structures and their layout in microscopic images of breast tissues. Similarly, part-level and whole-level features capture are useful in detecting interesting regions scattered in high-resolution histopathological images at local and whole image levels. At each level, we use VGG16 models pre-trained on ImageNet and Places datasets to extract foreground and background features, respectively. All features are extracted from mid-level pooling layers of such models. We show that proposed fused features with a Support Vector Classifier (SVM) produce better classification accuracy than recent methods on BACH dataset and our approach is orders of magnitude faster than the best performing recent method (EMS-Net). We believe that our method would be another alternative in the diagnosis of breast cancer because of performance and prediction time.

Classification of Breast Cancer Images Using Ensembles of Transfer Learning

It is a challenging task to estimate the cancerous cells and tissues via computer-aided diagnosis systems on high-resolution histopathological images. In this study, it is suggested to use transfer learning and ensemble learning methods together in order to reduce the difficulty of this task and better diagnose cancer patients. In the studies, histopathological images with 40× and 100× magnification factors are analyzed. In order to prove the success of the study with experimental studies, firstly, the results provided by pre-modeled deep learning architectures trained by histopathological image dataset, then the results acquired by different transfer learning approaches and the results obtained with the ensembles of deeply learned features using transfer learning methods are presented comparatively. Three different approaches are applied for transfer learning by fine-tuning the pre-trained convolution neural networks. In the experimental section, results of single classifiers (i.e., support vector machines, logistic regression, k-nearest neighbor and bagging) are presented by employing features of CNN models obtained by defined transfer learning approaches. Then, decisions of each classifier model are combined separately by weighted decision fusion (WDF) and stacking decision fusion (SDF) ensemble learning methods that have proven to improve the classification performance of the proposed classification system.

Multipatch-GLCM for Texture Feature Extraction on Classification of the Colon Histopathology Images using Deep Neural Network with GPU Acceleration

Cancer is one of the leading causes of death in the world. It is the main reason why research in this field becomes challenging. Not only for the pathologist but also from the view of a computer scientist. Hematoxylin and Eosin (H&E) images are the most common modalities used by the pathologist for cancer detection. The status of cancer with histopathology images can be classified based on the shape, morphology, intensity, and texture of the image. The use of full high-resolution histopathology images will take a longer time for the extraction of all information due to the huge amount of data. This study proposed advance texture extraction by multi-patch images pixel method with sliding windows that minimize loss of information in each pixel patch. We use texture feature Gray Level Co-Occurrence Matrix (GLCM) with a mean-shift filter as the data pre-processing of the images. The mean-shift filter is a low-pass filter technique that considers the surrounding pixels of the images. The proposed GLCM method is then trained using Deep Neural Networks (DNN) and compared to other classification techniques for benchmarking. For training, we use two hardware: NVIDIA GPU GTX-980 and TESLA K40c. According to the study, Deep Neural Network outperforms other classifiers with the highest accuracy and deviation standard 96.72±0.48 for four cross-validations. The additional information is that training using Theano framework is faster than Tensorflow for both in GTX-980 and Tesla K40c.

Attention by Selection: A Deep Selective Attention Approach to Breast Cancer Classification.

Deep learning approaches are widely applied to histopathological image analysis due to the impressive levels of performance achieved. However, when dealing with high-resolution histopathological images, utilizing the original image as input to the deep learning model is computationally expensive, while resizing the original image to achieve low resolution incurs information loss. Some hard-attention based approaches have emerged to select possible lesion regions from images to avoid processing the original image. However, these hard-attention based approaches usually take a long time to converge with weak guidance, and valueless patches may be trained by the classifier. To overcome this problem, we propose a deep selective attention approach that aims to select valuable regions in the original images for classification. In our approach, a decision network is developed to decide where to crop and whether the cropped patch is necessary for classification. These selected patches are then trained by the classification network, which then provides feedback to the decision network to update its selection policy. With such a co-evolution training strategy, we show that our approach can achieve a fast convergence rate and high classification accuracy. Our approach is evaluated on a public breast cancer histopathological image database, where it demonstrates superior performance compared to state-of-the-art deep learning approaches, achieving approximately 98% classification accuracy while only taking 50% of the training time of the previous hard-attention approach.

A deep learning algorithm for one-step contour aware nuclei segmentation of histopathology images

This paper addresses the task of nuclei segmentation in high-resolution histopathology images. We propose an automatic end-to-end deep neural network algorithm for segmentation of individual nuclei. A nucleus-boundary model is introduced to predict nuclei and their boundaries simultaneously using a fully convolutional neural network. Given a color-normalized image, the model directly outputs an estimated nuclei map and a boundary map. A simple, fast, and parameter-free post-processing procedure is performed on the estimated nuclei map to produce the final segmented nuclei. An overlapped patch extraction and assembling method is also designed for seamless prediction of nuclei in large whole-slide images. We also show the effectiveness of data augmentation methods for nuclei segmentation task. Our experiments showed our method outperforms prior state-of-the-art methods. Moreover, it is efficient that one 1000×1000 image can be segmented in less than 5 s. This makes it possible to precisely segment the whole-slide image in acceptable time. The source code is available at https://github.com/easycui/nuclei_segmentation . Graphical Abstract The neural network for nuclei segmentation.

Convolutional neural network initialized active contour model with adaptive ellipse fitting for nuclear segmentation on breast histopathological images.

Automated detection and segmentation of nuclei from high-resolution histopathological images is a challenging problem owing to the size and complexity of digitized histopathologic images. In the context of breast cancer, the modified Bloom-Richardson Grading system is highly correlated with the morphological and topological nuclear features are highly correlated with Modified Bloom-Richardson grading. Therefore, to develop a computer-aided prognosis system, automated detection and segmentation of nuclei are critical prerequisite steps. We present a method for automated detection and segmentation of breast cancer nuclei named a convolutional neural network initialized active contour model with adaptive ellipse fitting (CoNNACaeF). The CoNNACaeF model is able to detect and segment nuclei simultaneously, which consist of three different modules: convolutional neural network (CNN) for accurate nuclei detection, (2) region-based active contour (RAC) model for subsequent nuclear segmentation based on the initial CNN-based detection of nuclear patches, and (3) adaptive ellipse fitting for overlapping solution of clumped nuclear regions. The performance of the CoNNACaeF model is evaluated on three different breast histological data sets, comprising a total of 257 H&E-stained images. The model is shown to have improved detection accuracy of F-measure 80.18%, 85.71%, and 80.36% and average area under precision-recall curves (AveP) 77%, 82%, and 74% on a total of 3 million nuclei from 204 whole slide images from three different datasets. Additionally, CoNNACaeF yielded an F-measure at 74.01% and 85.36%, respectively, for two different breast cancer datasets. The CoNNACaeF model also outperformed the three other state-of-the-art nuclear detection and segmentation approaches, which are blue ratio initialized local region active contour, iterative radial voting initialized local region active contour, and maximally stable extremal region initialized local region active contour models.

Efficient FPGA Implementation of Automatic Nuclei Detection in Histopathology Images

Accurate and efficient detection of cell nuclei is an important step towards the development of a pathology-based Computer Aided Diagnosis. Generally, high-resolution histopathology images are very large, in the order of billion pixels, therefore nuclei detection is a highly compute intensive task, and software implementation requires a significant amount of processing time. To assist the doctors in real time, special hardware accelerators, which can reduce the processing time, are required. In this paper, we propose a Field Programmable Gate Array (FPGA) implementation of automated nuclei detection algorithm using generalized Laplacian of Gaussian filters. The experimental results show that the implemented architecture has the potential to provide a significant improvement in processing time without losing detection accuracy.

Cell segmentation in histopathological images with deep learning algorithms by utilizing spatial relationships.

In many computerized methods for cell detection, segmentation, and classification in digital histopathology that have recently emerged, the task of cell segmentation remains a chief problem for image processing in designing computer-aided diagnosis (CAD) systems. In research and diagnostic studies on cancer, pathologists can use CAD systems as second readers to analyze high-resolution histopathological images. Since cell detection and segmentation are critical for cancer grade assessments, cellular and extracellular structures should primarily be extracted from histopathological images. In response, we sought to identify a useful cell segmentation approach with histopathological images that uses not only prominent deep learning algorithms (i.e., convolutional neural networks, stacked autoencoders, and deep belief networks), but also spatial relationships, information of which is critical for achieving better cell segmentation results. To that end, we collected cellular and extracellular samples from histopathological images by windowing in small patches with various sizes. In experiments, the segmentation accuracies of the methods used improved as the window sizes increased due to the addition of local spatial and contextual information. Once we compared the effects of training sample size and influence of window size, results revealed that the deep learning algorithms, especially convolutional neural networks and partly stacked autoencoders, performed better than conventional methods in cell segmentation.

Clinically-inspired automatic classification of ovarian carcinoma subtypes

Context: It has been shown that ovarian carcinoma subtypes are distinct pathologic entities with differing prognostic and therapeutic implications. Histotyping by pathologists has good reproducibility, but occasional cases are challenging and require immunohistochemistry and subspecialty consultation. Motivated by the need for more accurate and reproducible diagnoses and to facilitate pathologists’ workflow, we propose an automatic framework for ovarian carcinoma classification. Materials and Methods: Our method is inspired by pathologists’ workflow. We analyse imaged tissues at two magnification levels and extract clinically-inspired color, texture, and segmentation-based shape descriptors using image-processing methods. We propose a carefully designed machine learning technique composed of four modules: A dissimilarity matrix, dimensionality reduction, feature selection and a support vector machine classifier to separate the five ovarian carcinoma subtypes using the extracted features. Results: This paper presents the details of our implementation and its validation on a clinically derived dataset of eighty high-resolution histopathology images. The proposed system achieved a multiclass classification accuracy of 95.0% when classifying unseen tissues. Assessment of the classifier’s confusion (confusion matrix) between the five different ovarian carcinoma subtypes agrees with clinician’s confusion and reflects the difficulty in diagnosing endometrioid and serous carcinomas. Conclusions: Our results from this first study highlight the difficulty of ovarian carcinoma diagnosis which originate from the intrinsic class-imbalance observed among subtypes and suggest that the automatic analysis of ovarian carcinoma subtypes could be valuable to clinician’s diagnostic procedure by providing a second opinion.

Stacked Sparse Autoencoder (SSAE) for Nuclei Detection on Breast Cancer Histopathology Images.

Automated nuclear detection is a critical step for a number of computer assisted pathology related image analysis algorithms such as for automated grading of breast cancer tissue specimens. The Nottingham Histologic Score system is highly correlated with the shape and appearance of breast cancer nuclei in histopathological images. However, automated nucleus detection is complicated by 1) the large number of nuclei and the size of high resolution digitized pathology images, and 2) the variability in size, shape, appearance, and texture of the individual nuclei. Recently there has been interest in the application of "Deep Learning" strategies for classification and analysis of big image data. Histopathology, given its size and complexity, represents an excellent use case for application of deep learning strategies. In this paper, a Stacked Sparse Autoencoder (SSAE), an instance of a deep learning strategy, is presented for efficient nuclei detection on high-resolution histopathological images of breast cancer. The SSAE learns high-level features from just pixel intensities alone in order to identify distinguishing features of nuclei. A sliding window operation is applied to each image in order to represent image patches via high-level features obtained via the auto-encoder, which are then subsequently fed to a classifier which categorizes each image patch as nuclear or non-nuclear. Across a cohort of 500 histopathological images (2200 × 2200) and approximately 3500 manually segmented individual nuclei serving as the groundtruth, SSAE was shown to have an improved F-measure 84.49% and an average area under Precision-Recall curve (AveP) 78.83%. The SSAE approach also out-performed nine other state of the art nuclear detection strategies.