Histopathology Image Segmentation Research Articles

Multi-level feature fusion network for nuclei segmentation in digital histopathological images

Multi-level feature fusion network for nuclei segmentation in digital histopathological images

High resolution histopathology image generation and segmentation through adversarial training.

Semantic segmentation of histopathology images can be a vital aspect of computer-aided diagnosis, and deep learning models have been effectively applied to this task with varying levels of success. However, their impact has been limited due to the small size of fully annotated datasets. Data augmentation is one avenue to address this limitation. Generative Adversarial Networks (GANs) have shown promise in this respect, but previous work has focused mostly on classification tasks applied to MR and CT images, both of which have lower resolution and scale than histopathology images. There is limited research that applies GANs as a data augmentation approach for large-scale image semantic segmentation, which requires high-quality image-mask pairs. In this work, we propose a multi-scale conditional GAN for high-resolution, large-scale histopathology image generation and segmentation. Our model consists of a pyramid of GAN structures, each responsible for generating and segmenting images at a different scale. Using semantic masks, the generative component of our model is able to synthesize histopathology images that are visually realistic. We demonstrate that these synthesized images along with their masks can be used to boost segmentation performance, especially in the semi-supervised scenario.

Gland segmentation in colorectal cancer histopathological images using U-net inspired convolutional network

Gland segmentation in colorectal cancer histopathological images using U-net inspired convolutional network

A Classification-Guided Segmentation Algorithm Based on Deep Learning for Epithelium Segmentation in Histopathological Images of Radicular Cysts.

In histopathological analysis of radicular cysts (RCs), lesions in epithelium can provide pathologists with rich information on pathologic degree, which is helpful to determine the type of periapical lesions and make precise treatment planning. Automatic segmentation and localization of epithelium from whole slide images (WSIs) can assist pathologists to complete pathological diagnosis more quickly. However, the class imbalance problem caused by the small proportion of fragmented epithelium in RCs imposes challenge on the typical automatic one-stage segmentation method. In this paper, we proposed a classification-guided segmentation algorithm (CGSA) for accurate segmentation. Our method was a two-stage model, including a classification network for region of interest (ROI) location and a segmentation network guided by classification. The classification stage eliminated most irrelevant areas and alleviated the class imbalance problem faced by the segmentation model. The results of 5-fold cross validation demonstrated that CGSA outperformed the one-stage segmentation method which was lacking in prior epithelium localization information. The epithelium segmentation achieved an overall Dice's coefficient of 0.722, and intersection over union (IoU) of 0.593, which improved by 5.5% and 5.9% respectively compared with the one-stage segmentation method using UNet.Clinical Relevance- This work presents a framework for automatic epithelium segmentation in histopathological images of RCs. It can be applied to make up for the shortcomings of manual annotation which is labor-intensive, time-consuming and objective.

Deep learning for colon cancer histopathological images analysis.

Nowadays, digital pathology plays a major role in the diagnosis and prognosis of tumours. Unfortunately, existing methods remain limited when faced with the high resolution and size of Whole Slide Images (WSIs) coupled with the lack of richly annotated datasets. Regarding the ability of the Deep Learning (DL) methods to cope with the large scale applications, such models seem like an appealing solution for tissue classification and segmentation in histopathological images. This paper focuses on the use of DL architectures to classify and highlight colon cancer regions in a sparsely annotated histopathological data context. First, we review and compare state-of-the-art Convolutional Neural networks (CNN) including the AlexNet, vgg, ResNet, DenseNet and Inception models. To cope with the shortage of rich WSI datasets, we have resorted to the use of transfer learning techniques. This strategy comes with the hallmark of relying on a large size computer vision dataset (ImageNet) to train the network and generate a rich collection of learnt features. The testing and evaluation of such models on our AiCOLO colon cancer dataset ensure accurate patch-level classification results reaching up to 96.98% accuracy rate with ResNet. The CNN models have also been tested and evaluated with the CRC-5000, nct-crc-he-100k and merged datasets. ResNet respectively achieves 96.77%, 99.76% and 99.98% for the three publicly available datasets. Then, we present a pixel-wise segmentation strategy for colon cancer WSIs through the use of both UNet and SegNet models. We introduce a multi-step training strategy as a remedy for the sparse annotation of histopathological images. UNet and SegNet are used and tested in different training scenarios including data augmentation and transfer learning and ensure up to 76.18% and 81.22% accuracy rates. Besides, we test our training strategy and models on the CRC-5000, nct-crc-he-100k and Warwick datasets. Respective accuracy rates of 98.66%, 99.12% and 78.39% were achieved by SegNet. Finally, we analyze the existing models to discover the most suitable network and the most effective training strategy for our colon tumour segmentation case study.1.

Automated Computer Aided Diagnosis Using Altered Multi-Phase Level Sets in Application to Categorize the Breast Cancer Biopsy Images

As macroscopic Breast Cancer (BC) imaging diagnostics do not provide satisfied data, microscopic image analysis is used to get optimum cell-nuclei segmentation for image categorization. The paper presents a Novel Hybrid Segmentation Method which uses the energy detail coefficients attained from Non-Subsampled Contourlet Transform for cell-nuclei segmentation of BC histopathological images. NHSM solves the local inhomogeneities problem while preserving all necessary details. The system evaluation is done using training dataset of 98 BC histopathological images and testing dataset of 26 images. Total of 25 segmented features was extracted for classification using Decision Trees, k-Nearest Neighbor, multilayer perceptron, Multiclass-Support Vector Machine (M-SVM). The proposed NHSM segmentation results were undergone through both subjective and objective performance evaluation. Result evaluation proved that NHSM along with Multi Class-Support Vector Machine classifier provides highly precise and accurate classified result for BC tissue images compared to state-of-the-art segmentation methods. Also, the edge detection model initialization is highly robust and insensitive.

MASG-GAN: A multi-view attention superpixel-guided generative adversarial network for efficient and simultaneous histopathology image segmentation and classification

Efficient analysis of Haematoxylin and Eosin stained histopathology images has become a challenge in digital pathology work-flow. We propose a Multi-view Attention Superpixel-guided Generative Adversarial Network (MASG-GAN) to achieve the multi-task learning for nuclei segmentation and benign-malignant tissue classification. Firstly, a novel superpixel segmentation approach driven by Bounded Asymmetric Gaussian Mixture Model (BAGMM) is presented for generating superpixel-prior probability map with high-level semantics. Then, we develop a generator network that integrates student-branch and teacher-branch. Concretely, the teacher-branch takes superpixel-prior probability map as input and guides the student-branch for accurate segmentation and classification. Then in generator, we build a light-weight U-shaped block (LUB) that consists of depthwise separable convolutions with mini encoding-decoding structure to reduce network computational cost. Finally, a Multi-view Attention Module (MVAM) is designed for further enhance the segmentation quality of nuclei with small area and unclear boundary. Extensive experiments on five benchmark datasets demonstrate that our pipeline outperforms some state-of-the-art methods, especially in terms of efficiency.

Polar representation-based cell nucleus segmentation in non-small cell lung cancer histopathological images

Image segmentation is a major area of interest within the field of medical image analysis and processing. In the last few decades, cell nucleus segmentation in non-small lung cancer histopathological images has spawned considerable critical attention. However, most of the previously presented studies have only been concerned with revealing the representation of contours in Cartesian coordinates and suffer from a lack of clarity in connecting the points into a whole contour. Bearing this in mind, we propose a polar representation-based nucleus segmentation model by leveraging the fully convolutional one-stage object detection. In general, center classification and length regression are simultaneously adapted to yield the contour of the nucleus in a polar coordinate. To evaluate the performance of our approach, the comparing experiments are conducted between the state-of-the-art deep learning-based object detection algorithms and the proposed method on one manually collected dataset. Experimental results demonstrate that our model outperformed the state-of-the-art both in efficiency and effectiveness.

Histopathology image segmentation and classification for cancer revelation

Histopathology image segmentation and classification for cancer revelation

Minimizing Labeling Cost for Nuclei Instance Segmentation and Classification with Cross-domain Images and Weak Labels

Nucleus instance segmentation and classification in histopathological images is an essential prerequisite in pathology diagnosis/prognosis. However, nucleus annotations (e.g., segmentation and labeling) require domain experts, and annotating nuclei at pixel-level is time-consuming and labor-intensive. Moreover, nuclei from different cancer types vary in shapes and appearances. These inter-cancer variations require careful annotations for specific cancer types. Therefore, to minimize the labeling cost, we propose a novel application that considers each cancer type as an individual domain and apply domain adaptation techniques to improve the segmentation/classification performance among different cancer types. Unlike the previous studies that focus on unsupervised or weakly-supervised domain adaptation independently, we would like to discover what kinds of labeling can achieve the most cost-effective domain adaptation performance in nucleus instance segmentation and classification. Specifically, we propose a unified framework that is applicable to different level annotations: no annotations, image-level, and point-level annotations. Cyclic adaptation with pseudo labels and adversarial discriminator are utilized for unsupervised domain alignment. Image-level or point-level annotations are additionally adopted to supervise the nucleus classification and refine the pseudo labels. Experiments demonstrate the effectiveness and efficacy of the proposed framework (jointly using unsupervised and weakly supervised learning) on adapting the segmentation and classification model from one cancer type to 18 other cancer types.

Weakly Supervised Histopathology Image Segmentation With Sparse Point Annotations.

Digital histopathology image segmentation can facilitate computer-assisted cancer diagnostics. Given the difficulty of obtaining manual annotations, weak supervision is more suitable for the task than full supervision is. However, most weakly supervised models are not ideal for handling severe intra-class heterogeneity and inter-class homogeneity in histopathology images. Therefore, we propose a novel end-to-end weakly supervised learning framework named WESUP. With only sparse point annotations, it performs accurate segmentation and exhibits good generalizability. The training phase comprises two major parts, hierarchical feature representation and deep dynamic label propagation. The former uses superpixels to capture local details and global context from the convolutional feature maps obtained via transfer learning. The latter recognizes the manifold structure of the hierarchical features and identifies potential targets with the sparse annotations. Moreover, these two parts are trained jointly to improve the performance of the whole framework. To further boost test performance, pixel-wise inference is adopted for finer prediction. As demonstrated by experimental results, WESUP is able to largely resolve the confusion between histological foreground and background. It outperforms several state-of-the-art weakly supervised methods on a variety of histopathology datasets with minimal annotation efforts. Trained by very sparse point annotations, WESUP can even beat an advanced fully supervised segmentation network.

System for quantitative evaluation of DAB&H-stained breast cancer biopsy digital images (CHISEL)

This study presents CHISEL (Computer-assisted Histopathological Image Segmentation and EvaLuation), an end-to-end system capable of quantitative evaluation of benign and malignant (breast cancer) digitized tissue samples with immunohistochemical nuclear staining of various intensity and diverse compactness. It stands out with the proposed seamless segmentation based on regions of interest cropping as well as the explicit step of nuclei cluster splitting followed by a boundary refinement. The system utilizes machine learning and recursive local processing to eliminate distorted (inaccurate) outlines. The method was validated using two labeled datasets which proved the relevance of the achieved results. The evaluation was based on the IISPV dataset of tissue from biopsy of breast cancer patients, with markers of T cells, along with Warwick Beta Cell Dataset of DAB&H-stained tissue from postmortem diabetes patients. Based on the comparison of the ground truth with the results of the detected and classified objects, we conclude that the proposed method can achieve better or similar results as the state-of-the-art methods. This system deals with the complex problem of nuclei quantification in digitalized images of immunohistochemically stained tissue sections, achieving best results for DAB&H-stained breast cancer tissue samples. Our method has been prepared with user-friendly graphical interface and was optimized to fully utilize the available computing power, while being accessible to users with fewer resources than needed by deep learning techniques.

A hybrid deep learning approach for gland segmentation in prostate histopathological images

BackgroundIn digital pathology, the morphology and architecture of prostate glands have been routinely adopted by pathologists to evaluate the presence of cancer tissue. The manual annotations are operator-dependent, error-prone and time-consuming. The automated segmentation of prostate glands can be very challenging too due to large appearance variation and serious degeneration of these histological structures. MethodA new image segmentation method, called RINGS (Rapid IdentificatioN of Glandural Structures), is presented to segment prostate glands in histopathological images. We designed a novel glands segmentation strategy using a multi-channel algorithm that exploits and fuses both traditional and deep learning techniques. Specifically, the proposed approach employs a hybrid segmentation strategy based on stroma detection to accurately detect and delineate the prostate glands contours. ResultsAutomated results are compared with manual annotations and seven state-of-the-art techniques designed for glands segmentation. Being based on stroma segmentation, no performance degradation is observed when segmenting healthy or pathological structures. Our method is able to delineate the prostate gland of the unknown histopathological image with a dice score of 90.16 % and outperforms all the compared state-of-the-art methods. ConclusionsTo the best of our knowledge, the RINGS algorithm is the first fully automated method capable of maintaining a high sensitivity even in the presence of severe glandular degeneration. The proposed method will help to detect the prostate glands accurately and assist the pathologists to make accurate diagnosis and treatment. The developed model can be used to support prostate cancer diagnosis in polyclinics and community care centres.

A new complete color normalization method for H&E stained histopatholgical images

The popularity of digital histopathology is growing rapidly in the development of computer aided disease diagnosis systems. However, the color variations due to manual cell sectioning and stain concentration make the process challenging in various digital pathological image analysis such as histopathological image segmentation and classification. Hence, the normalization of these variations are needed to obtain the promising results. The proposed research intends to introduce a reliable and robust new complete color normalization method, addressing the problems of color and stain variability. The new complete color normalization involves three phases, namely enhanced fuzzy illuminant normalization, fuzzy-based stain normalization, and modified spectral normalization. The extensive simulations are performed and validated on histopathological images. The presented algorithm outperforms the existing conventional normalization methods by overcoming the certain limitations and challenges. As per the experimental quality metrics and comparative analysis, the proposed algorithm performs efficiently and provides promising results.

FAST INTERACTIVE REGIONAL PATTERN MERGING FOR GENERIC TISSUE SEGMENTATION IN HISTOPATHOLOGY IMAGES

The image segmentation of histopathological tissue images has always been a challenge due to the overlapping of tissue color distributions, the complexity of extracellular texture and the large image size. In this paper, we introduce a new region-merging algorithm, namely, the Regional Pattern Merging (RPM) for interactive color image segmentation and annotation, by efficiently retrieving and applying the user’s prior knowledge of stroke-based interaction. Low-level color/texture features of each region are used to compose a regional pattern adapted to differentiating a foreground object from the background scene. This iterative region-merging is based on a modified Region Adjacency Graph (RAG) model built from initial segmented results of the mean shift to speed up the merging process. The foreground region of interest (ROI) is segmented by the reduction of the background region and discrimination of uncertain regions. We then compare our method against state-of-the-art interactive image segmentation algorithms in both natural images and histological images. Taking into account the homogeneity of both color and texture, the resulting semi-supervised classification and interactive segmentation capture histological structures more completely than other intensity or color-based methods. Experimental results show that the merging of the RAG model runs in a linear time according to the number of graph edges, which is essentially faster than both traditional graph-based and region-based methods.

Interpretability-Driven Sample Selection Using Self Supervised Learning for Disease Classification and Segmentation.

In supervised learning for medical image analysis, sample selection methodologies are fundamental to attain optimum system performance promptly and with minimal expert interactions (e.g. label querying in an active learning setup). In this article we propose a novel sample selection methodology based on deep features leveraging information contained in interpretability saliency maps. In the absence of ground truth labels for informative samples, we use a novel self supervised learning based approach for training a classifier that learns to identify the most informative sample in a given batch of images. We demonstrate the benefits of the proposed approach, termed Interpretability-Driven Sample Selection (IDEAL), in an active learning setup aimed at lung disease classification and histopathology image segmentation. We analyze three different approaches to determine sample informativeness from interpretability saliency maps: (i) an observational model stemming from findings on previous uncertainty-based sample selection approaches, (ii) a radiomics-based model, and (iii) a novel data-driven self-supervised approach. We compare IDEAL to other baselines using the publicly available NIH chest X-ray dataset for lung disease classification, and a public histopathology segmentation dataset (GLaS), demonstrating the potential of using interpretability information for sample selection in active learning systems. Results show our proposed self supervised approach outperforms other approaches in selecting informative samples leading to state of the art performance with fewer samples.

Multi-scale fully convolutional neural networks for histopathology image segmentation: From nuclear aberrations to the global tissue architecture.

Histopathologic diagnosis relies on simultaneous integration of information from a broad range of scales, ranging from nuclear aberrations (≈O(0.1μm)) through cellular structures (≈O(10μm)) to the global tissue architecture (⪆O(1mm)). To explicitly mimic how human pathologists combine multi-scale information, we introduce a family of multi-encoder fully-convolutional neural networks with deep fusion. We present a simple block for merging model paths with differing spatial scales in a spatial relationship-preserving fashion, which can readily be included in standard encoder-decoder networks. Additionally, a context classification gate block is proposed as an alternative for the incorporation of global context. Our experiments were performed on three publicly available whole-slide images of recent challenges (PAIP 2019: hepatocellular carcinoma segmentation; BACH 2020: breast cancer segmentation; CAMELYON 2016: metastasis detection in lymph nodes). The multi-scale architectures consistently outperformed the baseline single-scale U-Nets by a large margin. They benefit from local as well as global context and particularly a combination of both. If feature maps from different scales are fused, doing so in a manner preserving spatial relationships was found to be beneficial. Deep guidance by a context classification loss appeared to improve model training at low computational costs. All multi-scale models had a reduced GPU memory footprint compared to ensembles of individual U-Nets trained on different image scales. Additional path fusions were shown to be possible at low computational cost, opening up possibilities for further, systematic and task-specific architecture optimisation. The findings demonstrate the potential of the presented family of human-inspired, end-to-end trainable, multi-scale multi-encoder fully-convolutional neural networks to improve deep histopathologic diagnosis by extensive integration of largely different spatial scales.

A Hybrid-Attention Nested UNet for Nuclear Segmentation in Histopathological Images.

Nuclear segmentation of histopathological images is a crucial step in computer-aided image analysis. There are complex, diverse, dense, and even overlapping nuclei in these histopathological images, leading to a challenging task of nuclear segmentation. To overcome this challenge, this paper proposes a hybrid-attention nested UNet (Han-Net), which consists of two modules: a hybrid nested U-shaped network (H-part) and a hybrid attention block (A-part). H-part combines a nested multi-depth U-shaped network and a dense network with full resolution to capture more effective features. A-part is used to explore attention information and build correlations between different pixels. With these two modules, Han-Net extracts discriminative features, which effectively segment the boundaries of not only complex and diverse nuclei but also small and dense nuclei. The comparison in a publicly available multi-organ dataset shows that the proposed model achieves the state-of-the-art performance compared to other models.

AttentionBoost: Learning What to Attend for Gland Segmentation in Histopathological Images by Boosting Fully Convolutional Networks.

Fully convolutional networks (FCNs) are widely used for instance segmentation. One important challenge is to sufficiently train these networks to yield good generalizations for hard-to-learn pixels, correct prediction of which may greatly affect the success. A typical group of such hard-to-learn pixels are boundaries between instances. Many studies have developed strategies to pay more attention to learning these boundary pixels. They include designing multi-task networks with an additional task of boundary prediction and increasing the weights of boundary pixels' predictions in the loss function. Such strategies require defining what to attend beforehand and incorporating this defined attention to the learning model. However, there may exist other groups of hard-to-learn pixels and manually defining and incorporating the appropriate attention for each group may not be feasible. In order to provide an adaptable solution to learn different groups of hard-to-learn pixels, this article proposes AttentionBoost, which is a new multi-attention learning model based on adaptive boosting, for the task of gland instance segmentation in histopathological images. AttentionBoost designs a multi-stage network and introduces a new loss adjustment mechanism for an FCN to adaptively learn what to attend at each stage directly on image data without necessitating any prior definition. This mechanism modulates the attention of each stage to correct the mistakes of previous stages, by adjusting the loss weight of each pixel prediction separately with respect to how accurate the previous stages are on this pixel. Working on histopathological images of colon tissues, our experiments demonstrate that the proposed AttentionBoost model improves the results of gland segmentation compared to its counterparts.

Automated segmentation technique with self‐driven post‐processing for histopathological breast cancer images

Automated segmentation of histopathological images is a challenging task to detect cancerous cells in breast tissue. Recent reviews state high accuracy to segment image, but depends on user input, say window area size, time steps, level set, magnification factor and so on. To extract the region of interest effectively, the subject expert performs post-processing operations several times on the segmentation results with different input values for different parameters say, area opening, fill holes and selects most appropriate enhanced image required for further analysis. The authors proposed an automated segmentation technique followed by self-driven post-processing operations to detect cancerous cells effectively. The post-processing method itself determines the value of different parameters for different operations based on segmented results obtained. The proposed technique has the following features: (i) technique is context sensitive; (ii) no prior setting of time step, weighted area coefficient parameters is required; (iii) magnification independent; (iv) post-processing operations are self-driven which enhance segmentation results adaptively. The experimental results are compared with four state-of-the-art techniques: fuzzy C-means, spatial fuzzy C-means, spatial neutrosophic distance regularised level set and convolutional neural network-based PangNet. Experimental results obtained on two publicly available data sets show that the proposed technique outperforms effectively.