Nuclei Segmentation Of Histopathology Images Research Articles

Unsupervised Color-Based Nuclei Segmentation in Histopathology Images with Various Color Spaces and K Values Selection

The development of digital pathology offers a significant opportunity to evaluate and analyze the whole slides of disease tissue effectively. In particular, the segmentation of nuclei from histopathology images plays an important role in quantitatively measuring and evaluating the acquired diseased tissue. There are many automatic methods to segment cell nuclei in histopathology images. One widely used unsupervised segmentation approach is based on standard k-means or fuzzy c-means (FCM) to process the color histopathology images to segment cell nuclei. Compared with the supervised learning method, this approach can obtain segmented nuclei without annotated nuclei labels for training, which saves a lot of labeling and training time. The color space and [Formula: see text] value among this method plays a crucial role in determining the nuclei segmentation performance. However, few works have investigated various color spaces and [Formula: see text] value selection simultaneously in unsupervised color-based nuclei segmentation with [Formula: see text]-means or FCM algorithms. In this study, we will present color-based nuclei segmentation methods with standard [Formula: see text]-means and FCM algorithms for histopathology images. Several color spaces of Haematoxylin and Eosin (H&E) stained histopathology data and various [Formula: see text] values among [Formula: see text]-means and FCM are investigated correspondingly to explore the suitable selection for nuclei segmentation. A comprehensive nuclei dataset with 7 different organs is used to validate our proposed method. Related experimental results indicate that [Formula: see text] and the YCbCr color spaces with a [Formula: see text] of 4 are more reasonable for nuclei segmentation via [Formula: see text]-means, while the [Formula: see text] color space with [Formula: see text] of 4 is useful via FCM.

SFE-Net: Spatial-Frequency Enhancement Network for robust nuclei segmentation in histopathology images

Morphological features of individual nuclei serve as a dependable foundation for pathologists in making accurate diagnoses. Existing methods that rely on spatial information for feature extraction have achieved commendable results in nuclei segmentation tasks. However, these approaches are not sufficient to extract edge information of nuclei with small sizes and blurred outlines. Moreover, the lack of attention to the interior of the nuclei leads to significant internal inconsistencies. To address these challenges, we introduce a novel Spatial-Frequency Enhancement Network (SFE-Net) to incorporate spatial-frequency features and promote intra-nuclei consistency for robust nuclei segmentation. Specifically, SFE-Net incorporates a distinctive Spatial-Frequency Feature Extraction module and a Spatial-Guided Feature Enhancement module, which are designed to preserve spatial-frequency information and enhance feature representation respectively, to achieve comprehensive extraction of edge information. Furthermore, we introduce the Label-Guided Distillation method, which utilizes semantic features to guide the segmentation network in strengthening boundary constraints and learning the intra-nuclei consistency of individual nuclei, to improve the robustness of nuclei segmentation. Extensive experiments on three publicly available histopathology image datasets (MoNuSeg, TNBC and CryoNuSeg) demonstrate the superiority of our proposed method, which achieves 79.23%, 81.96% and 73.26% Aggregated Jaccard Index, respectively. The proposed model is available at https://github.com/jinshachen/SFE-Net.

Learning to Generalize over Subpartitions for Heterogeneity-Aware Domain Adaptive Nuclei Segmentation

Abstract Annotation scarcity and cross-modality/stain data distribution shifts are two major obstacles hindering the application of deep learning models for nuclei analysis, which holds a broad spectrum of potential applications in digital pathology. Recently, unsupervised domain adaptation (UDA) methods have been proposed to mitigate the distributional gap between different imaging modalities for unsupervised nuclei segmentation in histopathology images. However, existing UDA methods are built upon the assumption that data distributions within each domain should be uniform. Based on the over-simplified supposition, they propose to align the histopathology target domain with the source domain integrally, neglecting severe intra-domain discrepancy over subpartitions incurred by mixed cancer types and sampling organs. In this paper, for the first time, we propose to explicitly consider the heterogeneity within the histopathology domain and introduce open compound domain adaptation (OCDA) to resolve the crux. In specific, a two-stage disentanglement framework is proposed to acquire domain-invariant feature representations at both image and instance levels. The holistic design addresses the limitations of existing OCDA approaches which struggle to capture instance-wise variations. Two regularization strategies are specifically devised herein to leverage the rich subpartition-specific characteristics in histopathology images and facilitate subdomain decomposition. Moreover, we propose a dual-branch nucleus shape and structure preserving module to prevent nucleus over-generation and deformation in the synthesized images. Experimental results on both cross-modality and cross-stain scenarios over a broad range of diverse datasets demonstrate the superiority of our method compared with state-of-the-art UDA and OCDA methods. Graphical abstract

BAWGNet: Boundary aware wavelet guided network for the nuclei segmentation in histopathology images

Precise cell nucleus segmentation is very critical in many biologically related analyses and disease diagnoses. However, the variability in nuclei structure, color, and modalities of histopathology images make the automatic computer-aided nuclei segmentation task very difficult. Traditional encoder–decoder based deep learning schemes mainly utilize the spatial domain information that may limit the performance of recognizing small nuclei regions in subsequent downsampling operations. In this paper, a boundary aware wavelet guided network (BAWGNet) is proposed by incorporating a boundary aware unit along with an attention mechanism based on a wavelet domain guidance in each stage of the encoder–decoder output. Here the high-frequency 2 Dimensional discrete wavelet transform (2D-DWT) coefficients are utilized in the attention mechanism to guide the spatial information obtained from the encoder–decoder output stages to leverage the nuclei segmentation task. On the other hand, the boundary aware unit (BAU) captures the nuclei’s boundary information, ensuring accurate prediction of the nuclei pixels in the edge region. Furthermore, the preprocessing steps used in our methodology confirm the data’s uniformity by converting it to similar color statistics. Extensive experimentations conducted on three benchmark histopathology datasets (DSB, MoNuSeg and TNBC) exhibit the outstanding segmentation performance of the proposed method (with dice scores 90.82%, 85.74%, and 78.57%, respectively). Implementation of the proposed architecture is available at https://github.com/tamjidimtiaz/BAWGNet.

SAC-Net: Learning with weak and noisy labels in histopathology image segmentation

Deep convolutional neural networks have been highly effective in segmentation tasks. However, segmentation becomes more difficult when training images include many complex instances to segment, such as the task of nuclei segmentation in histopathology images. Weakly supervised learning can reduce the need for large-scale, high-quality ground truth annotations by involving non-expert annotators or algorithms to generate supervision information for segmentation. However, there is still a significant performance gap between weakly supervised learning and fully supervised learning approaches. In this work, we propose a weakly-supervised nuclei segmentation method in a two-stage training manner that only requires annotation of the nuclear centroids. First, we generate boundary and superpixel-based masks as pseudo ground truth labels to train our SAC-Net, which is a segmentation network enhanced by a constraint network and an attention network to effectively address the problems caused by noisy labels. Then, we refine the pseudo labels at the pixel level based on Confident Learning to train the network again. Our method shows highly competitive performance of cell nuclei segmentation in histopathology images on three public datasets. Code will be available at: https://github.com/RuoyuGuo/MaskGA_Net.

MIU-Net: MIX-Attention and Inception U-Net for Histopathology Image Nuclei Segmentation

In the medical field, hematoxylin and eosin (H&E)-stained histopathology images of cell nuclei analysis represent an important measure for cancer diagnosis. The most valuable aspect of the nuclei analysis is the segmentation of the different nuclei morphologies of different organs and subsequent diagnosis of the type and severity of the disease based on pathology. In recent years, deep learning techniques have been widely used in digital histopathology analysis. Automated nuclear segmentation technology enables the rapid and efficient segmentation of tens of thousands of complex and variable nuclei in histopathology images. However, a challenging problem during nuclei segmentation is the blocking of cell nuclei, overlapping, and background complexity of the tissue fraction. To address this challenge, we present MIU-net, an efficient deep learning network structure for the nuclei segmentation of histopathology images. Our proposed structure includes two blocks with modified inception module and attention module. The advantage of the modified inception module is to balance the computation and network performance of the deeper layers of the network, combined with the convolutional layer using different sizes of kernels to learn effective features in a fast and efficient manner to complete kernel segmentation. The attention module allows us to extract small and fine irregular boundary features from the images, which can better segment cancer cells that appear disorganized and fragmented. We test our methodology on public kumar datasets and achieve the highest AUC score of 0.92. The experimental results show that the proposed method achieves better performance than other state-of-the-art methods.

FRE-Net: Full-region enhanced network for nuclei segmentation in histopathology images

Accurate nuclei segmentation is a critical step for physicians to achieve essential information about a patient’s disease through digital pathology images, enabling an effective diagnosis and evaluation of subsequent treatments. Since pathology images contain many nuclei, manual segmentation is time-consuming and error-prone. Therefore, developing a precise and automatic method for nuclei segmentation is urgent. This paper proposes a novel multi-task segmentation network that incorporates background and contour segmentation into the nuclei segmentation method and produces more accurate segmentation results. The convolution and attention modules are merged with the model to increase its global focus and enhance good segmentation results indirectly. We propose a reverse feature enhance module for contour extraction that facilitates feature integration between auxiliary tasks. The multi-feature fusion module is embedded in the final decoding branch to use different levels of features from auxiliary segmentation branches with varying concerns. We evaluate the proposed method on four challenging nuclei segmentation datasets. The proposed method achieves excellent performance on all four datasets. We found that the Dice coefficient reached 0.8563±0.0323, 0.8183±0.0383, 0.9222±0.0216, and 0.9220±0.0602 on the TNBC, MoNuSeg, KMC, and Glas. Our method produces better boundary accuracy and less sticking than other end-to-end segmentation methods. The results show that our method can perform better than other proposed state-of-the-art methods.

Wide kernels and their DCT compression in convolutional networks for nuclei segmentation

The locality and spatial field of view of image operators have played a major role in image analysis, from hand-crafted to deep learning methods. In Convolutional Neural Networks (CNNs), the field of view is traditionally set to very small values (e.g. 3 × 3 pixels) for individual kernels and grown throughout the network by cascading layers. Automatically learning or adapting the best spatial support of the kernels can be done by using large kernels. Due to the computation requirements of standard CNN architectures, this has been little investigated in the literature. However, if large receptive fields are needed to capture wider contextual information on a given task, it could be learned from the data. Obtaining an optimal receptive field with few layers is very relevant in applications with a limited amount of annotated training data, e.g. in medical imaging.We show that CNNs (2D U-Nets) with large kernels outperform similar models with standard small kernels on the task of nuclei segmentation in histopathology images. We observe that the large kernels mostly capture low-frequency information, which motivates the need for large kernels and their efficient compression via the Discrete Cosine Transform (DCT). Following this idea, we develop a U-Net model with wide and compressed DCT kernels that leads to similar performance and trends to the standard U-Net, with reduced complexity. Visualizations of the kernels in the spatial and frequency domains, as well as the effective receptive fields, provide insights into the models’ behaviors and the learned features.

High-resolution deep transferred ASPPU-Net for nuclei segmentation of histopathology images.

Increasing cancer disease incidence worldwide has become a major public health issue. Manual histopathological analysis is a common diagnostic method for cancer detection. Due to the complex structure and wide variability in the texture of histopathology images, it has been challenging for pathologists to diagnose manually those images. Automatic segmentation of histopathology images to diagnose cancer disease is a continuous exploration field in recent times. Segmentation and analysis for diagnosis of histopathology images by using an efficient deep learning algorithm are the purpose of the proposed method. To improve the segmentation performance, we proposed a deep learning framework that consists of a high-resolution encoder path, an atrous spatial pyramid pooling bottleneck module, and a powerful decoder. Compared to the benchmark segmentation models having a deep and thin path, our network is wide and deep that effectively leverages the strength of residual learning as well as encoder-decoder architecture. We performed careful experimentation and analysis on three publically available datasets namely kidney dataset, Triple Negative Breast Cancer (TNBC) dataset, and MoNuSeg histopathology image dataset. We have used the two most preferred performance metrics called F1 score and aggregated Jaccard index (AJI) to evaluate the performance of the proposed model. The measured values of F1 score and AJI score are (0.9684, 0.9394), (0.8419, 0.7282), and (0.8344, 0.7169) on the kidney dataset, TNBC histopathology dataset, and MoNuSeg dataset, respectively. Our proposed method yields better results as compared to benchmark segmentation methods on three histopathology datasets. Visual segmentation results justify the high value of the F1 score and AJI scores which indicated that it is a very good prediction by our proposed model.

Efficient deep learning architecture with dimension-wise pyramid pooling for nuclei segmentation of histopathology images

Image segmentation remains to be one of the most vital tasks in the area of computer vision and more so in the case of medical image processing. Image segmentation quality is the main metric that is often considered with memory and computation efficiency overlooked, limiting the use of power hungry models for practical use. In this paper, we propose a novel framework (Kidney-SegNet) that combines the effectiveness of an attention based encoder-decoder architecture with atrous spatial pyramid pooling with highly efficient dimension-wise convolutions. The segmentation results of the proposed Kidney-SegNet architecture have been shown to outperform existing state-of-the-art deep learning methods by evaluating them on two publicly available kidney and TNBC breast H&E stained histopathology image datasets. Further, our simulation experiments also reveal that the computational complexity and memory requirement of our proposed architecture is very efficient compared to existing deep learning state-of-the-art methods for the task of nuclei segmentation of H&E stained histopathology images. The source code of our implementation will be available at https://github.com/Aaatresh/Kidney-SegNet.

Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images.

Nuclei mymargin segmentation is a fundamental task for various computational pathology applications including nuclei morphology analysis, cell type classification, and cancer grading. Deep learning has emerged as a powerful approach to segmenting nuclei but the accuracy of convolutional neural networks (CNNs) depends on the volume and the quality of labeled histopathology data for training. In particular, conventional CNN-based approaches lack structured prediction capabilities, which are required to distinguish overlapping and clumped nuclei. Here, we present an approach to nuclei segmentation that overcomes these challenges by utilizing a conditional generative adversarial network (cGAN) trained with synthetic and real data. We generate a large dataset of H&E training images with perfect nuclei segmentation labels using an unpaired GAN framework. This synthetic data along with real histopathology data from six different organs are used to train a conditional GAN with spectral normalization and gradient penalty for nuclei segmentation. This adversarial regression framework enforces higher-order spacial-consistency when compared to conventional CNN models. We demonstrate that this nuclei segmentation approach generalizes across different organs, sites, patients and disease states, and outperforms conventional approaches, especially in isolating individual and overlapping nuclei.

Content-based image retrieval algorithm for nuclei segmentation in histopathology images

In today’s world, the medical diagnostic system shows a high reliance on medical imagery and digital nosology. To facilitate the fast and precise screening of samples, technology is leading towards the computer-aided disease diagnosis and grading. Image segmentation possesses high worth in the computer-aided disease diagnosis and grading systems to extract the region of interest. This paper presents a content-based image retrieval algorithm for histopathology image segmentation for identification and extraction of nuclei. The proposed technique furnishes nuclei segmentation in three cascaded stages; pre-processing, nuclei points and region refining, and composite nuclei segmentation. The performance of nuclei segmentation is investigated on six hematoxylins and eosin (H&E) stained histopathology images datasets. Simulation outcomes of the segmentation schemes confirm the superiority of the proposed method for nuclei segmentation in histopathology images in qualitative and quantitative analysis.

BENDING LOSS REGULARIZED NETWORK FOR NUCLEI SEGMENTATION IN HISTOPATHOLOGY IMAGES.

Separating overlapped nuclei is a major challenge in histopathology image analysis. Recently published approaches have achieved promising overall performance on public datasets; however, their performance in segmenting overlapped nuclei are limited. To address the issue, we propose the bending loss regularized network for nuclei segmentation. The proposed bending loss defines high penalties to contour points with large curvatures, and applies small penalties to contour points with small curvature. Minimizing the bending loss can avoid generating contours that encompass multiple nuclei. The proposed approach is validated on the MoNuSeg dataset using five quantitative metrics. It outperforms six state-of-the-art approaches on the following metrics: Aggregate Jaccard Index, Dice, Recognition Quality, and Panoptic Quality.

An automatic nuclei segmentation method based on deep convolutional neural networks for histopathology images

BackgroundSince nuclei segmentation in histopathology images can provide key information for identifying the presence or stage of a disease, the images need to be assessed carefully. However, color variation in histopathology images, and various structures of nuclei are two major obstacles in accurately segmenting and analyzing histopathology images. Several machine learning methods heavily rely on hand-crafted features which have limitations due to manual thresholding.ResultsTo obtain robust results, deep learning based methods have been proposed. Deep convolutional neural networks (DCNN) used for automatically extracting features from raw image data have been proven to achieve great performance. Inspired by such achievements, we propose a nuclei segmentation method based on DCNNs. To normalize the color of histopathology images, we use a deep convolutional Gaussian mixture color normalization model which is able to cluster pixels while considering the structures of nuclei. To segment nuclei, we use Mask R-CNN which achieves state-of-the-art object segmentation performance in the field of computer vision. In addition, we perform multiple inference as a post-processing step to boost segmentation performance. We evaluate our segmentation method on two different datasets. The first dataset consists of histopathology images of various organ while the other consists histopathology images of the same organ. Performance of our segmentation method is measured in various experimental setups at the object-level and the pixel-level. In addition, we compare the performance of our method with that of existing state-of-the-art methods. The experimental results show that our nuclei segmentation method outperforms the existing methods.ConclusionsWe propose a nuclei segmentation method based on DCNNs for histopathology images. The proposed method which uses Mask R-CNN with color normalization and multiple inference post-processing provides robust nuclei segmentation results. Our method also can facilitate downstream nuclei morphological analyses as it provides high-quality features extracted from histopathology images.

Integrating deep convolutional neural networks with marker-controlled watershed for overlapping nuclei segmentation in histopathology images

Nuclei segmentation in histopathology images plays a crucial role in the morphological quantitative analysis of tissue structure and has become a hot research topic. Though numerous efforts have been tried in this research area, the overlapping and touching nuclei segmentation remains a challenging problem. In this paper, we present a novel and effective instance segmentation method for tackling this challenge by integrating Deep Convolutional Neural Networks with Marker-controlled Watershed. Firstly, we design a novel network architecture with multiple segmentation tasks, called Deep Interval-Marker-Aware Network, for learning the foreground, marker, and interval of nuclei, simultaneously. Then the learned interval between overlapping nuclei is used to refine the foreground result of nuclei by using the logical operators. Finally, the learned marker result and the nuclei segmentation result refined by interval are transmitted into the Marker-controlled Watershed for splitting the touching nuclei. The experiments on the standard public datasets demonstrate that our method achieves a substantial improvement compared with state-of-the-art methods. Source codes are available at https://github.com/appiek/Nuclei_Segmentation_Experiments_Demo.

A deep learning algorithm for one-step contour aware nuclei segmentation of histopathology images

This paper addresses the task of nuclei segmentation in high-resolution histopathology images. We propose an automatic end-to-end deep neural network algorithm for segmentation of individual nuclei. A nucleus-boundary model is introduced to predict nuclei and their boundaries simultaneously using a fully convolutional neural network. Given a color-normalized image, the model directly outputs an estimated nuclei map and a boundary map. A simple, fast, and parameter-free post-processing procedure is performed on the estimated nuclei map to produce the final segmented nuclei. An overlapped patch extraction and assembling method is also designed for seamless prediction of nuclei in large whole-slide images. We also show the effectiveness of data augmentation methods for nuclei segmentation task. Our experiments showed our method outperforms prior state-of-the-art methods. Moreover, it is efficient that one 1000×1000 image can be segmented in less than 5 s. This makes it possible to precisely segment the whole-slide image in acceptable time. The source code is available at https://github.com/easycui/nuclei_segmentation . Graphical Abstract The neural network for nuclei segmentation.