Accurate Nuclei Segmentation Research Articles

Cell nuclei image segmentation using U-Net and DeepLabV3+ with transfer learning and regularization

Semantic nuclei segmentation is a challenging area of computer vision. Accurate nuclei segmentation can help medics in diagnosing many diseases. Automatic nuclei segmentation can help medics in diagnosing many diseases such as cancer by providing automatic tissue analysis. Deep learning algorithms allow automatic feature extraction from medical images, however, hematoxylin and eosin (H&E) stained images are challenging due to variability in staining and textures. Using pre-trained models in deep learning speeds up development and improves their performance. This paper compares Deeplabv3+ and U-Net deep learning methods with the pre-trained models ResNet-50 and EfficientNetB4 embedded in their architecture. In addition, different regularization and dropout parameters are applied to prevent overtraining. The experiment was conducted on the PanNuke dataset consisting of nearly 8,000 histological images and annotated nuclei. As a result, the ResNet50-based DeepLabV3+ model with L2 regularization of 0.02 and dropout of 0.7 showed efficiency with dice coefficient (DCS) of 0.8356, intersection over union (IOU) of 0.7280, and loss of 0.3212 on the test set.

Deep Learning-driven Automatic Nuclei Segmentation of Label-free Live Cell Chromatin-sensitive Partial Wave Spectroscopic Microscopy Imaging.

Chromatin-sensitive Partial Wave Spectroscopic (csPWS) microscopy offers a non-invasive glimpse into the mass density distribution of cellular structures at the nanoscale, leveraging the spectroscopic information. Such capability allows us to analyze the chromatin structure and organization and the global transcriptional state of the cell nuclei for the study of its role in carcinogenesis. Accurate segmentation of the nuclei in csPWS microscopy images is an essential step in isolating them for further analysis. However, manual segmentation is error-prone, biased, time-consuming, and laborious, resulting in disrupted nuclear boundaries with partial or over-segmentation. Here, we present an innovative deep-learning-driven approach to automate the accurate nuclei segmentation of label-free live cell csPWS microscopy imaging data. Our approach, csPWS-seg, harnesses the Convolutional Neural Networks-based U-Net model with an attention mechanism to automate the accurate cell nuclei segmentation of csPWS microscopy images. We leveraged the structural, physical, and biological differences between the cytoplasm, nucleus, and nuclear periphery to construct three distinct csPWS feature images for nucleus segmentation. Using these images of HCT116 cells, csPWS-seg achieved superior performance with a median Intersection over Union (IoU) of 0.80 and a Dice Similarity Coefficient (DSC) score of 0.88. The csPWS-seg overcame the segmentation performance over the baseline U-Net model and another attention-based model, SE-U-Net, marking a significant improvement in segmentation accuracy. Further, we analyzed the performance of our proposed model with four loss functions: binary cross-entropy loss, focal loss, dice loss, and Jaccard loss. The csPWS-seg with focal loss provided the best results compared to other loss functions. The automatic and accurate nuclei segmentation offered by the csPWS-seg not only automates, accelerates, and streamlines csPWS data analysis but also enhances the reliability of subsequent chromatin analysis research, paving the way for more accurate diagnostics, treatment, and understanding of cellular mechanisms for carcinogenesis.

Nuclei segmentation using attention aware and adversarial networks

Accurate segmentation of nuclei plays a critical role in pathology since assessments and diagnoses are mainly based on the recognition, measurement, and counting of nuclei. However, in digital pathology, automated nucleus segmentation is a challenging issue because of various factors such as color inconsistency in stained images, unclear nucleus boundaries, low contrast between background and nuclei, different shapes and sizes of nuclei, and intensity inhomogeneity not only inside nuclei but also across them and the background. In this work, an efficient method has been developed for nuclei segmentation. Its efficiency has been achieved through tissue and chemical invariant normalization, feature extraction with dense convolution layers, merging of local and global features, mask generation, and adversarial learning. For fair comparisons, state-of-the-art nuclei segmentations have been employed to the same data sets, and their performances have been evaluated using the same metrics. This paper’s main contributions are threefold: (i) Introducing a novel technique for nuclei segmentation using an adversarial network and a hybrid attention-aware network. (ii) Presenting the effective merging of global and local features to enhance pattern recognition, and the utilization of efficient hybrid attention blocks for extracting desired global information and improving relationships between feature regions at different locations. (iii) Presenting experimental results showing that the proposed technique accomplishes nuclei detection and extraction with higher accuracy (a minimum improvement of 3.7%) than other recent methods. Also, each stage provides considerable contributions to the segmentation performance. Particularly, the hybrid attention-aware network has improved the performance by 4.2% according to the dice coefficient.

NuSEA: Nuclei Segmentation With Ellipse Annotations.

Nuclei segmentation is a crucial pre-task for pathological microenvironment quantification. However, the acquisition of manually precise nuclei annotations for improving the performance of deep learning models is time-consuming and expensive. In this paper, an efficient nuclear annotation tool called NuSEA is proposed to achieve accurate nucleus segmentation, where a simple but effective ellipse annotation is applied. Specifically, the core network U-Light of NuSEA is lightweight with only 0.86 M parameters, which is suitable for real-time nuclei segmentation. In addition, an Elliptical Field Loss and a Texture Loss are proposed to enhance the edge segmentation and constrain the smoothness simultaneously. Extensive experiments on three public datasets (MoNuSeg, CPM-17, and CoNSeP) demonstrate that NuSEA is superior to the state-of-the-art (SOTA) methods and better than existing algorithms based on point, rectangle, and text annotations. With the assistance of NuSEA, a new dataset called NuSEA-dataset v1.0, encompassing 118,857 annotated nuclei from the whole-slide images of 12 organs is released. NuSEA provides a rapid and effective annotation tool for nuclei in histopathological images, benefiting future explorations in deep learning algorithms.

An Automated Framework for Histopathological Nucleus Segmentation With Deep Attention Integrated Networks.

Clinical management and accurate disease diagnosis are evolving from qualitative stage to the quantitative stage, particularly at the cellular level. However, the manual process of histopathological analysis is lab-intensive and time-consuming. Meanwhile, the accuracy is limited by the experience of the pathologist. Therefore, deep learning-empowered computer-aided diagnosis (CAD) is emerging as an important topic in digital pathology to streamline the standard process of automatic tissue analysis. Automated accurate nucleus segmentation can not only help pathologists make more accurate diagnosis, save time and labor, but also achieve consistent and efficient diagnosis results. However, nucleus segmentation is susceptible to staining variation, uneven nucleus intensity, background noises, and nucleus tissue differences in biopsy specimens. To solve these problems, we propose Deep Attention Integrated Networks (DAINets), which mainly built on self-attention based spatial attention module and channel attention module. In addition, we also introduce a feature fusion branch to fuse high-level representations with low-level features for multi-scale perception, and employ the mark-based watershed algorithm to refine the predicted segmentation maps. Furthermore, in the testing phase, we design Individual Color Normalization (ICN) to settle the dyeing variation problem in specimens. Quantitative evaluations on the multi-organ nucleus dataset indicate the priority of our automated nucleus segmentation framework.

A dual-path instance segmentation network based on nuclei contour in histology image

Accurate segmentation of nuclei in histology images is essential for digital pathology. However, previous work should have considered integrating nuclei contour information into network learning. The main problems are (1) nuclei contour information needs to be fully considered, resulting in inaccurate spatial location of nuclei. (2) Small nuclei in dense regions or irregularly shaped nuclei cannot be segmented. (3) Mainstream networks consider more long-distance semantic information and larger receptive fields and do not consider the fusion of feature maps with different semantics. To solve the above problems, we have proposed a contour-based dual-path instance segmentation network. Firstly we reconstructed the nuclei contour information using algorithms and morphological manipulations. Provide input for subsequent contouring networks. Then we designed a dual-path network. It can extract nuclei features and contour features independently in the encoding stage and fuse the feature maps at different scales in the decoding stage. In the decoding stage, we use the attention module with a newly designed fusion mechanism, which allows us to fuse different semantics of feature maps for simple and efficient fusion. Finally, using the watershed algorithm, we use the network segmentation results to get the instance segmentation results of the nuclei. We used four publicly available datasets, including Kumar, CPM-15, CPM-17, and TNBC, which contain the nuclei of many organs and different forms of nuclei in the human body. Compared with the mainstream methods, we obtained the best AJI metrics, which are 0.656,0.717, and 0.602. In conclocess can effectively improve the performance of the nuclei instance segmentation network. It can accurately locate the scattered nuclei in space with small segmented nuclei in dense regions.

NuSegDA: Domain adaptation for nuclei segmentation.

The accurate segmentation of nuclei is crucial for cancer diagnosis and further clinical treatments. To successfully train a nuclei segmentation network in a fully-supervised manner for a particular type of organ or cancer, we need the dataset with ground-truth annotations. However, such well-annotated nuclei segmentation datasets are highly rare, and manually labeling an unannotated dataset is an expensive, time-consuming, and tedious process. Consequently, we require to discover a way for training the nuclei segmentation network with unlabeled dataset. In this paper, we propose a model named NuSegUDA for nuclei segmentation on the unlabeled dataset (target domain). It is achieved by applying Unsupervised Domain Adaptation (UDA) technique with the help of another labeled dataset (source domain) that may come from different type of organ, cancer, or source. We apply UDA technique at both of feature space and output space. We additionally utilize a reconstruction network and incorporate adversarial learning into it so that the source-domain images can be accurately translated to the target-domain for further training of the segmentation network. We validate our proposed NuSegUDA on two public nuclei segmentation datasets, and obtain significant improvement as compared with the baseline methods. Extensive experiments also verify the contribution of newly proposed image reconstruction adversarial loss, and target-translated source supervised loss to the performance boost of NuSegUDA. Finally, considering the scenario when we have a small number of annotations available from the target domain, we extend our work and propose NuSegSSDA, a Semi-Supervised Domain Adaptation (SSDA) based approach.

FRE-Net: Full-region enhanced network for nuclei segmentation in histopathology images

Accurate nuclei segmentation is a critical step for physicians to achieve essential information about a patient’s disease through digital pathology images, enabling an effective diagnosis and evaluation of subsequent treatments. Since pathology images contain many nuclei, manual segmentation is time-consuming and error-prone. Therefore, developing a precise and automatic method for nuclei segmentation is urgent. This paper proposes a novel multi-task segmentation network that incorporates background and contour segmentation into the nuclei segmentation method and produces more accurate segmentation results. The convolution and attention modules are merged with the model to increase its global focus and enhance good segmentation results indirectly. We propose a reverse feature enhance module for contour extraction that facilitates feature integration between auxiliary tasks. The multi-feature fusion module is embedded in the final decoding branch to use different levels of features from auxiliary segmentation branches with varying concerns. We evaluate the proposed method on four challenging nuclei segmentation datasets. The proposed method achieves excellent performance on all four datasets. We found that the Dice coefficient reached 0.8563±0.0323, 0.8183±0.0383, 0.9222±0.0216, and 0.9220±0.0602 on the TNBC, MoNuSeg, KMC, and Glas. Our method produces better boundary accuracy and less sticking than other end-to-end segmentation methods. The results show that our method can perform better than other proposed state-of-the-art methods.

AlexSegNet: an accurate nuclei segmentation deep learning model in microscopic images for diagnosis of cancer

AlexSegNet: an accurate nuclei segmentation deep learning model in microscopic images for diagnosis of cancer

A Cascaded Deep Learning Framework for Segmentation of Nuclei in Digital Histology Images.

Accurate segmentation of nuclei is an essential step in analysis of digital histology images for diagnostic and prognostic applications. Despite recent advances in automated frameworks for nuclei segmentation, this task is still challenging. Specifically, detecting small nuclei in large-scale histology images and delineating the border of touching nuclei accurately is a complicated task even for advanced deep neural networks. In this study, a cascaded deep learning framework is proposed to segment nuclei accurately in digitized microscopy images of histology slides. A U-Net based model with customized pixel-wised weighted loss function is adapted in the proposed framework, followed by a U-Net based model with VGG16 backbone and a soft Dice loss function. The model was pretrained on the Post-NAT-BRCA public dataset before training and independent evaluation on the MoNuSeg dataset. The cascaded model could outperform the other state-of-the-art models with an AJI of 0.72 and a F1-score of 0.83 on the MoNuSeg test set.

MSAL-Net: improve accurate segmentation of nuclei in histopathology images by multiscale attention learning network

BackgroundThe digital pathology images obtain the essential information about the patient’s disease, and the automated nuclei segmentation results can help doctors make better decisions about diagnosing the disease. With the speedy advancement of convolutional neural networks in image processing, deep learning has been shown to play a significant role in the various analysis of medical images, such as nuclei segmentation, mitosis detection and segmentation etc. Recently, several U-net based methods have been developed to solve the automated nuclei segmentation problems. However, these methods fail to deal with the weak features representation from the initial layers and introduce the noise into the decoder path. In this paper, we propose a multiscale attention learning network (MSAL-Net), where the dense dilated convolutions block captures more comprehensive nuclei context information, and a newly modified decoder part is introduced, which integrates with efficient channel attention and boundary refinement modules to effectively learn spatial information for better prediction and further refine the nuclei cell of boundaries.ResultsBoth qualitative and quantitative results are obtained on the publicly available MoNuseg dataset. Extensive experiment results verify that our proposed method significantly outperforms state-of-the-art methods as well as the vanilla Unet method in the segmentation task. Furthermore, we visually demonstrate the effect of our modified decoder part.ConclusionThe MSAL-Net shows superiority with a novel decoder to segment the touching and blurred background nuclei cells obtained from histopathology images with better performance for accurate decoding.

An Integrative Segmentation Framework for Cell Nucleus of Fluorescence Microscopy.

Nucleus segmentation of fluorescence microscopy is a critical step in quantifying measurements in cell biology. Automatic and accurate nucleus segmentation has powerful applications in analyzing intrinsic characterization in nucleus morphology. However, existing methods have limited capacity to perform accurate segmentation in challenging samples, such as noisy images and clumped nuclei. In this paper, inspired by the idea of cascaded U-Net (or W-Net) and its remarkable performance improvement in medical image segmentation, we proposed a novel framework called Attention-enhanced Simplified W-Net (ASW-Net), in which a cascade-like structure with between-net connections was used. Results showed that this lightweight model could reach remarkable segmentation performance in the BBBC039 testing set (aggregated Jaccard index, 0.90). In addition, our proposed framework performed better than the state-of-the-art methods in terms of segmentation performance. Moreover, we further explored the effectiveness of our designed network by visualizing the deep features from the network. Notably, our proposed framework is open source.

MDC-net: A new convolutional neural network for nucleus segmentation in histopathology images with distance maps and contour information

Accurate segmentation of nuclei in digital pathology images can assist doctors in diagnosing diseases and evaluating subsequent treatments. Manual segmentation of nuclei from pathology images is time-consuming because of the large number of nuclei and is also error-prone. Therefore, accurate and automatic nucleus segmentation methods are required. Owing to the large variations in the characterization of nuclei, it is difficult to accurately segment nuclei using traditional methods. In this study, we propose a new method for nucleus segmentation. The proposed method uses a deep fully convolutional neural network to perform end-to-end segmentation on pathological tissue slices. Multiple short residual connections were used to fuse feature maps from different scales to better utilize the context information. Dilated convolutions with different dilation ratios were used to increase the receptive fields. In addition, we incorporated the distance map and contour information into the segmentation method to segment touching nuclei, which is difficult via traditional segmentation methods. Finally, post-processing was used to improve the segmentation results. The results demonstrate that our segmentation method can obtain comparable or better performance than other state-of-the-art methods on the public nuclei histopathology datasets.

SEENS: Nuclei segmentation in Pap smear images with selective edge enhancement

Accurate nuclei segmentation, as an indispensable basis and core link for multi-cell cervical image analysis, plays an important role in automatic pre-cancer detection. However, poor image quality due to the uneven staining, complex backgrounds and overlapped cell clusters poses a great challenge in nuclei segmentation. In this paper, we propose a new Selective-Edge-Enhancement-based Nuclei Segmentation method (SEENS). In the proposed method, selective search is integrated with mathematical operators to segment whole slide cervical images into small regions of interest (ROI) while automatically avoiding repeated segmentation as well as eliminating non-nuclei regions. In addition, an edge enhancement method based on the canny operator and mathematical morphology is presented to extract edge information as a weight to enhance the nucleus edge selectively. As a result, the enhanced ROI is then segmented by the Chan–Vese model with a higher accuracy. We evaluate our method with 18 whole slide images for a total of 395 cell nuclei. Experimental results demonstrate that SEENS achieves higher accuracy in cervical nuclei segmentation. Notably our method performs particularly better in low-contrast scenarios than baselines.

Three segmentation techniques to predict the dysplasia in cervical cells in the presence of debris

Cervical Cancer is one of the most pandemic causes of cancer related death in females. Pap smear test is one of the most commonly used screening test for the cervical cancer. Existing algorithms focus on the segmentation of nucleus and cytoplasm either using single-cell images or multiple cells images. Images captured from the Pap smear slides are called smear images. Smear image contains cervical cells along with debris, debris are inflammatory cells, red blood cells, dye. Debris significantly influence the outcome of image segmentation. An accurate nuclei segmentation method can improve the success rate of cervical cancer screening. Therefore, this paper reveals about three segmentation techniques which are used for automated segmentation of cervical cell nuclei in the presence of debris. Three segmentation techniques namely, Automated Seed Region Growing, Extended Edge Based Detection and Modified Moving k-means techniques are proposed to extract the cervical cell nuclei. These techniques are extracting the area of nuclei from smear images using the morphological property of nucleus. Some debris have area that corresponds with the area of nucleus of normal cells, it may interfere with outcome and may give false positive results. The empirical area threshold value demonstrate the superior performance of all proposed methods. The qualitative and quantitative analysis also done on proposed techniques. Experimental analysis shows that Modified Moving k-means give favorable result in dysplasia detection in the presence of debris. A new dataset PapsmearJP is collected during the study with the help of a pathologist for the validation of the work.

An Accurate Nuclei Segmentation Algorithm in Pathological Image Based on Deep Semantic Network

Cell (nuclei) segmentation is the basic and key step of pathological image analysis. However, robust and accurate cell (nuclei) segmentation is a difficult problem due to the enormous variability of staining, cell sizes, morphologies and cell adhesion or overlapping. In this paper, we extend U-Net with atrous depthwise separable convolution (AS-UNet) for cell (nuclei) segmentation. AS-UNet consists of three parts: encoder module, decoder module and atrous convolution module. The encoder module obtains the high-level semantic information of the cell image layer by layer, while the decoder module gradually recovers the spatial information. The atrous convolution module is composed of cascade and parallel atrous convolution operations. It can extract and combine multi-scale features so that the model can get strong perception ability for both small and large cells. At the same time, the atrous convolution can significantly increase the receptive field of the network model without hurting the segmentation performance or increasing the computational cost. During the training period, Log-Dice loss and Focal loss are combined, while Adam optimization method is employed to optimize the network. In order to increase the penalty for the smaller prediction result of the Dice coefficient, which is carried out by logarithmic operation and the negative value is taken as the Log-Dice loss. The above optimization is beneficial for the convergence speed. Additionally, some data augmentation techniques are applied to increase online data, which contribute to improving the robustness of the model. Compared with several state-of-the-art semantic segmentation algorithms, our method achieves the promising performance on two latest released pathological image datasets.

Accurate segmentation of nuclei in pathological images via sparse reconstruction and deep convolutional networks

Automated cell segmentation is a critical step for computer assisted pathology related image analysis, such as automated grading of breast cancer tissue specimens. However, automated cell segmentation is complicated by (1) complexity of the data (possibly touching cells, stains, background clutters, and image artifacts) and (2) the variability in size, shape, appearance, and texture of the individual nuclei. Recently, there has been a growing interest in the application of “Deep Learning” strategies for the analysis of natural and pathological images. Histopathology, given its diversity and complexity, represents an excellent use case for application of deep learning strategies. In this paper, we put forward an automated nuclei segmentation method that works with hematoxylin and eosin (H&E) stained breast cancer histopathology images, which represent regions of whole digital slides. The procedure can be divided into three main stages. Initially, the sparse reconstruction method is employed to roughly remove the background and accentuate the nuclei of pathological images. Then, deep convolutional networks (DCN), cascaded by multi-layer convolution networks, are trained using gradient descent techniques to efficiently segment the cell nuclei from the background. In this stage, input patches and its corresponding labels are randomly sampled from the pathological images and fed to the training networks. The size of the sampled patches can be flexible, and the proposed method is robust when the times of sampling and the number of feature maps vary in a wide range. Finally, morphological operations and some prior knowledge are introduced to improve the segmentation performance and reduce the errors. Our method achieves about 92.45% pixel-wise segmentation accuracy and the F1-measure is 0.8393. This result leads to a promising segmentation performance, equivalent and sometimes surpassing recently published leading alternative segmentation methods with the same benchmark datasets.

Automatic nuclei segmentation and spatial FISH analysis for cancer detection.

Spatial analysis of gene localization using fluorescent in-situ hybridization (FISH) labeling is potentially a new method for early cancer detection. Current methodology relies heavily upon accurate segmentation of cell nuclei and FISH signals in tissue sections. While automatic FISH signal detection is a relatively simpler task, accurate nuclei segmentation is still a manual process which is fairly time consuming and subjective. Hence to use the methodology as a clinical application, it is necessary to automate all the steps involved in the process of spatial FISH signal analysis using fast, robust and accurate image processing techniques. In this work, we describe an intelligent framework for analyzing the FISH signals by coupling hybrid nuclei segmentation algorithm with pattern recognition algorithms to automatically identify well segmented nuclei. Automatic spatial statistical analysis of the FISH spots was carried out on the output from the image processing and pattern recognition unit. Results are encouraging and show that the method could evolve into a full fledged clinical application for cancer detection.