Nuclei In Histopathology Images Research Articles

DB-FCN: An end-to-end dual-branch fully convolutional nucleus detection model

Detecting cells or nuclei in histopathological image analysis is a crucial prerequisite for subsequent cancer diagnosis. By precisely locating cell nuclei, pathologists can quantitatively analyze the morphology of each cell nucleus. This enables accurate cancer grading, allowing for the implementation of tailored treatment plans based on distinct cancer stages. Manual nucleus detection methods are labor-intensive, making the development of automatic cell nucleus detection algorithms highly necessary. However, due to the small size of the nucleus and increased adhesions between cells, tissue staining may be uneven, leading to a higher occurrence of false positives in the results of the current automatic nucleus detection algorithm. This paper introduces an end-to-end dual-branch-based fully convolutional neural network (DB-FCN) that effectively addresses the aforementioned challenges, thereby enhancing the accuracy of automatic nucleus detection. This algorithm introduces for the first time the use of two detection branches, namely the coarse detection branch and the fine detection branch, to accomplish the detection task. The role of the coarse detection branch is to identify all cell regions in the pathological image as comprehensively as possible and then transmit the detection results as prior information to the fine detection branch. The fine detection branch is necessary solely to conduct more precise detection based on the coarse detection results. Given that the coarse detection branch has already eliminated interference from many background regions, the fine detection branch can focus on detecting the nucleus region, thereby significantly enhancing the efficiency and accuracy of model detection. The detection model proposed in this paper was evaluated on three classic datasets and compared with many existing detection algorithms. Compared with other algorithms, the detection algorithm proposed in this paper has made significant progress in detecting cell nuclei in histopathological images.

Impact of imperfect annotations on CNN training and performance for instance segmentation and classification in digital pathology

Segmentation and classification of large numbers of instances, such as cell nuclei, are crucial tasks in digital pathology for accurate diagnosis. However, the availability of high-quality datasets for deep learning methods is often limited due to the complexity of the annotation process. In this work, we investigate the impact of noisy annotations on the training and performance of a state-of-the-art CNN model for the combined task of detecting, segmenting and classifying nuclei in histopathology images. In this context, we investigate the conditions for determining an appropriate number of training epochs to prevent overfitting to annotation noise during training. Our results indicate that the utilisation of a small, correctly annotated validation set is instrumental in avoiding overfitting and maintaining model performance to a large extent. Additionally, our findings underscore the beneficial role of pre-training.

Unsupervised Color-Based Nuclei Segmentation in Histopathology Images with Various Color Spaces and K Values Selection

The development of digital pathology offers a significant opportunity to evaluate and analyze the whole slides of disease tissue effectively. In particular, the segmentation of nuclei from histopathology images plays an important role in quantitatively measuring and evaluating the acquired diseased tissue. There are many automatic methods to segment cell nuclei in histopathology images. One widely used unsupervised segmentation approach is based on standard k-means or fuzzy c-means (FCM) to process the color histopathology images to segment cell nuclei. Compared with the supervised learning method, this approach can obtain segmented nuclei without annotated nuclei labels for training, which saves a lot of labeling and training time. The color space and [Formula: see text] value among this method plays a crucial role in determining the nuclei segmentation performance. However, few works have investigated various color spaces and [Formula: see text] value selection simultaneously in unsupervised color-based nuclei segmentation with [Formula: see text]-means or FCM algorithms. In this study, we will present color-based nuclei segmentation methods with standard [Formula: see text]-means and FCM algorithms for histopathology images. Several color spaces of Haematoxylin and Eosin (H&E) stained histopathology data and various [Formula: see text] values among [Formula: see text]-means and FCM are investigated correspondingly to explore the suitable selection for nuclei segmentation. A comprehensive nuclei dataset with 7 different organs is used to validate our proposed method. Related experimental results indicate that [Formula: see text] and the YCbCr color spaces with a [Formula: see text] of 4 are more reasonable for nuclei segmentation via [Formula: see text]-means, while the [Formula: see text] color space with [Formula: see text] of 4 is useful via FCM.

Real-time and accurate deep learning-based multi-organ nucleus segmentation in histology images

Automated nucleus segmentation is considered the gold standard for diagnosing some severe diseases. Accurate instance segmentation of nuclei is still very challenging because of the large number of clustered nuclei, and the different appearance of nuclei for different tissue types. In this paper, a neural network is proposed for fast and accurate instance segmentation of nuclei in histopathology images. The network is inspired by the Unet and residual nets. The main contribution of the proposed model is enhancing the classification accuracy of nuclear boundaries by moderately preserving the spatial features by relatively d the size of feature maps. Then, a proposed 2D convolution layer is used instead of the conventional 3D convolution layer, the core of CNN-based architectures, where the feature maps are first compacted before being convolved by 2D kernel filters. This significantly reduces the processing time and avoids the out of memory problem of the GPU. Also, more features are extracted when getting deeper into the network without degrading the spatial features dramatically. Hence, the number of layers, required to compensate the loss of spatial features, is reduced that also reduces the processing time. The proposed approach is applied to two multi-organ datasets and evaluated by the Aggregated Jaccard Index (AJI), F1-score and the number of frames per second. Also, the formula of AJI is modified to reflect the object- and pixel-level errors more accurately. The proposed model is compared to some state-of-the-art architectures, and it shows better performance in terms of the segmentation speed and accuracy.

NuSEA: Nuclei Segmentation With Ellipse Annotations.

Nuclei segmentation is a crucial pre-task for pathological microenvironment quantification. However, the acquisition of manually precise nuclei annotations for improving the performance of deep learning models is time-consuming and expensive. In this paper, an efficient nuclear annotation tool called NuSEA is proposed to achieve accurate nucleus segmentation, where a simple but effective ellipse annotation is applied. Specifically, the core network U-Light of NuSEA is lightweight with only 0.86 M parameters, which is suitable for real-time nuclei segmentation. In addition, an Elliptical Field Loss and a Texture Loss are proposed to enhance the edge segmentation and constrain the smoothness simultaneously. Extensive experiments on three public datasets (MoNuSeg, CPM-17, and CoNSeP) demonstrate that NuSEA is superior to the state-of-the-art (SOTA) methods and better than existing algorithms based on point, rectangle, and text annotations. With the assistance of NuSEA, a new dataset called NuSEA-dataset v1.0, encompassing 118,857 annotated nuclei from the whole-slide images of 12 organs is released. NuSEA provides a rapid and effective annotation tool for nuclei in histopathological images, benefiting future explorations in deep learning algorithms.

Mitotic Nuclei Segmentation and Classification Using Chaotic Butterfly Optimization Algorithm with Deep Learning on Histopathology Images.

Histopathological grading of the tumors provides insights about the patient's disease conditions, and it also helps in customizing the treatment plans. Mitotic nuclei classification involves the categorization and identification of nuclei in histopathological images based on whether they are undergoing the cell division (mitosis) process or not. This is an essential procedure in several research and medical contexts, especially in diagnosis and prognosis of cancer. Mitotic nuclei classification is a challenging task since the size of the nuclei is too small to observe, while the mitotic figures possess a different appearance as well. Automated calculation of mitotic nuclei is a stimulating one due to their great similarity to non-mitotic nuclei and their heteromorphic appearance. Both Computer Vision (CV) and Machine Learning (ML) approaches are used in the automated identification and the categorization of mitotic nuclei in histopathological images that endure the procedure of cell division (mitosis). With this background, the current research article introduces the mitotic nuclei segmentation and classification using the chaotic butterfly optimization algorithm with deep learning (MNSC-CBOADL) technique. The main objective of the MNSC-CBOADL technique is to perform automated segmentation and the classification of the mitotic nuclei. In the presented MNSC-CBOADL technique, the U-Net model is initially applied for the purpose of segmentation. Additionally, the MNSC-CBOADL technique applies the Xception model for feature vector generation. For the classification process, the MNSC-CBOADL technique employs the deep belief network (DBN) algorithm. In order to enhance the detection performance of the DBN approach, the CBOA is designed for the hyperparameter tuning model. The proposed MNSC-CBOADL system was validated through simulation using the benchmark database. The extensive results confirmed the superior performance of the proposed MNSC-CBOADL system in the classification of mitotic nuclei.

CA-Net: Context Aggregation Network for Nuclei Classification in Histopathology Image

Accurately classifying nuclei in histopathology images is essential for cancer diagnosis and prognosis. However, due to the touching nuclei, nucleus shape variation, background complexity, and image artifacts, end-to-end nucleus classification is still difficult and challenging. In this manuscript, we propose a context aggregation network (CA-Net) for nuclei classification by fusing global contextual information which is critical for classifying nuclei in histopathology images. Specifically, we propose a multi-level semantic supervision (MSS) module focusing on extracting multi-scale context information by varying three different kernel sizes, and dynamically aggregating the context information from high to low level. Furthermore, we employ the GPG and SAPF modules in encoder and decoder networks to exact and aggregate global context information. Finally, the proposed network is verified on a mainstream nuclei classification image datasets (PanNuke) and achieves an improved global accuracy of 0.816. Our proposed MSS module can be easily transferred into any UNet-liked architecture as a deep supervision mechanism.

MIU-Net: MIX-Attention and Inception U-Net for Histopathology Image Nuclei Segmentation

In the medical field, hematoxylin and eosin (H&E)-stained histopathology images of cell nuclei analysis represent an important measure for cancer diagnosis. The most valuable aspect of the nuclei analysis is the segmentation of the different nuclei morphologies of different organs and subsequent diagnosis of the type and severity of the disease based on pathology. In recent years, deep learning techniques have been widely used in digital histopathology analysis. Automated nuclear segmentation technology enables the rapid and efficient segmentation of tens of thousands of complex and variable nuclei in histopathology images. However, a challenging problem during nuclei segmentation is the blocking of cell nuclei, overlapping, and background complexity of the tissue fraction. To address this challenge, we present MIU-net, an efficient deep learning network structure for the nuclei segmentation of histopathology images. Our proposed structure includes two blocks with modified inception module and attention module. The advantage of the modified inception module is to balance the computation and network performance of the deeper layers of the network, combined with the convolutional layer using different sizes of kernels to learn effective features in a fast and efficient manner to complete kernel segmentation. The attention module allows us to extract small and fine irregular boundary features from the images, which can better segment cancer cells that appear disorganized and fragmented. We test our methodology on public kumar datasets and achieve the highest AUC score of 0.92. The experimental results show that the proposed method achieves better performance than other state-of-the-art methods.

Detecting and Classifying Nuclei Using Multi-Scale Fully Convolutional Network.

The detection and classification of nuclei play an important role in the histopathological analysis. It aims to find out the distribution of nuclei in the histopathology images for the next step of analysis and research. However, it is very challenging to detect and localize nuclei in histopathology images because the size of nuclei accounts for only a few pixels in images, making it difficult to be detected. Most automatic detection machine learning algorithms use patches, which are small pieces of images including a single cell, as training data, and then apply a sliding window strategy to detect nuclei on histopathology images. These methods require preprocessing of data set, which is a very tedious work, and it is also difficult to localize the detected results on original images. Fully convolutional network-based deep learning methods are able to take images as raw inputs, and output results of corresponding size, which makes it well suited for nuclei detection and classification task. In this study, we propose a novel multi-scale fully convolution network, named Cell Fully Convolutional Network (CFCN), with dilated convolution for fine-grained nuclei classification and localization in histology images. We trained CFCN in a typical histology image data set, and the experimental results show that CFCN outperforms the other state-of-the-art nuclei classification models, and the F1 score reaches 0.750.

MSAL-Net: improve accurate segmentation of nuclei in histopathology images by multiscale attention learning network

BackgroundThe digital pathology images obtain the essential information about the patient’s disease, and the automated nuclei segmentation results can help doctors make better decisions about diagnosing the disease. With the speedy advancement of convolutional neural networks in image processing, deep learning has been shown to play a significant role in the various analysis of medical images, such as nuclei segmentation, mitosis detection and segmentation etc. Recently, several U-net based methods have been developed to solve the automated nuclei segmentation problems. However, these methods fail to deal with the weak features representation from the initial layers and introduce the noise into the decoder path. In this paper, we propose a multiscale attention learning network (MSAL-Net), where the dense dilated convolutions block captures more comprehensive nuclei context information, and a newly modified decoder part is introduced, which integrates with efficient channel attention and boundary refinement modules to effectively learn spatial information for better prediction and further refine the nuclei cell of boundaries.ResultsBoth qualitative and quantitative results are obtained on the publicly available MoNuseg dataset. Extensive experiment results verify that our proposed method significantly outperforms state-of-the-art methods as well as the vanilla Unet method in the segmentation task. Furthermore, we visually demonstrate the effect of our modified decoder part.ConclusionThe MSAL-Net shows superiority with a novel decoder to segment the touching and blurred background nuclei cells obtained from histopathology images with better performance for accurate decoding.

FABNet: Fusion Attention Block and Transfer Learning for Laryngeal Cancer Tumor Grading in P63 IHC Histopathology Images.

Laryngeal cancer tumor (LCT) grading is a challenging task in P63 Immunohistochemical (IHC) histopathology images due to small differences between LCT levels in pathology images, the lack of precision in lesion regions of interest (LROIs) and the paucity of LCT pathology image samples. The key to solving the LCT grading problem is to transfer knowledge from other images and to identify more accurate LROIs, but the following problems occur: 1) transferring knowledge without a priori experience often causes negative transfer and creates a heavy workload due to the abundance of image types, and 2) convolutional neural networks (CNNs) constructing deep models by stacking cannot sufficiently identify LROIs, often deviate significantly from the LROIs focused on by experienced pathologists, and are prone to providing misleading second opinions. So we propose a novel fusion attention block network (FABNet) to address these problems. First, we propose a model transfer method based on clinical a priori experience and sample analysis (CPESA) that analyzes the transfer ability by integrating clinical a priori experience using indicators such as the relationship between the cancer onset location and morphology and the texture and staining degree of cell nuclei in histopathology images; our method further validates these indicators by the probability distribution of cancer image samples. Then, we propose a fusion attention block (FAB) structure, which can both provide an advanced non-uniform sparse representation of images and extract spatial relationship information between nuclei; consequently, the LROI can be more accurate and more relevant to pathologists. We conducted extensive experiments, compared with the best Baseline model, the classification accuracy is improved 25%, and It is demonstrated that FABNet performs better on different cancer pathology image datasets and outperforms other state of the art (SOTA) models.

Automated segmentation and classification of nuclei in histopathological images

Automated segmentation and classification of nuclei in histopathological images

Automated segmentation and classification of nuclei in histopathological images

Various kinds of cancer are detected and diagnosed using histopathological analysis. Recent advances in whole slide scanner technology and the shift towards digitisation of whole slides have inspired the application of computational methods on histological data. Digital analysis of histopathological images has the potential to tackle issues accompanying conventional histological techniques, like the lack of objectivity and high variability. In this paper, we present a framework for the automated segmentation of nuclei from human histopathological whole slide images, and their classification using morphological and colour characteristics of the nuclei. The segmentation stage consists of two methods, thresholding and the watershed transform. The features of the segmented regions are recorded for the classification stage. Experimental results show that the knowledge from the selected features is capable of classifying a segmented object as a candidate nucleus and filtering out the incorrectly identified segments.

Research on data classification and feature fusion method of cancer nuclei image based on deep learning

AbstractThere are many different types of nuclei in a tumor tissue. We can identify the specific nuclei and their distribution in the tissue to reflect the current cancer state of histopathological images. However, due to the existence of cellular heterogeneity, the recognition of nuclei in histopathological images has always been a problem of computer vision. In the paper, we use the transfer learning of Deep Convolutional Neural Network to classify nuclei, and found that adjusting the size of the nuclear image to a certain size can improve the accuracy of the nuclei classification model, while not significantly reducing the nuclei classification efficiency. Through further research, it was confirmed that the environment around the nuclei can bring great help to the model classification. Based on the principle, we design a feature fusion model. We extract features from nuclei image different sizes by CNN, fuse the features, and then use fully connected layer to classify the features. Experiments have proved that the feature fusion model has a considerable improvement in accuracy compared to the normal classification model.

Mask RCNN algorithm for nuclei detection on breast cancer histopathological images

AbstractNuclei detection is a key step in computer assisted pathology. Due to the variability of the size, shape, appearance, and texture of breast cancer nuclei in histopathological images, automated nuclei detection has always been a difficult aspect of computer‐aided pathology research. In this article, Mask RCNN is presented for the automatic detection of nuclei on high‐resolution histopathological images of breast cancer. Mask RCNN uses the ResNet network and effectively combines modules such as feature pyramid networks (FPN), ROIAlign, and fully convolutional networks (FCN). FPN can efficiently extract features of various dimensions in images. ROIAlign can improve the accuracy of the detection model in the detection task. FCN renders the prediction results more detailed. The experiment results show that the application of this algorithm is superior to other algorithms in terms of its intuitive vision, as well as in performance indicators such as accuracy, recall, and F‐Measure.

Foveal blur-boosted segmentation of nuclei in histopathology images with shape prior knowledge and probability map constraints.

In most tissue-based biomedical research, the lack of sufficient pathology training images with well-annotated ground truth inevitably limits the performance of deep learning systems. In this study, we propose a convolutional neural network with foveal blur enriching datasets with multiple local nuclei regions of interest derived from original pathology images. We further propose a human-knowledge boosted deep learning system by inclusion to the convolutional neural network new loss function terms capturing shape prior knowledge and imposing smoothness constraints on the predicted probability maps. Our proposed system outperforms all state-of-the-art deep learning and non-deep learning methods by Jaccard coefficient, Dice coefficient, Accuracy and Panoptic Quality in three independent datasets. The high segmentation accuracy and execution speed suggest its promising potential for automating histopathology nuclei segmentation in biomedical research and clinical settings. The codes, the documentation and example data are available on an open source at: https://github.com/HongyiDuanmu26/FovealBoosted. Supplementary data are available at Bioinformatics online.

Feature-driven local cell graph (FLocK): New computational pathology-based descriptors for prognosis of lung cancer and HPV status of oropharyngeal cancers.

Local spatial arrangement of nuclei in histopathology images of different cancer subtypes has been shown to have prognostic value. In order to capture localized nuclear architectural information, local cell cluster graph-based measurements have been proposed. However, conventional ways of cell graph construction only utilize nuclear spatial proximity, and do not differentiate between different cell types while constructing the graph. In this paper, we present feature-driven local cell cluster graph (FLocK), a new approach to constructing local cell graphs by simultaneously considering spatial proximity and attributes of the individual nuclei (e.g. shape, size, texture). In addition, we have designed a new set of quantitative graph-derived metrics to be extracted from FLocKs, in turn capturing the interplay between different proximally located clusters of nuclei. We have evaluated the efficacy of FLocK features extracted from H&E stained tissue images in two clinical applications: to classify short-term vs. long-term survival among patients of early stage non-small cell lung cancer (ES-NSCLC), and also to predict human papillomavirus (HPV) status of oropharyngeal squamous cell carcinoma (OP-SCCs). In the classification of long-term vs. short-term survival among patients of ES-NSCLC (training cohort, n=434), the top 10 discriminative FLocK features related to the variation of FLocK size and intersected FLocK distance were identified, via Minimum Redundancy and Maximum Relevance (MRMR) selection, in 100 runs of 10-fold cross-validation, and in conjunction with a linear discriminant classifier yielded a mean AUCof0.68 for predicting survival in the training cohort. This is better than other state-of-art histomorphometric and deep learning classifiers (cell cluster graphs (AUC=0.62), global cell graph (AUC=0.56), nuclear shape (AUC=0.54), nuclear orientation (AUC=0.61), AlexNet (AUC=0.55), ResNet (AUC=0.56)). The FLocK-based classifier yielded an AUC of 0.70 in an independent testing cohort (n=150). The patients identified as "high-risk" had significantly poorer overall survival in the testing cohort, with a hazard ratio (95% confidence interval) of 2.24 (1.24-4.05), p=0.01144). In the classification of HPV status of OP-SCC, the top three FLocK features pertaining to the portion of intersected FLocKs were used to construct a classifier, which yielded an AUC of 0.80 in the training cohort (n=50), and an accuracy of 0.78 in an independent testing cohort (n=35). The combination of FLocK measurements with cell cluster graphs, nuclear orientation, and nuclear shape improved the training AUC to 0.87, 0.91 and 0.85, respectively. Deep learning approaches yielded marginally better performance than the FLocK-based classifier in this application, with AUC=0.78 for AlexNet, AUC=0.81 for ResNet, and AUC=0.76 for FLocK-based classifier in the testing cohort. However, the combination of two hand-crafted features: FLocK and nuclear orientation yielded a better performance (AUC=0.84). FLocK provides a unique and quantitative way to analyze histology images of solid tumors and interrogate tumor morphology from a different aspect than existing histomorphometrics. The source code can be accessed at https://github.com/hacylu/FLocK.

Tumor Malignancy Detection Using Histopathology Imaging

Image segmentation and classification in the biomedical imaging field has high worth in cancer diagnosis and grading. The proposed method classifies the images based on a combination of handcrafted features and shape features using bag of visual words (BoW). The multistage segmentation technique to localize the nuclei in histopathology images includes the stain decomposition and histogram equalization to highlight the nucleus region, followed by the nuclei key point extraction using fast radial symmetry transform, normalized graph cut based on the nuclei region estimation, and nuclei boundary estimation using modified gradient. Subsequently, features from localized regions termed as handcrafted features and the shape features using BoW are extracted for classification. The experiments are performed using both the handcrafted features and BoW to take the advantages of both local nuclei features and globally spatial features. The simulation is performed on the Bisque and BreakHis data sets (with corresponding average accuracies of 93.87% and 96.96%, respectively) and confirms better diagnosis performance using the proposed method.

Staged Detection–Identification Framework for Cell Nuclei in Histopathology Images

Histopathology image is an important basis for pathologists to evaluate disease at the cellular level and colon cancer tissue sections usually contain many different types of nuclei, which should be automatically detected and identified. However, the detection and the identification of cell nuclei are challenging tasks due to the complex tissue structure and the diversity of nuclear morphology. In this paper, a staged detection-identification framework is proposed for cell nuclei in colon cancer histopathology images. First, nuclei positions are detected by a position of interest network, which encodes context-aware representation on input image and decodes features on proximity map. Meanwhile, a cascade residual fusion block is presented to enhance the detection performance during the decoding process. Second, a multicropping network is developed to identify the detected cell nuclei. For reducing the impact of uncertainty, a multicropping module is designed for effectively capturing contextual feature contents around the center of a nucleus. The proposed detection-identification framework is evaluated on an available colorectal adenocarcinoma images data set, which has 100 images including more than 20 000 marked nuclei. Compared with state-of-the-art methods, the proposed approach demonstrates excellent performance with better prediction scores.

Segmentation of Nuclei in Histopathology Images by Deep Regression of the Distance Map

The advent of digital pathology provides us with the challenging opportunity to automatically analyze whole slides of diseased tissue in order to derive quantitative profiles that can be used for diagnosis and prognosis tasks. In particular, for the development of interpretable models, the detection and segmentation of cell nuclei is of the utmost importance. In this paper, we describe a new method to automatically segment nuclei from Haematoxylin and Eosin (H&E) stained histopathology data with fully convolutional networks. In particular, we address the problem of segmenting touching nuclei by formulating the segmentation problem as a regression task of the distance map. We demonstrate superior performance of this approach as compared to other approaches using Convolutional Neural Networks.