Histopathology Image Segmentation Research Articles

HookNet: Multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide images.

We propose HookNet, a semantic segmentation model for histopathology whole-slide images, which combines context and details via multiple branches of encoder-decoder convolutional neural networks. Concentric patches at multiple resolutions with different fields of view, feed different branches of HookNet, and intermediate representations are combined via a hooking mechanism. We describe a framework to design and train HookNet for achieving high-resolution semantic segmentation and introduce constraints to guarantee pixel-wise alignment in feature maps during hooking. We show the advantages of using HookNet in two histopathology image segmentation tasks where tissue type prediction accuracy strongly depends on contextual information, namely (1) multi-class tissue segmentation in breast cancer and, (2) segmentation of tertiary lymphoid structures and germinal centers in lung cancer. We show the superiority of HookNet when compared with single-resolution U-Net models working at different resolutions as well as with a recently published multi-resolution model for histopathology image segmentation. We have made HookNet publicly available by releasing the source code1 as well as in the form of web-based applications2,3 based on the grand-challenge.org platform.

Gastric histopathology image segmentation using a hierarchical conditional random field

For the Convolutional Neural Networks (CNNs) applied in the intelligent diagnosis of gastric cancer, existing methods mostly focus on individual characteristics or network frameworks without a policy to depict the integral information. Mainly, conditional random field (CRF), an efficient and stable algorithm for analyzing images containing complicated contents, can characterize spatial relation in images. In this paper, a novel hierarchical conditional random field (HCRF) based gastric histopathology image segmentation (GHIS) method is proposed, which can automatically localize abnormal (cancer) regions in gastric histopathology images obtained by an optical microscope to assist histopathologists in medical work. This HCRF model is built up with higher order potentials, including pixel-level and patch-level potentials, and graph-based post-processing is applied to further improve its segmentation performance. Especially, a CNN is trained to build up the pixel-level potentials and another three CNNs are fine-tuned to build up the patch-level potentials for sufficient spatial segmentation information. In the experiment, a hematoxylin and eosin (H&E) stained gastric histopathological dataset with 560 abnormal images are divided into training, validation and test sets with a ratio of 1:1:2. Finally, segmentation accuracy, recall and specificity of 78.91%, 65.59%, and 81.33% are achieved on the test set. Our HCRF model demonstrates high segmentation performance and shows its effectiveness and future potential in the GHIS field.

Content-based image retrieval algorithm for nuclei segmentation in histopathology images

In today’s world, the medical diagnostic system shows a high reliance on medical imagery and digital nosology. To facilitate the fast and precise screening of samples, technology is leading towards the computer-aided disease diagnosis and grading. Image segmentation possesses high worth in the computer-aided disease diagnosis and grading systems to extract the region of interest. This paper presents a content-based image retrieval algorithm for histopathology image segmentation for identification and extraction of nuclei. The proposed technique furnishes nuclei segmentation in three cascaded stages; pre-processing, nuclei points and region refining, and composite nuclei segmentation. The performance of nuclei segmentation is investigated on six hematoxylins and eosin (H&E) stained histopathology images datasets. Simulation outcomes of the segmentation schemes confirm the superiority of the proposed method for nuclei segmentation in histopathology images in qualitative and quantitative analysis.

Hierarchical conditional random field model for multi‐object segmentation in gastric histopathology images

In this Letter, a hierarchical conditional random field (HCRF) model-based gastric histopathology image segmentation (GHIS) method is proposed, which can localise abnormal (cancer) regions in gastric histopathology images to assist histopathologists in medical work. First, to obtain pixel-level segmentation information, the authors retrain a convolutional neural network (CNN) to build up their pixel-level potentials. Then, to obtain abundant spatial segmentation information in patch level, they fine tune another three CNNs to build up their patch-level potentials. Thirdly, based on the pixel- and patch-level potentials, their HCRF model is structured. Finally, a graph-based post-processing is applied to further improve their segmentation performance. In the experiment, a segmentation accuracy of 78.91 % is achieved on a haematoxylin and eosin stained gastric histopathological dataset with 560 images, showing the effectiveness and future potential of the proposed GHIS method.

Design of a histopathology image segmentation algorithm for CAD of cancer

In the present world, medical diagnosis systems depend a lot on medical imaging and digital nosology. The digital nosology is leading to computer-aided diagnosis to facilitate the rapid and accurate screening of specimens. In this work, we present a method based on object shaped kernel for histopathology image segmentation to identify and analyze the cell nuclei region. The proposed segmentation technique is a multi-stage procedure, which includes contrast enhancement of the image, nuclei region extraction, nucleus centroid marking, nuclei area refinement and, complex nuclei separation. The performance of proposed method is measured using three standard H&E stained histopathology image datasets and a proposed dataset. The simulation results of the segmentation schemes are evaluated using the F1-score, Hausdorff distance and Jaccard index. The proposed system also introduced a novel performance measure as cumulative factor. The superiority of proposed segmentation system is verified in terms of cumulative factor in comparison to various the state of art methods.

Segmentation method of multiple sclerosis lesions based on 3D‐CNN networks

Histopathology image segmentation is an important area in the field of computer aided diagnosis using image processing. The segmentation of Multiple sclerosis (MS) lesions from MR images can establish the basis for subsequent lesion reconstruction, volume estimation, and course evaluation. This study proposes a method for automatically segmenting MS lesions based on 3D convolutional neural network (CNN). The method is divided into two stages, each of which includes two convolution layers and two pooling layers. The alternative lesion voxels are selected in the first stage, while in the second stage, the final lesion voxels are segmented from the lesion voxels which are obtained in the first stage by restricting the conditions. The method has been tested on the MICCAI 2008 and 2016 datasets and compared to the other baseline methods. The experiment results show that the method has better performance than the other baseline methods on different evaluation indicators, including dice similarity coefficient, absolute difference in lesion volume, true positive rate, false positive rate, and predictive positivity value.

Robust nuclei segmentation in histopathology using ASPPU-Net and boundary refinement

Automated nuclear segmentation in histopathological images is a prerequisite for a computer-aided diagnosis framework. However, it remains a challenging problem due to the nucleus occlusion or overlapping, shape variation, and image background complexity. Recently, deep learning techniques are widely used in analyzing digital histopathology. We present a computerized image-based method for automatically segmenting nuclei using an integration of a deep learning model and an improved concave point detection algorithm. A modified atrous spatial pyramid pooling U-Net (ASPPU-Net) is derived to capture multi-scale nuclei features and obtain nuclei context information without reducing the spatial resolution of feature map. A weighted binary cross entropy loss function with Dice loss function is used to better handle the data unbalance problem. An accelerated concave point detection method allows to effectively and accurately segmenting highly overlapping nuclei. Our ASPPU-Net based method was tested on four independent data cohorts and achieved the highest Dice similarity coefficient of 0.83, and pixel wise accuracy of 0.95. The experimental results suggested that the combination of ASPPU-Net model and concave point detection method was able to gain improved performance in separating both isolated and touching clustered nuclei in histopathology.

Histopathological cells segmentation using exponential grasshopper optimisation algorithm-based fuzzy clustering method

Automated cell segmentation in histopathological images is a challenging problem due to the complexities of these images. In this paper, a new exponential grasshopper optimisation algorithm is presented which is further used to find the optimal fuzzy clusters for segmenting the cells in histopathological images. For better cluster quality, compactness is considered as the objective function. The performance of the proposed method is validated in terms of F1 score and aggregated Jaccard index value on two standard histopathological image datasets, namely TNBC patients cancer dataset and UCSB bio segmentation images dataset. The simulation results show the effectiveness of the proposed method over other state-of-the-art clustering segmentation methods such as K-means and fuzzy c-means.

A Comparative Evaluation of Texture Features for Semantic Segmentation of Breast Histopathological Images

Breast histopathological image analysis helps in understanding the structure and distribution of the nucleus, thereby assisting in the detection of breast cancer. But analysis of histopathological image is challenging due to various reasons such as heterogeneity of nucleus structure, overlapping nuclei, clustered nuclei, variations in illumination, presence of noise etc. Limited availability of breast histopathological image dataset with fine annotations for detection of nucleus has restricted the analysis of histopathological images at the pixel-level. This paper presents fine annotations for nucleus segmentation of breast histopathological image datasets. Various textures such as Filter Banks, Gray Level Co-occurrence matrix and Local Binary Patterns are studied along with colour features for semantic segmentation of nuclei from histopathological images. Support Vector Machine and Multi Layer Perceptron algorithms are trained to perform pixelwise classification. The performance of the three texture features are evaluated on the two datasets and the results are presented in this paper.

A Comprehensive Review for Breast Histopathology Image Analysis Using Classical and Deep Neural Networks

Breast cancer is one of the most common and deadliest cancers among women. Since histopathological images contain sufficient phenotypic information, they play an indispensable role in the diagnosis and treatment of breast cancers. To improve the accuracy and objectivity of Breast Histopathological Image Analysis (BHIA), Artificial Neural Network (ANN) approaches are widely used in the segmentation and classification tasks of breast histopathological images. In this review, we present a comprehensive overview of the BHIA techniques based on ANNs. First of all, we categorize the BHIA systems into classical and deep neural networks for in-depth investigation. Then, the relevant studies based on BHIA systems are presented. After that, we analyze the existing models to discover the most suitable algorithms. Finally, publicly accessible datasets, along with their download links, are provided for the convenience of future researchers.

Enhancement and segmentation of histopathological images of cancer using dynamic stochastic resonance

Pathologists face difficulty in cell image detection as uneven dye causes the low contrast and inhomogeneity. The proposed discrete cosine transform (DCT)-based dynamic stochastic resonance (DSR) technique enhances the histopathological images of cancer. Further, the DSR-based Otsu's thresholding processed image helps in the better segmentation of histopathological images of four types of cancer cells, i.e., breast, cervix, ovarian and prostate cancer. The comparison of segmentation results were performed on the University of California, Santabarbara (UCSB) available breast cancer datasets for analysis. The algorithm has been applied to total 22 breast cancer images including benign and malignant and compared with region of interest (ROI) segmented ground truth images to validate the performance of proposed DSR-based Otsu's thresholding. DSR-based Otsu's segmentation obtained better results with 0.776 average correlation, 0.979 average normalised probabilistic rand (NPR) index, 0.011 average global consistency error (GCE), and 0.185 average variation of information (VI). These indices are higher than the other conventional segmentation methods and have the advantage to identify the target objects in low contrast images.

Enhancement and segmentation of histopathological images of cancer using dynamic stochastic resonance

Pathologists face difficulty in cell image detection as uneven dye causes the low contrast and inhomogeneity. The proposed discrete cosine transform (DCT)-based dynamic stochastic resonance (DSR) technique enhances the histopathological images of cancer. Further, the DSR-based Otsu's thresholding processed image helps in the better segmentation of histopathological images of four types of cancer cells, i.e., breast, cervix, ovarian and prostate cancer. The comparison of segmentation results were performed on the University of California, Santabarbara (UCSB) available breast cancer datasets for analysis. The algorithm has been applied to total 22 breast cancer images including benign and malignant and compared with region of interest (ROI) segmented ground truth images to validate the performance of proposed DSR-based Otsu's thresholding. DSR-based Otsu's segmentation obtained better results with 0.776 average correlation, 0.979 average normalised probabilistic rand (NPR) index, 0.011 average global consistency error (GCE), and 0.185 average variation of information (VI). These indices are higher than the other conventional segmentation methods and have the advantage to identify the target objects in low contrast images.

Exploration and Performance Assessment of Efficient Histopathology Image Pre-Processing and Segmentation Techniques for Breast Cancer Prediction

Evaluation of features after segmentation is a significant process in image processing, especially in the medical field. Medical imaging technologies are widely utilized in clinical diagnosis to guide therapeutic and surgical decisions and to monitor disease advancement, understanding occurrence and re-occurrence of infection and view treatment response. Among all cancer kinds, Breast Cancer (BC) now-a-days has turn out to be a common form of cancer amongst ladies around the world and is the second common cause of most cancers deaths. At present, there aren't any powerful methods to save you and remedy breast cancer, because its cause is not absolutely known. Early detection is the most effective way to enhance breast cancer survivals and might deliver a better hazard of full healing. The main motive of this paper is to implement various efficient image segmentation techniques after preprocessing histopathology BC images of different magnification level and exhaustively compare them to achieve best results. We analyze the outcomes of widely used edge detection based image segmentation methods such as Adaptive K-Means, Multi Class Fuzzy C-Means, Canny, Gabor filter and Homogeneity based PSO and evaluate them via metrics which includes accuracy, precision, recall and F-degree. Various preprocessing strategies like histogram equalization (HE), Adaptive equalization (AE) and Contrast Stretching (CS) are compared and used to enhance the performance of above segmentation methods. Contrast Improvement Index (CII) is chosen as a selection criteria for preprocessed image. Finally one vs. all multiclass SVM classifier is used for BC image classification. Our implementation uses breast cancer dataset having two classes as benign and malignant each in turn having four sub-classes. Images of different magnification levels such as 40x, 100x, 200x and 400x, are considered for BC detection.

Toward the modification of flower pollination algorithm in clustering-based image segmentation

Flower pollination algorithm (FPA) is a new bio-inspired optimization algorithm, which has shown an effective performance on solving many optimization problems. However, the effectiveness of FPA significantly depends on the balance achieved by the exploration and exploitation evolutionary stages. Since purely exploration procedure promotes non-accurate solutions, meanwhile, purely exploitation operation promotes sub-optimal solutions in the presence of multiple optima. In this study, three global search and two local search strategies have been designed to improve balance among evolutionary stages, increasing the efficiency and robustness of the original FPA methodology. Additionally, some parameter adaptation techniques are also incorporated in the proposed methodology. The modified FPA has been successfully applied for histopathological image segmentation problem. The experimental and computational effort results indicate its effectiveness over existing swarm intelligence algorithms and machine learning methods.

Strategies to Reduce the Expert Supervision Required for Deep Learning-Based Segmentation of Histopathological Images.

The emergence of computational pathology comes with a demand to extract more and more information from each tissue sample. Such information extraction often requires the segmentation of numerous histological objects (e.g., cell nuclei, glands, etc.) in histological slide images, a task for which deep learning algorithms have demonstrated their effectiveness. However, these algorithms require many training examples to be efficient and robust. For this purpose, pathologists must manually segment hundreds or even thousands of objects in histological images, i.e., a long, tedious and potentially biased task. The present paper aims to review strategies that could help provide the very large number of annotated images needed to automate the segmentation of histological images using deep learning. This review identifies and describes four different approaches: the use of immunohistochemical markers as labels, realistic data augmentation, Generative Adversarial Networks (GAN), and transfer learning. In addition, we describe alternative learning strategies that can use imperfect annotations. Adding real data with high-quality annotations to the training set is a safe way to improve the performance of a well configured deep neural network. However, the present review provides new perspectives through the use of artificially generated data and/or imperfect annotations, in addition to transfer learning opportunities.

An efficient nuclei segmentation method based on roulette wheel whale optimization and fuzzy clustering

Image segmentation is a technique in which an image is partitioned into different categories or regions that correspond to objects or parts of objects. The performance of segmentation methods is generally reduced for the hematoxylin and eosin stained histopathological images due to complex background and varying intensity values. Therefore, in this paper, the roulette wheel selection whale optimization based fuzzy clustering method is introduced for the nuclei segmentation of histopathological images. The proposed clustering method finds the optimal clusters using the objective function that reduces the sum of squared error or compactness. The performance of the proposed clustering method has been examined on the histopathological image dataset of Triple-Negative Breast Cancer patients and compared with k-means and fuzzy c-means in respect of F1 score and aggregated Jaccard index. The proposed method attains 0.6701 mean F1 scores and 0.7387 mean aggregated Jaccard index value, which are the best values among other methods. The experimental outcomes validate the efficiency of the introduced method over the other clustering-based segmentation methods.

Upgraded Segmentation of Histopathological Images for classification of intraductal breast lesions

Cancer among women and the second most common cancer in the world is Breast Cancer(BC). This type of cancer-initiating from breast tissue, mostly from the inner region of milk ducts. The current progress in high-throughput and getting of digitized histological studies have made it possible to use histological pattern with image analysis to facilitate disease classification using computer-aided technology (CAT). The practice of analysis has become a part of the routine clinical discovery of breast cancer. In fact, CAT has become recent research subjects in the diagnostic of medical imaging and radiology. The vast increase in the capability of image acquisition and computational power in recent decades has prompted the development of several image segmentation algorithms. For the analysis of histopathological images, the automatic dissection of cell nuclei is an important stage. Its prime objective is to determine the exact location of the nuclei and boundary points of the cells. To accurately model the preference for histological structures (ducts, vessels, tumor nets, adipose, etc.). In the proposed method, additional k means clustering algorithm used for evaluating segmentation algorithms. Here demonstrate the in proposed methods over the state-of-the-art system in performance measures.

A deep learning algorithm for one-step contour aware nuclei segmentation of histopathology images

This paper addresses the task of nuclei segmentation in high-resolution histopathology images. We propose an automatic end-to-end deep neural network algorithm for segmentation of individual nuclei. A nucleus-boundary model is introduced to predict nuclei and their boundaries simultaneously using a fully convolutional neural network. Given a color-normalized image, the model directly outputs an estimated nuclei map and a boundary map. A simple, fast, and parameter-free post-processing procedure is performed on the estimated nuclei map to produce the final segmented nuclei. An overlapped patch extraction and assembling method is also designed for seamless prediction of nuclei in large whole-slide images. We also show the effectiveness of data augmentation methods for nuclei segmentation task. Our experiments showed our method outperforms prior state-of-the-art methods. Moreover, it is efficient that one 1000×1000 image can be segmented in less than 5 s. This makes it possible to precisely segment the whole-slide image in acceptable time. The source code is available at https://github.com/easycui/nuclei_segmentation . Graphical Abstract The neural network for nuclei segmentation.

A Clustering Based Classification Approach Based on Modified Cuckoo Search Algorithm

Cuckoo Search Algorithm (CSA) is one of the new swarm intelligence based optimization algorithms, which has shown an effective performance on many optimization problems. However, the effectiveness of CSA significantly depends on the exploration and exploitation potential and it may also possible to increase its efficiency when solving complex optimization problems. In this study, some mechanisms have been employed on CSA to increase its efficiency such as use of global best and individual best solutions to guide the other solutions, self-adaption techniques for parameters and so on. The modified CSA (i.e., MCSA) is successfully employed in clustering based classification domain. The experimental results and execution time prove its effectiveness over existing modified CSAs and other employed swarm intelligence algorithms. The proposed clustering model is also employed in color histopathological image segmentation domain and provides effective result.

A U-net based approach to epidermal tissue segmentation in whole slide histopathological images

Malignant melanoma is a severe and aggressive type of skin cancer, with a rapid decrease in survival rate if not diagnosed and treated at an early stage. Histopathological examination of hematoxylin and eosin stained tissue biopsies under a light microscope is currently the gold standard for diagnosis. However, this manual examination is a difficult and time-consuming task, and diagnosis is often subject to intra- and inter-observer variability. With more pathology departments starting to convert conventional glass slides into digital resources, a Computer Aided Diagnostic (CAD) system that can automate part of the diagnostic process will help address these challenges. It is expected to reduce examination time, increase diagnostic accuracy, and reduce diagnostic variations. An important initial step in developing such a system is an automated epidermis segmentation algorithm, since several important diagnostic factors are within or seen relatively to the epidermis’ location. In this paper, we propose a new epidermis segmentation technique built on Convolutional Neural Networks. We trained an U-net based architecture end-to-end, with sim 380hbox {k} overlapping high resolution image patches at 512 times 512 pixels, extracted and augmented from 36 digitized histopathological images from two different clinical sites, to discriminate pixels as either epidermal or non-epidermal. The proposed technique was evaluated on 33 test images, where we achieved a mean Positive Predictive Value at 0.89pm 0.16, Sensitivity at 0.92pm 0.1, Dice Similarity Coefficient at 0.89pm 0.13 and a Matthews Correlation Coefficient at 0.89pm 0.11, showing a superior performance when compared to existing techniques. Our algorithm also proves to be robust to variations in staining, tissue thickness and laboratory pre-processing.