Segmentation Of Cell Nuclei Research Articles

Abstract PO-043: Combining multiplexed immunohistochemistry and deep learning to spatially map the tumor microenvironment

Abstract The tumour microenvironment(TME) is a highly complex mixture containing epithelium, stroma and a diverse network of immune cells and the spatial organization of these immune cells within the TME reflects a crucial process in anti-tumor immunity. The usual standard of care for assessing if a patient has cancer, its stage and its likely future biological behaviour is visual examination of one or more H&E and/or Immunohistochemistry(IHC) stained sections. The paradigm of digital pathology has changed, moving from single-marker IHC towards multiplexed labeling, increasing the need for more advanced techniques that can be easily integrated in routine clinical pathology. Although recent advances in multiple immunostaining have enabled characterization of several parameters on a single tissue section. For a higher dimensional chromogen based methodology, we have developed a multiplexed IHC procedure combining multiple labels per round with several sequential rounds, where multiple is 12-14 chromogen based antibodies on a single tissue section. Thus, enabling analysis of complex immune cell population’s on a single slide through consecutive cycles of staining, destaining, hyperspectral imaging and spectral unmixing of the chromogen biomarkers in each round. The process presented is using absorption microscopy, enabling these images to be done in a reasonable time frame. Not only does this allow us to completely visualize and spatially map the TME, but through the application of AI we can further recognize common histological features. Stoichiometric DNA following mIHC, improves nuclei identification so we can identify various cell populations present in the tissue. Thus, providing us with an accurate spatial cell level representation of any tissue section using only a single staining process. Although there have been several uprising technologies which characterize the complexities of the TME, our process is capable of doing so in a shorter time frame and at a reduced cost using absorption based methods. Robust, accurate, segmentation of cell nuclei for overlapping nuclei is one of the most significant unsolved issues in digital pathology. By combining a multiplexed IHC technique which enables the detection of multiple markers on a single slide with deep learning segmentation methods to segment every individual cell nuclei in tissue sections with an accuracy comparable to human annotation. We can analyze the cell-cell interactions between immune and tumour cells and identify clinically relevant patterns that may improve patients outcome by informing on the likelihood of success of possible treatments. These two techniques joined can be scaled up to the entire tissue section level, improving our understanding of the biological aggressiveness of specific cancers and enabling an accurate spatial cell level representation of the tissue. Citation Format: Kouther Noureddine, Paul Gallagher, Martial Guillaud, Calum MacAulay. Combining multiplexed immunohistochemistry and deep learning to spatially map the tumor microenvironment [abstract]. In: Proceedings of the AACR Virtual Special Conference on Artificial Intelligence, Diagnosis, and Imaging; 2021 Jan 13-14. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(5_Suppl):Abstract nr PO-043.

Abstract PO-009: Towards solving overlapping nuclei segmentation: Sequential CNNs for one to many mapping of pixels to objects

Abstract Automated cell nucleus segmentation is key to quantifying cell features and functionality to identify the disease state of tissue and its likely biological future development. Despite considerable advances in automated segmentation, it is a challenging task to split overlapping clustered nuclei where pixels can belong to more than one nuclei. To substantially improve upon current cell/nuclei segmentation methods our group has developed a promising nuclear segmentation method using sequential rounds of modified UNet CNNs which we denote SUNet. This sequential application of two separate CNNs is required to allow the reparsing the original image into multiple overlapping sub images which explicitly allows for the SUNet approach to perform a one pixel mapping to many objects that is missing from other CNN based segmentation methods in Digital Pathology. We initially constructed two human annotated 8000 overlapping nuclei training sets one each from lung and prostate tissue sections quantitatively stained for DNA. These were used to train two SUNets. Their performance was evaluated on ~20,000 nuclei from multiple tissues for each tissue type. The average correct visual nuclei segmentation rate was observed to be ~84%. A subset of the training images were hand annotated by multiple observers and the Jaccard similarity coefficient (JSC) was used to quantify the similarity between these annotations. Four human observers have annotated the same 1065 nuclei from 100 nuclei clusters. For the 6 pairwise comparisons of these individuals, the average JSC was found to be 0.69 (range 0.63-0.75). This low number reflects the inability of humans to exactly trace the same boundary by hand and that not all observers recognized the same nuclei within the highly complex clusters. The SUNets JSC was similar to the average JSC and within the range of the human observers. Indicating that for these images the SUNet performs as well as a human at segmenting nuclei within complex clusters of nuclei. Comparing the performance of SUNet with a published mask_RCNN based nuclei segmentation and a recently published nucleAIzer method on a non-complex (not a lot of overlapping nuclei) image; we find the three are almost equivalent. However, the two published methods miss a few overlapping nuclei that are found by SUNet. On images with highly complex overlapping clusters of nuclei SUnet correctly segments more (10-20%, image complexity dependent) nuclei than the other methods in areas of high complexity. In addition, SUNet recognizes and segments large numbers of nuclei completely missed by the other methods. An added benefit of allowing a one to many assignment of pixels, is that it correctly preserves the complete shape of the nuclei as it is able to assign the same pixel(s) in an image to the multiple nuclei involved in the area of their overlap as apposed to assigning it to one of the nuclei involved. In summary, the sequential CNN proposed approach can segment nuclei in complex cluster with an accuracy equivalent to human observers. Citation Format: Calum E. MacAulay, Kouther Noureddine, Martial Guillaud, Paul Gallagher. Towards solving overlapping nuclei segmentation: Sequential CNNs for one to many mapping of pixels to objects [abstract]. In: Proceedings of the AACR Virtual Special Conference on Artificial Intelligence, Diagnosis, and Imaging; 2021 Jan 13-14. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(5_Suppl):Abstract nr PO-009.

Abstract PS3-29: Pixel-level tissue classification from ultrasound images of breast cancer and direct comparison to matched histological measurements

Abstract Introduction: The use of noninvasive ultrasound (US) for quantitative tissue characterization has been an exciting research prospect for several decades now. Herein the challenge is to find hidden patterns in the US data to reveal more information about tissue function and pathology that cannot be seen in the more conventional US images. A new pixel-level analysis technique has been developed by our group for tissue classification. Termed H-scan US (H stands for hue or Hermite), this imaging approach links a special class of nth-ordered Gaussian-weighted Hermite functions (GHn) to the physics of US scattering and reflection from different tissue structures. The sensitivity of in vivo H-scan US imaging to subtle changes in scatterer size is a question that has not been fully answered. To that end, the purpose of this study was to compare local H-scan US image intensity to direct (co-registered) histological measures made at the cellular level. Methodology Female nude athymic mice were implanted into mammary fatty pad with breast cancer cells (N = 8). Implanted tumors were allowed to grow for about four weeks before study enrollment. Image data was acquired using a programmable US scanner (Vantage 256, Verasonics Inc) equipped with a 256-element L22-8v CMUT linear array transducer (Kolo Medical). Plane wave imaging with 5 angles was performed at a center frequency of 15 MHz. To generate the H-scan US image, three parallel convolution filters (GH2, GH6, and GH10) were applied to the radiofrequency (RF) data sequences to measure the relative strength of the received signals. After envelope detection, the relative strength of the filter outputs is color coded whereby the lower frequency (GH2 = 9 MHz) backscattered US signal components are assigned to the R channel, moderate frequency (GH6 = 15 MHz) signals are assigned to the G channel, and the higher frequency (GH10 = 21 MHz) signals to the B channel. After performing H-scan US, the imaging cross-section was marked, and the tumors were excised and sliced along the same plane for histologic processing. After nuclear staining, tissue sections were scanned and digitized using confocal microscopy and fluorescent filters for DAPI. Automated segmentation of each cancer cell nucleus in the histologic sections was performed using an active contour technique. US images were interpolated to the same number of pixels as the histology image and spatial alignment between the two was performed. Lastly, nucleus size and density from histologic sections was compared to local H-scan US image features. Results: Custom software was developed to compare local US image features at the pixel level to that of co-registered histologic images and measurements of nuclear size and density. In an attempt to properly match the histological and the H-scan US images, three different sized kernels of 0.16 ✗ 0.16 mm, 0.32 ✗ 0.32 mm or 0.5 ✗ 0.5 mm were used to partition both the H-scan US and histology images into 1888, 448 or 62 distinct region-of-interests (ROIs), respectively. Mean nuclear size and density measurements from the histology ROIs was compared to the mean H-scan US image intensity from the many spatially matched ROI locations. A statistically significant linear relationship was found between local H-scan US image intensity and nuclear size (R2 > 0.4, < 0.001) and density (R2 > 0.6, p < 0.001). Conclusions: Preliminary results from use of an animal model of breast cancer reveals that in vivo H-scan US images positively correlated with physical measures of nucleus size and density as quantified from co-registered histologic images. Citation Format: Mawia Khairalseed, Shreya Reddy, Jane Song, Girdhari Rijal, Kenneeth Hoyt. Pixel-level tissue classification from ultrasound images of breast cancer and direct comparison to matched histological measurements [abstract]. In: Proceedings of the 2020 San Antonio Breast Cancer Virtual Symposium; 2020 Dec 8-11; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2021;81(4 Suppl):Abstract nr PS3-29.

FBUNet: Full Convolutional Network Based on Fusion Block Architecture for Biomedical Image Segmentation

Precise segmentation of organs or tumors is essential for diagnosis and prognosis. We propose two novel improved end-to-end segmentation models, i.e. FBUNet-1 and FBUNet-2. The FBUNet-1 model shows higher performance by reducing the loss of spatial information in convolutional operations than the classic U-Net. The FBUNet-2 model can further increase accuracy by modifying the loss function based on the FBUNet-1 model. In this research, we compare the proposed models with the classic U-Net and deep residual U-Net models against four evaluation indexes, i.e. Dice coefficient, Jaccard similarity and Sensitivity and Precision respectively. The experimental results show that with a cut of almost one-third of the training parameters, the FBUNet-1 and FBUNet-2 models can still improve comprehensive performance in the cell edge segmentation, blood vessel segmentation, lung segmentation and cell nuclei segmentation. For example, the average Dice coefficient is 93.96%, Jaccard Similarity 88.62%, Sensitivity 94.19% and Precision 93.73% in cell segmentation. In addition, the average of fivefold cross validation of the proposed FBUNet-2 model increases by 0.5% of Jaccard Similarity, 0.3% of Dice coefficient and 0.9% of Jaccard Similarity, 0.6% of Dice coefficient for cell edge segmentation and cell nuclei segmentation compare with U-Net model. Compared with deep residual U-Net and classic U-Net models, the FBUNet-1 and FBUNet-2 models have potential and practical clinical applications.

The Role of a Mask - Understanding the Performance of Deep Neural Networks to Detect, Segment, and Extract Cellular Nuclei from Microscopy Images

Cell segmentation is the task of identifying cell nuclei instances in fluorescence microscopy images. It plays a key role in biomedical analysis tasks like cell characterization, cancer cell identification, and gene expression measurement. There has been a recent proliferation of deep neural network-based object detection techniques for cell segmentation. However, previous literature does not suitably address the proper understanding of the different methods and the merits of the various neural architectures proposed. One key design point relates to the learning of segmentation masks and how neural methods go about detecting, segmenting, and extracting nuclei from unseen images. The goal of this paper is to benchmark the performance of representative deep learning techniques for cell nuclei segmentation using standard datasets and common evaluation criteria. We investigate whether for cell nuclei segmentation, learning the nuclei masks in parallel with boundary detection features of the image provides superior performance and significantly cleaner separation of nuclei from images. We further explore the trade-off between increased accuracy, achieved through more complex deep learning models, and the heavy requirements imposed on both computational resources and training times. We believe this paper establishes an important baseline for cell nuclei segmentation, enabling researchers to continually refine and deploy neural models for real-world clinical applications.

Deep Learning Model for Cell Nuclei Segmentation and Lymphocyte Identification in Whole Slide Histology Images

Anti-cancer immunotherapy dramatically changes the clinical management of many types of tumours towards less harmful and more personalized treatment plans than conventional chemotherapy or radiation. Precise analysis of the spatial distribution of immune cells in the tumourous tissue is necessary to select patients that would best respond to the treatment. Here, we introduce a deep learning-based workflow for cell nuclei segmentation and subsequent immune cell identification in routine diagnostic images. We applied our workflow on a set of hematoxylin and eosin (H&E) stained breast cancer and colorectal cancer tissue images to detect tumour-infiltrating lymphocytes. Firstly, to segment all nuclei in the tissue, we applied the multiple-image input layer architecture (Micro-Net, Dice coefficient (DC) $0.79\pm 0.02$). We supplemented the Micro-Net with an introduced texture block to increase segmentation accuracy (DC = $0.80\pm 0.02$). We preserved the shallow architecture of the segmentation network with only 280 K trainable parameters (e.g. U-net with ∼1900 K parameters, DC = $0.78\pm 0.03$). Subsequently, we added an active contour layer to the ground truth images to further increase the performance (DC = $0.81\pm 0.02$). Secondly, to discriminate lymphocytes from the set of all segmented nuclei, we explored multilayer perceptron and achieved a 0.70 classification f-score. Remarkably, the binary classification of segmented nuclei was significantly improved (f-score = 0.80) by colour normalization. To inspect model generalization, we have evaluated trained models on a public dataset that was not put to use during training. We conclude that the proposed workflow achieved promising results and, with little effort, can be employed in multi-class nuclei segmentation and identification tasks.

Cellpose: a generalist algorithm for cellular segmentation.

Many biological applications require the segmentation of cell bodies, membranes and nuclei from microscopy images. Deep learning has enabled great progress on this problem, but current methods are specialized for images that have large training datasets. Here we introduce a generalist, deep learning-based segmentation method called Cellpose, which can precisely segment cells from a wide range of image types and does not require model retraining or parameter adjustments. Cellpose was trained on a new dataset of highly varied images of cells, containing over 70,000 segmented objects. We also demonstrate a three-dimensional (3D) extension of Cellpose that reuses the two-dimensional (2D) model and does not require 3D-labeled data. To support community contributions to the training data, we developed software for manual labeling and for curation of the automated results. Periodically retraining the model on the community-contributed data will ensure that Cellpose improves constantly.

An Empirical Evaluation of Nuclei Segmentation from H&E Images in a Real Application Scenario

Cell nuclei segmentation is a challenging task, especially in real applications, when the target images significantly differ between them. This task is also challenging for methods based on convolutional neural networks (CNNs), which have recently boosted the performance of cell nuclei segmentation systems. However, when training data are scarce or not representative of deployment scenarios, they may suffer from overfitting to a different extent, and may hardly generalise to images that differ from the ones used for training. In this work, we focus on real-world, challenging application scenarios when no annotated images from a given dataset are available, or when few images (even unlabelled) of the same domain are available to perform domain adaptation. To simulate this scenario, we performed extensive cross-dataset experiments on several CNN-based state-of-the-art cell nuclei segmentation methods. Our results show that some of the existing CNN-based approaches are capable of generalising to target images which resemble the ones used for training. In contrast, their effectiveness considerably degrades when target and source significantly differ in colours and scale.

Abstract PO-082: Combining multiplex immunohistochemistry and deep learning: A new approach to tracing the tumour microenvironment

Abstract The tumour microenvironment(TME) is a highly complex mixture containing epithelium, stroma and a diverse network of immune cells and the spatial organization of these immune cells within the TME reflects a crucial process in anti-tumor immunity. Not all tumour cells interact similarly; micro-environmental features are often key indicators of treatment response and patient outcome. The usual standard of care for assessing if a patient has cancer, its stage and its likely future biological behaviour is visual examination of one or more H&E and/or Immunohistochemistry(IHC) stained sections. The paradigm of digital pathology has changed, moving from single-¬marker IHC towards multiplexed labeling, increasing the need for more advanced techniques that can be easily integrated in routine clinical pathology. Although recent advances in multiple immunostaining have enabled characterization of several parameters on a single tissue section. For a higher dimensional chromogen based methodology, we have developed a multiplexed IHC procedure combining multiple labels per round with multiple sequential rounds, where multiple is 12-14 chromogen based antibodies on a single tissue section. Thus, enabling analysis of complex immune cell population’s on a single slide through consecutive cycles of staining, destaining, hyperspectral imaging and spectral unmixing of the chromogen biomarkers in each round. That the process presented is chromogen based (absorption microscopy) means that high throughput imaging of 12-14 markers across entire slides is feasible in a reasonable time frame. Robust, accurate, segmentation of cell nuclei for touching/overlapping nuclei is one of the most significant unsolved issues in digital pathology. Analyzing cell-cell interactions between immune and tumour cells and identifying clinically relevant patterns may improve patients outcome by informing on the likelihood of success of possible treatments. By combining a multiplexed IHC technique which enables the detection of multiple markers on a single slide with deep learning segmentation methods to recognize/segment every individual cell nuclei in tissue sections with an accuracy comparable to human annotation, we can improve the accuracy of tissue classification based upon the measured characteristics of these cells and their spatial organization. These two techniques joined can be scaled up to the entire tissue section level, improving our understanding of the biological aggressiveness of specific cancers and enabling an accurate spatial cell level representation of the tissue. Citation Format: Kouther Noureddine, Paul Gallagher, Martial Guillaud, Calum MacAulay. Combining multiplex immunohistochemistry and deep learning: A new approach to tracing the tumour microenvironment [abstract]. In: Proceedings of the AACR Virtual Special Conference on Tumor Heterogeneity: From Single Cells to Clinical Impact; 2020 Sep 17-18. Philadelphia (PA): AACR; Cancer Res 2020;80(21 Suppl):Abstract nr PO-082.

NuSeT: A deep learning tool for reliably separating and analyzing crowded cells.

Segmenting cell nuclei within microscopy images is a ubiquitous task in biological research and clinical applications. Unfortunately, segmenting low-contrast overlapping objects that may be tightly packed is a major bottleneck in standard deep learning-based models. We report a Nuclear Segmentation Tool (NuSeT) based on deep learning that accurately segments nuclei across multiple types of fluorescence imaging data. Using a hybrid network consisting of U-Net and Region Proposal Networks (RPN), followed by a watershed step, we have achieved superior performance in detecting and delineating nuclear boundaries in 2D and 3D images of varying complexities. By using foreground normalization and additional training on synthetic images containing non-cellular artifacts, NuSeT improves nuclear detection and reduces false positives. NuSeT addresses common challenges in nuclear segmentation such as variability in nuclear signal and shape, limited training sample size, and sample preparation artifacts. Compared to other segmentation models, NuSeT consistently fares better in generating accurate segmentation masks and assigning boundaries for touching nuclei.

Deep learning based breast cancer detection and classification using fuzzy merging techniques

Automatic identification of abnormal and normal cells is a critical step in computer-assisted pathology, owing to certain heterogeneous characteristics of cancer cells. However, automated nuclei detection is problematic in unevenly shaped, overlapping and touching nuclei. It is, consequently, essential to detect single and overlapping nuclei and distinguish them from single ones for a reasonable quantitative analysis. Diagnosis is improved by introducing a computer-aided diagnosis system to automatically detect breast cancer tissue nuclei from whole slide images of hematoxylin and eosin stains. We propose a method for the automatic cell nuclei detection, segmentation, and classification of breast cancer using a deep convolutional neural network (Deep-CNN) approach. The main contribution of this work is the detection of nuclei using anisotropic diffusion in a filter and applying a novel multilevel saliency nuclei detection model in ductal carcinoma of breast cancer tissue. The detected nuclei are classified into benign and malignant cells by applying the new Deep-CNN model. Finally, the novel multilevel saliency nuclei detection technique is integrated with the Deep-CNN to produce an nMSDeep-CNN model that turns out to be the most accurate results with very less computation time. The accuracy, sensitivity and specificity of the proposed system are 98.62%, 0.947 and 0.964, respectively. The classification for benign and malignant cells is evaluated by applying 10 fold cross-validation. Thus, the system can be clinically used for an objective, accurate, and rapid diagnosis of abnormal tissue. The effectiveness of the suggested framework is demonstrated through experiments on several datasets.

U-shaped Feature Extractor Used on Mask R-CNN for Cell Nuclei Image Segmentation

The cell nuclei segmentation of is a challenging task in microscopy image analysis. The problems of noise, small cell nuclei, and few training data samples in the data set will all affect the effectiveness of the model to varying degrees. This paper presents a new approach to nuclear image segmentation based on convolutional neural networks. Our approach is based on Mask R-CNN with some modification, which combines low-level semantic features for model training. In order to make the model specific to each low-level feature map, the attention mechanism was used to assign weights to each low-level feature map, making the model learning more purposeful. Our method achieves an average precision value of 62.8%, which is 2.7% higher than that of the Mask R-CNN (ResNet50) basic model and 6.3% of the Mask R-CNN (ResNet101) basic model.

Evaluation of semi-supervised learning using sparse labeling to segment cell nuclei

Abstract The analysis of microscopic images from cell cultures plays an important role in the development of drugs. The segmentation of such images is a basic step to extract the viable information on which further evaluation steps are build. Classical image processing pipelines often fail under heterogeneous conditions. In the recent years deep neuronal networks gained attention due to their great potentials in image segmentation. One main pitfall of deep learning is often seen in the amount of labeled data required for training such models. Especially for 3D images the process to generate such data is tedious and time consuming and thus seen as a possible reason for the lack of establishment of deep learning models for 3D data. Efforts have been made to minimize the time needed to create labeled training data or to reduce the amount of labels needed for training. In this paper we present a new semisupervised training method for image segmentation of microscopic cell recordings based on an iterative approach utilizing unlabeled data during training. This method helps to further reduce the amount of labels required to effectively train deep learning models for image segmentation. By labeling less than one percent of the training data, a performance of 90% compared to a full annotation with 342 nuclei can be achieved.

Automatic three-dimensional segmentation of mouse embryonic stem cell nuclei by utilising multiple channels of confocal fluorescence images.

Time-lapse confocal fluorescence microscopy images from mouse embryonic stem cells (ESCs) carrying reporter genes, histone H2B-mCherry and Mvh-Venus, have been used to monitor dynamic changes in cellular/differentiation characteristics of live ESCs. Accurate cell nucleus segmentation is required to analyse the ESC dynamics and differentiation at a single cell resolution. Several methods used concavities on nucleus contours to segment overlapping cell nuclei. Our proposed method evaluates not only the concavities but also the size and shape of every 2D nucleus region to determine if any of the strait, extrusion, convexity and large diameter criteria is satisfied to segment overlapping nuclei inside the region. We then use a 3D segmentation method to reconstruct simple, convex, and reasonably sized 3D nuclei along the image stacking direction using the radius and centre of every segmented region in respective microscopy images.To avoid false concavities on nucleus boundaries, fluorescence images of the H2B-mCherry reporter are used for localisation of cell nuclei and Venus fluorescence images are used for determining the cell colony ranges. We use a series of image preprocessing procedures to remove noise outside and inside cell colonies, and in respective nuclei, and to smooth nucleus boundaries based on the colony ranges. We propose dynamic data structures to record every segmented nucleus region and solid in sets (volumes) of 3D confocal images. The experimental results show that the proposed image preprocessing method preserves the areas of mouse ESC nuclei on microscopy images and that the segmentation method effectively segment out every nucleus with a reasonable size and shape. All 3D nuclei in a set (volume) of confocal microscopy images can be accessed by the dynamic data structures for 3D reconstruction. The 3D nuclei in time-lapse confocal microscopy images can be tracked to calculate cell movement and proliferation in consecutive volumes for understanding the dynamics of the differentiation characteristics about ESCs. LAY DESCRIPTION: Embryonic stem cells (ESCs) are considered as an ideal source for basic cell biology study and producing medically useful cells in vitro.This study uses time-lapse confocal fluorescence microscopy images from mouse ESCs carrying reporter gene to monitor dynamic changes in cellular/differentiation characteristics of live ESCs. To automate analyses of ESC differentiation behaviours, accurate cell nucleus segmentation to distinguish respective cells are required. A series of image preprocessing procedures are implemented to remove noise in live-cell fluorescence images but yield overlapping cell nuclei. A segmentation method that evaluates boundary concavities and the size and shape of every nucleus is then used to determine if any of the strait, extrusion, convexity, large and local minimum diameter criteria satisfied to segment overlapping nuclei. We propose a dynamic data structure to record every newly segmented nucleus. The experimental results show that the proposed image preprocessing method preserves the areas of mouse ESC nuclei and that the segmentation method effectively detects overlapping nuclei. All segmented nuclei in confocal images can be accessed using the dynamic data structures to be visualised and manipulated for quantitative analyses of the ESC differentiation behaviours. The manipulation can be tracking of segmented 3D cell nuclei in time-lapse images to calculate their dynamics of differentiation characteristics.

Combining Deep Learning with Handcrafted Features for Cell Nuclei Segmentation.

Segmentation of cell nuclei in fluorescence microscopy images provides valuable information about the shape and size of the nuclei, its chromatin texture and DNA content. It has many applications such as cell tracking, counting and classification. In this work, we extended our recently proposed approach for nuclei segmentation based on deep learning, by adding to its input handcrafted features. Our handcrafted features introduce additional domain knowledge that nuclei are expected to have an approximately round shape. For round shapes the gradient vector of points at the border point to the center. To convey this information, we compute a map of gradient convergence to be used by the CNN as a new channel, in addition to the fluorescence microscopy image. We applied our method to a dataset of microscopy images of cells stained with DAPI. Our results show that with this approach we are able to decrease the number of missdetections and, therefore, increase the F1-Score when compared to our previously proposed approach. Moreover, the results show that faster convergence is obtained when handcrafted features are combined with deep learning.

Interphase Cell Cycle Staging using Deep Learning.

The progression of cells through the cell cycle is a tightly regulated process and is known to be key in maintaining normal tissue architecture and function. Disruption of these orchestrated phases will result in alterations that can lead to many diseases including cancer. Regrettably, reliable automatic tools to evaluate the cell cycle stage of individual cells are still lacking, in particular at interphase. Therefore, the development of new tools for a proper classification are urgently needed and will be of critical importance for cancer prognosis and predictive therapeutic purposes. Thus, in this work, we aimed to investigate three deep learning approaches for interphase cell cycle staging in microscopy images: 1) joint detection and cell cycle classification of nuclei patches; 2) detection of cell nuclei patches followed by classification of the cycle stage; 3) detection and segmentation of cell nuclei followed by classification of cell cycle staging. Our methods were applied to a dataset of microscopy images of nuclei stained with DAPI. The best results (0.908 F1-Score) were obtained with approach 3 in which the segmentation step allows for an intensity normalization that takes into account the intensities of all nuclei in a given image. These results show that for a correct cell cycle staging it is important to consider the relative intensities of the nuclei. Herein, we have developed a new deep learning method for interphase cell cycle staging at single cell level with potential implications in cancer prognosis and therapeutic strategies.

Hybrid adversarial-discriminative network for leukocyte classification in leukemia.

Leukemia is a lethal disease that is harmful to bone marrow and overall blood health. The classification of white blood cell images is crucial for leukemia diagnosis. The purpose of this study is to classify white blood cells by extracting discriminative information from cell segmentation and combining it with the fine-grained features. We propose a hybrid adversarial residual network with support vector machine (SVM), which utilizes the extracted features to improve the classification accuracy for human peripheral white cells. Firstly, we segment the cell and nucleus by utilizing an adversarial residual network, which contains a segmentation network and a discriminator network. To extract features that can handle the inter-class consistency problem effectively, we introduce the adversarial residual network. Then, we utilize convolutional neural network (CNN) features and histogram of oriented gradient (HOG) features, which can extract discriminative features from images of segmented cell nuclei. To utilize the representative features fully, a discriminative network is introduced to deal with neighboring information at different scales. Finally, we combine the vectors of HOG features with those of CNN features and feed them into a linear SVM to classify white blood cells into six types. We used three methods to evaluate the effect of leukocyte classification based on 5000 leukocyte images acquired from a local hospital. The first approach is to use the CNN features as the input of SVM to classify leukocytes, which achieved 94.23% specificity, 95.10% sensitivity, and 94.41% accuracy. The use of the HOG features for SVM achieved 83.50% specificity, 87.50% sensitivity, and 85.00% accuracy. The use of combined CNN and HOG features achieved 94.57% specificity, 96.11% sensitivity, and 95.93% accuracy. We propose a novel hybrid adversarial-discriminative network for the classification of microscopic leukocyte images. It improves the accuracy of cell classification, reduces the difficulty and time pressure of doctors' work, and economizes the valuable time of doctors in daily clinical diagnosis.

Three segmentation techniques to predict the dysplasia in cervical cells in the presence of debris

Cervical Cancer is one of the most pandemic causes of cancer related death in females. Pap smear test is one of the most commonly used screening test for the cervical cancer. Existing algorithms focus on the segmentation of nucleus and cytoplasm either using single-cell images or multiple cells images. Images captured from the Pap smear slides are called smear images. Smear image contains cervical cells along with debris, debris are inflammatory cells, red blood cells, dye. Debris significantly influence the outcome of image segmentation. An accurate nuclei segmentation method can improve the success rate of cervical cancer screening. Therefore, this paper reveals about three segmentation techniques which are used for automated segmentation of cervical cell nuclei in the presence of debris. Three segmentation techniques namely, Automated Seed Region Growing, Extended Edge Based Detection and Modified Moving k-means techniques are proposed to extract the cervical cell nuclei. These techniques are extracting the area of nuclei from smear images using the morphological property of nucleus. Some debris have area that corresponds with the area of nucleus of normal cells, it may interfere with outcome and may give false positive results. The empirical area threshold value demonstrate the superior performance of all proposed methods. The qualitative and quantitative analysis also done on proposed techniques. Experimental analysis shows that Modified Moving k-means give favorable result in dysplasia detection in the presence of debris. A new dataset PapsmearJP is collected during the study with the help of a pathologist for the validation of the work.

Semantic Segmentation of Cell Nuclei in Breast Cancer using Convolutional Neural Network

Semantic Segmentation of Cell Nuclei in Breast Cancer using Convolutional Neural Network

Fly-QMA: Automated analysis of mosaic imaginal discs in Drosophila.

Mosaic analysis provides a means to probe developmental processes in situ by generating loss-of-function mutants within otherwise wildtype tissues. Combining these techniques with quantitative microscopy enables researchers to rigorously compare RNA or protein expression across the resultant clones. However, visual inspection of mosaic tissues remains common in the literature because quantification demands considerable labor and computational expertise. Practitioners must segment cell membranes or cell nuclei from a tissue and annotate the clones before their data are suitable for analysis. Here, we introduce Fly-QMA, a computational framework that automates each of these tasks for confocal microscopy images of Drosophila imaginal discs. The framework includes an unsupervised annotation algorithm that incorporates spatial context to inform the genetic identity of each cell. We use a combination of real and synthetic validation data to survey the performance of the annotation algorithm across a broad range of conditions. By contributing our framework to the open-source software ecosystem, we aim to contribute to the current move toward automated quantitative analysis among developmental biologists.