Abstract

BackgroundNuclear segmentation is an important step for profiling aberrant regions of histology sections. If nuclear segmentation can be resolved, then new biomarkers of nuclear phenotypes and their organization can be predicted for the application of precision medicine. However, segmentation is a complex problem as a result of variations in nuclear geometry (e.g., size, shape), nuclear type (e.g., epithelial, fibroblast), nuclear phenotypes (e.g., vesicular, aneuploidy), and overlapping nuclei. The problem is further complicated as a result of variations in sample preparation (e.g., fixation, staining). Our hypothesis is that (i) deep learning techniques can learn complex phenotypic signatures that rise in tumor sections, and (ii) fusion of different representations (e.g., regions, boundaries) contributes to improved nuclear segmentation.ResultsWe have demonstrated that training of deep encoder-decoder convolutional networks overcomes complexities associated with multiple nuclear phenotypes, where we evaluate alternative architecture of deep learning for an improved performance against the simplicity of the design. In addition, improved nuclear segmentation is achieved by color decomposition and combining region- and boundary-based features through a fusion network. The trained models have been evaluated against approximately 19,000 manually annotated nuclei, and object-level Precision, Recall, F1-score and Standard Error are reported with the best F1-score being 0.91. Raw training images, annotated images, processed images, and source codes are released as a part of the Additional file 1.ConclusionsThere are two intrinsic barriers in nuclear segmentation to H&E stained images, which correspond to the diversity of nuclear phenotypes and perceptual boundaries between adjacent cells. We demonstrate that (i) the encoder-decoder architecture can learn complex phenotypes that include the vesicular type; (ii) delineation of overlapping nuclei is enhanced by fusion of region- and edge-based networks; (iii) fusion of ENets produces an improved result over the fusion of UNets; and (iv) fusion of networks is better than multitask learning. We suggest that our protocol enables processing a large cohort of whole slide images for applications in precision medicine.

Highlights

  • Nuclear segmentation is an important step for profiling aberrant regions of histology sections

  • Technical variations refer to non-uniformity in sample preparations, and biological heterogeneity refers to the fact that no two histology sections are alike

  • Technical variations are coupled with biological heterogeneity, which complicates the construction of a stable computational model for nuclear segmentation

Read more

Summary

Introduction

Nuclear segmentation is an important step for profiling aberrant regions of histology sections. (i) cancer cells tend to be larger than normal cells, and if coupled with high chromatin content, Khoshdeli et al BMC Bioinformatics (2018) 19:294 they may indicate aneuploidy; (ii) nuclei may have vesicular phenotypes; (iii) nuclei may have high pleomorphism in tumor sections; (iv) cells may be going through apoptosis or necrosis; (v) cell cytoplasm may be lost as a result of clear cell carcinoma; and (vi) cellular phenotypes may be altered as a result of macromolecules being secreted into the microenvironment We show that simultaneous delineation of vesicular and other phenotypes can be achieved with fusion of the deep learning models

Methods
Results
Conclusion
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call