Nuclei Segmentation Research Articles

High-throughput image processing software for the study of nuclear architecture and gene expression

High-throughput imaging (HTI) generates complex imaging datasets from a large number of experimental perturbations. Commercial HTI software programs for image analysis workflows typically do not allow full customization and adoption of new image processing algorithms in the analysis modules. While open-source HTI analysis platforms provide individual modules in the workflow, like nuclei segmentation, spot detection, or cell tracking, they are often limited in integrating novel analysis modules or algorithms. Here, we introduce the High-Throughput Image Processing Software (HiTIPS) to expand the range and customization of existing HTI analysis capabilities. HiTIPS incorporates advanced image processing and machine learning algorithms for automated cell and nuclei segmentation, spot signal detection, nucleus tracking, nucleus registration, spot tracking, and quantification of spot signal intensity. Furthermore, HiTIPS features a graphical user interface that is open to integration of new analysis modules for existing analysis pipelines and to adding new analysis modules. To demonstrate the utility of HiTIPS, we present three examples of image analysis workflows for high-throughput DNA FISH, immunofluorescence (IF), and live-cell imaging of transcription in single cells. Altogether, we demonstrate that HiTIPS is a user-friendly, flexible, and open-source HTI software platform for a variety of cell biology applications.

Augmented contour scoring snake for instance segmentation of placental separable villi

Segmenting terminal villous structures as separable instances is a prerequisite task for quantitative and explainable analysis of placental histopathology. Inspired by the fact that villous structure is typically surrounded by syncytiotrophoblast cells which yield critical hints for segmentation, we focus on designing a contour-based instance segmentation method. Previous contour-based methods usually utilize the confidence of object localization (or classification) as the instance score, without contour scores which explicitly measure the contour refinement quality (i.e., the distance discrepancy between the predicted contour and its ground truth). In this paper, we propose an augmented contour Scoring Snake framework, termed as SSnake, which learns both contour deformation and refinement quality estimation. To form contour quality measurement, we devise an axis-aware size-adaptive smooth function mapping from predicted contours to normalized scores in point resolution. Besides, to estimate scores of long-distance cases and strengthen the score learning process, a contour-augmented training scheme is designed for initial box contours. Our method recalibrates instance scores using contour quality by prioritizing instances with finer contours. Experimental results on the in-house separable villi segmentation dataset and a public cell nucleus segmentation dataset demonstrate that our proposed method significantly outperforms all competitors including state-of-the-art approaches. In villi segmentation dataset, our proposed SSnake achieves 3.7% improvements in COCO APm over the baseline DeepSnake. The source code is available at https://github.com/Psilym/SSnake.

AiSEGcell: User-friendly deep learning-based segmentation of nuclei in transmitted light images.

Segmentation is required to quantify cellular structures in microscopic images. This typically requires their fluorescent labeling. Convolutional neural networks (CNNs) can detect these structures also in only transmitted light images. This eliminates the need for transgenic or dye fluorescent labeling, frees up imaging channels, reduces phototoxicity and speeds up imaging. However, this approach currently requires optimized experimental conditions and computational specialists. Here, we introduce "aiSEGcell" a user-friendly CNN-based software to segment nuclei and cells in bright field images. We extensively evaluated it for nucleus segmentation in different primary cell types in 2D cultures from different imaging modalities in hand-curated published and novel imaging data sets. We provide this curated ground-truth data with 1.1 million nuclei in 20,000 images. aiSEGcell accurately segments nuclei from even challenging bright field images, very similar to manual segmentation. It retains biologically relevant information, e.g. for demanding quantification of noisy biosensors reporting signaling pathway activity dynamics. aiSEGcell is readily adaptable to new use cases with only 32 images required for retraining. aiSEGcell is accessible through both a command line, and a napari graphical user interface. It is agnostic to computational environments and does not require user expert coding experience.

A-ReSEUnet: Achieve no-label binary segmentation of nuclei in histology images

Histopathological images reveal a lot of information about tissue health. The task of nucleus segmentation can be applied to more diverse studies. The potential of deep learning has been demonstrated for the task of nuclei segmentation. However, the cost and time required for labeling data, as well as the lack of annotators, makes it difficult to keep up with the continuous generation of new pathological images. This paper proposes a method that combines attention mechanisms and self-supervised learning to train a model for nuclei segmentation on unlabeled data. The method is mainly divided into two stages. After the initial preprocessing of the image, the enhanced image of the nucleus area can be obtained in the first stage. Subsequently, the original image and the augmented image needed for self-supervised learning undergo feature extraction in the second stage. The results of this study show that the proposed method can achieve dice scores exceeding 0.7 in the nucleus binary segmentation task without labels. Fine-tuning the model for downstream tasks can also achieve the effect of F1 score 0.7 multi-category segmentation. Using the same experimental setting, the method proposed in this study achieves a dice score of 0.779 on the MoNuSeg dataset, which is only 0.03 lower than the highest dice score of 0.809 achieved by the compared supervised learning methods. In comparison to other unsupervised models, this method demonstrates superior segmentation performance. Moreover, this study applies the training weights to other histopathology image datasets, enabling the generation of the same binary segmentation effect.© 2017 Elsevier Inc. All rights reserved.

The influence of heavy metals on cytotoxicity in Tilapia zillii

The present study was designed to investigate the cytotoxic effects and bioaccumulation of heavy metals iron (Fe), zinc (Zn), copper (Cu), cadmium (Cd), lead (Pb), manganese (Mn), nickel (Ni), and chromium (Cr) in different parts (muscle, gills, and liver) of Tilapia zillii occurring in polluted drainage canal and fish farm, which is located in Abiece region in front of village number 10, Alexandria governorate, Egypt. Results of water analysis revealed the concentration of Cd, Pb, Mn, Ni, and Cr exceeded the limits defined by the American Public Health Association (APHA) in the polluted drainage canal. In addition, the concentration of Ni elevated to the standard limits of APHA and Cu was not detected in the fish farm. Different types of chromosomal aberrations were recorded (e.g., stickiness, fragmented chromosomes, centromeric gaps, chromatid break, chromatid deletion, and tetraploid). Micronucleus frequency was found to be 5.58 in the polluted drainage canal group and 0.32 in the fish farm group. Other nuclear abnormalities such as blebbed nucleus, segmented nucleus, enucleated erythrocyte, kidney-shaped nucleus, heart-shaped nucleus, polymorphic irregular nuclei, binucleated cell, nuclear fragmented erythrocyte, long nucleus, putative fragmented notched nucleus, lobed nuclei, fused erythrocytes, necrotic erythrocyte, and vacuolated nucleus were recorded. The total of erythrocytes nuclear morphological abnormalities was 70.33% in the polluted drainage canal and 1.78% in the fish farm.

Bend-Net: Bending Loss Regularized Multitask Learning Network for Nuclei Segmentation in Histopathology Images

Bend-Net: Bending Loss Regularized Multitask Learning Network for Nuclei Segmentation in Histopathology Images

A deep learning-based toolkit for 3D nuclei segmentation and quantitative analysis in cellular and tissue context.

We present a new set of computational tools that enable accurate and widely applicable 3D segmentation of nuclei in various 3D digital organs. We have developed an approach for ground truth generation and iterative training of 3D nuclear segmentation models, which we applied to popular CellPose, PlantSeg and StarDist algorithms. We provide two high-quality models trained on plant nuclei that enable 3D segmentation of nuclei in datasets obtained from fixed or live samples, acquired from different plant and animal tissues, and stained with various nuclear stains or fluorescent protein-based nuclear reporters. We also share a diverse high-quality training dataset of about 10,000 nuclei. Furthermore, we advanced the MorphoGraphX analysis and visualization software by, among other things, providing a method for linking 3D segmented nuclei to their surrounding cells in 3D digital organs. We found that the nuclear-to-cell volume ratio varies between different ovule tissues and during the development of a tissue. Finally, we extended the PlantSeg 3D segmentation pipeline with a proofreading tool that uses 3D segmented nuclei as seeds to correct cell segmentation errors in difficult-to-segment tissues.

Multi-headed U-Net: an automated nuclei segmentation technique using Tikhonov filter-based unsharp masking

ABSTRACT An automated nuclei segmentation is the key technique for understanding and analyzing cellular properties, which are helpful for disease diagnosis and support computer-aided digital pathology. However, this task is challenging because of the variability in nuclei size and morphology, which results in blurry boundaries and overlapping nuclei. To address such issues, a multi-headed U-Net convolutional neural network (CNN) architecture has been proposed. This architecture has multiple heads to extract multi-resolution features of the source image by using different kernel sizes of the filters. The source images are pre-processed using an unsharp masking approach based on the Tikhonov filter. The Tikhonov filter decomposes the input image into low-frequency and high-frequency band images. The unsharp masking method improves the high-frequency information of the input image by primarily enhancing features such as boundaries, contours, and fine details. We have incorporated intersection over union (IOU) and F1Score as measures along with accuracy for our proposed objective functions. The proposed objective functions are tried to be maximized by the optimization algorithm, and the higher value of the metrics indicates better segmentation performance in the spatial domain during the testing phase. The proposed method attained IOU(JI), Accuracy, Precision, and F1Score values as 0.8299, 0.9642, 0.8918, and 0.9070, respectively. The quantitative and qualitative experimental outcomes indicate that our proposed technique outperforms the state-of-the-art techniques.

Projected pooling loss for red nucleus segmentation with soft topology constraints.

Deep learning is the standard for medical image segmentation. However, it may encounter difficulties when the training set is small. Also, it may generate anatomically aberrant segmentations. Anatomical knowledge can be potentially useful as a constraint in deep learning segmentation methods. We propose a loss function based on projected pooling to introduce soft topological constraints. Our main application is the segmentation of the red nucleus from quantitative susceptibility mapping (QSM) which is of interest in parkinsonian syndromes. This new loss function introduces soft constraints on the topology by magnifying small parts of the structure to segment to avoid that they are discarded in the segmentation process. To that purpose, we use projection of the structure onto the three planes and then use a series of MaxPooling operations with increasing kernel sizes. These operations are performed both for the ground truth and the prediction and the difference is computed to obtain the loss function. As a result, it can reduce topological errors as well as defects in the structure boundary. The approach is easy to implement and computationally efficient. When applied to the segmentation of the red nucleus from QSM data, the approach led to a very high accuracy (Dice 89.9%) and no topological errors. Moreover, the proposed loss function improved the Dice accuracy over the baseline when the training set was small. We also studied three tasks from the medical segmentation decathlon challenge (MSD) (heart, spleen, and hippocampus). For the MSD tasks, the Dice accuracies were similar for both approaches but the topological errors were reduced. We propose an effective method to automatically segment the red nucleus which is based on a new loss for introducing topology constraints in deep learning segmentation.

Automatic cancer nuclei segmentation on histological images: comparison study of deep learning methods

Automatic cancer nuclei segmentation on histological images: comparison study of deep learning methods

Utilizing a Pathomics Biomarker to Predict the Effectiveness of Bevacizumab in Ovarian Cancer Treatment.

The purpose of this investigation is to develop and initially assess a quantitative image analysis scheme that utilizes histopathological images to predict the treatment effectiveness of bevacizumab therapy in ovarian cancer patients. As a widely accessible diagnostic tool, histopathological slides contain copious information regarding underlying tumor progression that is associated with tumor prognosis. However, this information cannot be readily identified by conventional visual examination. This study utilizes novel pathomics technology to quantify this meaningful information for treatment effectiveness prediction. Accordingly, a total of 9828 features were extracted from segmented tumor tissue, cell nuclei, and cell cytoplasm, which were categorized into geometric, intensity, texture, and subcellular structure features. Next, the best performing features were selected as the input for SVM (support vector machine)-based prediction models. These models were evaluated on an open dataset containing a total of 78 patients and 288 whole slides images. The results indicated that the sufficiently optimized, best-performing model yielded an area under the receiver operating characteristic (ROC) curve of 0.8312. When examining the best model's confusion matrix, 37 and 25 cases were correctly predicted as responders and non-responders, respectively, achieving an overall accuracy of 0.7848. This investigation initially validated the feasibility of utilizing pathomics techniques to predict tumor responses to chemotherapy at an early stage.

Unveiling nuclear chromatin distribution using IsoConcentraChromJ: A flourescence imaging plugin for IsoRegional and IsoVolumetric based ratios analysis.

Chromatin exhibits non-random distribution within the nucleus being arranged into discrete domains that are spatially organized throughout the nuclear space. Both the spatial distribution and structural rearrangement of chromatin domains in the nucleus depend on epigenetic modifications of DNA and/or histones and structural elements such as the nuclear envelope. These components collectively contribute to the organization and rearrangement of chromatin domains, thereby influencing genome architecture and functional regulation. This study develops an innovative, user-friendly, ImageJ-based plugin, called IsoConcentraChromJ, aimed quantitatively delineating the spatial distribution of chromatin regions in concentric patterns. The IsoConcentraChromJ can be applied to quantitative chromatin analysis in both two- and three-dimensional spaces. After DNA and histone staining with fluorescent probes, high-resolution images of nuclei have been obtained using advanced fluorescence microscopy approaches, including confocal and stimulated emission depletion (STED) microscopy. IsoConcentraChromJ workflow comprises the following sequential steps: nucleus segmentation, thresholding, masking, normalization, and trisection with specified ratios for either 2D or 3D acquisitions. The effectiveness of the IsoConcentraChromJ has been validated and demonstrated using experimental datasets consisting in nuclei images of pre-adipocytes and mature adipocytes, encompassing both 2D and 3D imaging. The outcomes allow to characterize the nuclear architecture by calculating the ratios between specific concentric nuclear areas/volumes of acetylated chromatin with respect to total acetylated chromatin and/or total DNA. The novel IsoConcentrapChromJ plugin could represent a valuable resource for researchers investigating the rearrangement of chromatin architecture driven by epigenetic mechanisms using nuclear images obtained by different fluorescence microscopy methods.

Applicability of non-invasive and live-cell holotomographic imaging on fungi

The ability to acquire three-dimensional (3D) information of cellular structures without the need for fluorescent tags or staining makes holotomographic imaging a powerful tool in cellular biology. It provides valuable insights by measuring the refractive index (RI), an optical parameter describing the phase delay of light that passes through the living cell.Here, we demonstrate holotomographic imaging on industrial relevant ascomycete fungi and study their development and morphogenesis. This includes conidial germination, subcellular dynamics, and cytoplasmic flow during hyphal growth in Aspergillus niger. In addition, growth and budding of Aureobasidium pullulans cells are captured using holotomographic microscopy. Coupled to fluorescence imaging, lipid droplets, vacuoles, the mitochondrial network, and nuclei are targeted and analyzed in the 3D RI reconstructed images. While lipid droplets and vacuoles can be assigned to a specific RI pattern, mitochondria and nuclei were not pronounced. We show, that the lower sensitivity of RI measurements derives from the fungal cell wall that acts as an additional barrier for the illumination light of the microscope. After cell wall digest of hyphae and protoplast formation of A. niger expressing GFP-tagged histone H2A, location of nuclei could be determined by non-invasive RI measurements. Furthermore, we used coupled fluorescence microscopy to observe migration of nuclei in unperturbed hyphal segments and duplication during growth on a single-cell level. Detailed micromorphological studies in Saccharomyces cerevisiae and Trichoderma reesei are challenging due to cell size restrictions.Overall, holotomography opens up new avenues for exploring dynamic cellular processes in real time and enables the visualization of fungi from a new perspective.

A fully automated and explainable algorithm for predicting malignant transformation in oral epithelial dysplasia

Oral epithelial dysplasia (OED) is a premalignant histopathological diagnosis given to lesions of the oral cavity. Its grading suffers from significant inter-/intra-observer variability, and does not reliably predict malignancy progression, potentially leading to suboptimal treatment decisions. To address this, we developed an artificial intelligence (AI) algorithm, that assigns an Oral Malignant Transformation (OMT) risk score based on the Haematoxylin and Eosin (H&E) stained whole slide images (WSIs). Our AI pipeline leverages an in-house segmentation model to detect and segment both nuclei and epithelium. Subsequently, a shallow neural network utilises interpretable morphological and spatial features, emulating histological markers, to predict progression. We conducted internal cross-validation on our development cohort (Sheffield; n = 193 cases) and independent validation on two external cohorts (Birmingham and Belfast; n = 89 cases). On external validation, the proposed OMTscore achieved an AUROC = 0.75 (Recall = 0.92) in predicting OED progression, outperforming other grading systems (Binary: AUROC = 0.72, Recall = 0.85). Survival analyses showed the prognostic value of our OMTscore (C-index = 0.60, p = 0.02), compared to WHO (C-index = 0.64, p = 0.003) and binary grades (C-index = 0.65, p < 0.001). Nuclear analyses elucidated the presence of peri-epithelial and intra-epithelial lymphocytes in highly predictive patches of transforming cases (p < 0.001). This is the first study to propose a completely automated, explainable, and externally validated algorithm for predicting OED transformation. Our algorithm shows comparable-to-human-level performance, offering a promising solution to the challenges of grading OED in routine clinical practice.

TshFNA-Examiner: A Nuclei Segmentation and Cancer Assessment Framework for Thyroid Cytology Image

TshFNA-Examiner: A Nuclei Segmentation and Cancer Assessment Framework for Thyroid Cytology Image

Investigating the Possible Ameliorating Impact of Black Seed (Nigella sativa) Hydroethanolic Extract on Liver and Brain Tissue in a Rat Model of Polycystic Ovary

Background: Polycystic ovary syndrome (PCOS) is the most frequent endocrine disorder in women of reproductive age, which may affect various organs, including the brain and liver. Objectives: This study aimed to investigate the potential ameliorating effects of black seed (Nigella sativa) hydroethanolic extract on the liver and brain tissues in a rat model of polycystic ovary. Methods: In this experimental study, eighteen female rats were divided into three groups: control (PCO rats on day 0 after induction of PCO), vehicle (PCO rats receiving saline via gavage for 28 days), and black seed extract (BSE) (PCO rats receiving BSE via gavage for 28 days). Polycystic ovary was induced by a single intraperitoneal injection of 2 mg/kg estradiol valerate. Liver tissue was evaluated for the average number of damaged hepatocytes (segmented nucleus and cytoplasm shrinkage), number of Kupffer cells, and number of lipid droplets, while brain tissue was assessed for vacuolization, apoptosis, and cell shrinkage. Results: In the BSE group, liver weight significantly decreased, while brain weight increased compared to the vehicle and control groups. Moreover, there was a significant decrease in the number of damaged hepatocytes and Kupffer cells in the BSE group compared to the vehicle group in the liver tissue. In the brain tissue, there was a significant reduction in the number of vacuolations in the BSE group compared to the vehicle group. Conclusions: Black seed extract demonstrated potential ameliorating effects on the adverse impacts of PCO in the liver (reducing damaged hepatocytes and Kupffer cells) and brain (reducing vacuolations). Further histologic and molecular analyses are needed to investigate the protective effects of BSE thoroughly.

WaveSeg‐UNet model for overlapped nuclei segmentation from multi‐organ histopathology images

AbstractNuclei segmentation is a challenging task in histopathology images. It is challenging due to the small size of objects, low contrast, touching boundaries, and complex structure of nuclei. Their segmentation and counting play an important role in cancer identification and its grading. In this study, WaveSeg‐UNet, a lightweight model, is introduced to segment cancerous nuclei having touching boundaries. Residual blocks are used for feature extraction. Only one feature extractor block is used in each level of the encoder and decoder. Normally, images degrade quality and lose important information during down‐sampling. To overcome this loss, discrete wavelet transform (DWT) alongside max‐pooling is used in the down‐sampling process. Inverse DWT is used to regenerate original images during up‐sampling. In the bottleneck of the proposed model, atrous spatial channel pyramid pooling (ASCPP) is used to extract effective high‐level features. The ASCPP is the modified pyramid pooling having atrous layers to increase the area of the receptive field. Spatial and channel‐based attention are used to focus on the location and class of the identified objects. Finally, watershed transform is used as a post processing technique to identify and refine touching boundaries of nuclei. Nuclei are identified and counted to facilitate pathologists. The same domain of transfer learning is used to retrain the model for domain adaptability. Results of the proposed model are compared with state‐of‐the‐art models, and it outperformed the existing studies.

BoNuS: Boundary Mining for Nuclei Segmentation With Partial Point Labels.

Nuclei segmentation is a fundamental prerequisite in the digital pathology workflow. The development of automated methods for nuclei segmentation enables quantitative analysis of the wide existence and large variances in nuclei morphometry in histopathology images. However, manual annotation of tens of thousands of nuclei is tedious and time-consuming, which requires significant amount of human effort and domain-specific expertise. To alleviate this problem, in this paper, we propose a weakly-supervised nuclei segmentation method that only requires partial point labels of nuclei. Specifically, we propose a novel boundary mining framework for nuclei segmentation, named BoNuS, which simultaneously learns nuclei interior and boundary information from the point labels. To achieve this goal, we propose a novel boundary mining loss, which guides the model to learn the boundary information by exploring the pairwise pixel affinity in a multiple-instance learning manner. Then, we consider a more challenging problem, i.e., partial point label, where we propose a nuclei detection module with curriculum learning to detect the missing nuclei with prior morphological knowledge. The proposed method is validated on three public datasets, MoNuSeg, CPM, and CoNIC datasets. Experimental results demonstrate the superior performance of our method to the state-of-the-art weakly-supervised nuclei segmentation methods. Code: https://github.com/hust-linyi/bonus.

Contrastive and uncertainty-aware nuclei segmentation and classification

Nuclei segmentation and classification play a crucial role in pathology diagnosis, enabling pathologists to analyze cellular characteristics accurately. Overlapping cluster nuclei, misdetection of small-scale nuclei, and pleomorphic nuclei-induced misclassification have always been major challenges in the nuclei segmentation and classification tasks. To this end, we introduce an auxiliary task of nuclei boundary-guided contrastive learning to enhance the representativeness and discriminative power of visual features, particularly for addressing the challenge posed by the unclear contours of adherent nuclei and small nuclei. In addition, misclassifications resulting from pleomorphic nuclei often exhibit low classification confidence, indicating a high level of uncertainty. To mitigate misclassification, we capitalize on the characteristic clustering of similar cells to propose a locality-aware class embedding module, offering a regional perspective to capture category information. Moreover, we address uncertain classification in densely aggregated nuclei by designing a top-k uncertainty attention module that leverages deep features to enhance shallow features, thereby improving the learning of contextual semantic information. We demonstrate that the proposed network outperforms the off-the-shelf methods in both nuclei segmentation and classification experiments, achieving the state-of-the-art performance.

Use of artificial intelligence–based digital pathology to predict outcomes for immune checkpoint inhibitor therapy in advanced gastro-esophageal cancer.

4013 Background: Accurate prediction of response and survival outcomes with anti-PD-1/PD-L1 immune checkpoint inhibition (ICI) remains a significant challenge in gastro-esophageal cancers. In this study, we use an artificial intelligence (AI)-based single-cell analysis of digitized whole-slide H&amp;E images (WSIs) to predict objective response and survival benefit of ICI in two independent cohorts of gastro-esophageal cancer patients. Methods: WSIs were obtained from 82 ICI-treated advanced gastroesophageal cancer patients (gastric, esophageal, and gastro-esophageal junction adenocarcinoma and squamous cell carcinoma) at Stanford University for discovery, and 189 ICI-treated advanced gastric adenocarcinoma patients at Southern Medical University for external validation. Deep learning models were deployed for automated tumor area detection and segmentation of cell nuclei within WSIs. We developed a fully automated cell annotation approach by leveraging multiplex immunofluorescence and trained a deep learning model to classify nuclei into four cell types from H&amp;E (tumor cells, lymphocytes, neutrophils, and macrophages). A total of 66 features were computed to quantify cell composition and cell-cell interactions within the tumor microenvironment. Treatment outcomes were assessed using progression-free survival (PFS) and best objective response per the Response Evaluation Criteria in Solid Tumors (v1.1), with statistical significance reported at the 95% confidence level. Results: In the training cohort, a spatial feature characterizing lymphocyte and neutrophil interaction had the strongest association with PFS (hazard ratio = 0.44, 95% CI 0.24-0.79, P = 0.0046). In the validation cohort, the spatial biomarker positive population had significantly improved PFS compared to the spatial biomarker negative population (hazard ratio = 0.41, 95% CI 0.27-0.61, P &lt; 0.0001; median time to event: 19 months vs 9 months). For predicting objective response in the validation cohort, a multivariate model combining the spatial features achieved AUROC = 0.81 compared to AUROC = 0.65 for PD-L1 CPS (P = 0.0014), while combining the multivariate model with PD-L1 CPS achieved AUROC = 0.84. Conclusions: A single-cell computational pathology approachidentifies spatial biomarkers with predictive utility for determining ICI treatment outcomes in advanced gastro-esophageal cancer. Further validation of these findings is being pursued in additional cohorts.