Nuclei Segmentation Research Articles

Dynamic U-Net for multi-organ nucleus segmentation

Dynamic U-Net for multi-organ nucleus segmentation

Crns: CLIP-driven referring nuclei segmentation

Crns: CLIP-driven referring nuclei segmentation

Automatic Watershed Segmentation of Cancerous Lesions in Unsupervised Breast Histology Images

Segmentation of nuclei in histology images is key in analyzing and quantifying morphology changes of nuclei features and tissue structures. Conventional diagnosis, segmenting, and detection methods have relied heavily on the manual-visual inspection of histology images. These methods are only effective on clearly visible cancerous lesions on histology images thus limited in their performance due to the complexity of tissue structures in histology images. Hence, early detection of breast cancer is key for treatment and profits from Computer-Aided-Diagnostic (CAD) systems introduced to efficiently and automatically segment and detect nuclei cells in pathology. This paper proposes, an automatic watershed segmentation method of cancerous lesions in unsupervised human breast histology images. Firstly, this approach pre-processes data through various augmentation methods to increase the size of dataset images, then a stain normalization technique is applied to these augmented images to isolate nuclei features from tissue structures. Secondly, data enhancement techniques namely; erosion, dilation, and distance transform are used to highlight foreground and background pixels while removing unwanted regions from the highlighted nuclei objects on the image. Consequently, the connected components method groups these highlighted pixel components with similar intensity values and, assigns them to their relevant labeled component binary mask. Once all binary masked groups have been determined, a deep-learning recurrent neural network from the Keras architecture uses this information to automatically segment nuclei objects with cancerous lesions and their edges on the image via watershed filling. This segmentation method is evaluated on an unsupervised, augmented human breast cancer histology dataset of 11,151 images. This proposed method produced a significant evaluation result of 98% F1-accuracy score.

AER-Net: Attention-Enhanced Residual Refinement Network for Nuclei Segmentation and Classification in Histology Images

The acurate segmentation and classification of nuclei in histological images are crucial for the diagnosis and treatment of colorectal cancer. However, the aggregation of nuclei and intra-class variability in histology images present significant challenges for nuclei segmentation and classification. In addition, the imbalance of various nuclei classes exacerbates the difficulty of nuclei classification and segmentation using deep learning models. To address these challenges, we present a novel attention-enhanced residual refinement network (AER-Net), which consists of one encoder and three decoder branches that have same network structure. In addition to the nuclei instance segmentation branch and nuclei classification branch, one branch is used to predict the vertical and horizontal distance from each pixel to its nuclear center, which is combined with output by the segmentation branch to improve the final segmentation results. The AER-Net utilizes an attention-enhanced encoder module to focus on more valuable features. To further refine predictions and achieve more accurate results, an attention-enhancing residual refinement module is employed at the end of each encoder branch. Moreover, the coarse predictions and refined predictions are combined by using a loss function that employs cross-entropy loss and generalized dice loss to efficiently tackle the challenge of class imbalance among nuclei in histology images. Compared with other state-of-the-art methods on two colorectal cancer datasets and a pan-cancer dataset, AER-Net demonstrates outstanding performance, validating its effectiveness in nuclear segmentation and classification.

Enhanced Computer-Aided Digital Imaging Technique for Predictions in Breast Cancer

Background: Breast cancer (BRCA) is the most frequently diagnosed cancer in women, with a rise in occurrences and fatalities. The field of BRCA prediction and diagnosis has witnessed significant advancements in recent years, particularly emphasizing enhanced computer-aided digital imaging techniques, and has emerged as a powerful ally in the prediction of BRCA through histopathology image analysis. A number of approaches have been suggested in recent years for the categorization of histopathology BRCA images into benign and malignant as it examines the images at cellular level. The histopathology slides must be manually analysed which is time consuming and tiresome and is prone to human error. Additionally, different laboratories occasionally have different interpretation of these images. Methods: This paper focuses on implementing a framework for Computer-Aided digital imaging technique that can serve as a decision support. With recent advancements in computing power the analysis of BRCA histopathology image samples has become easier. Stain normalization (SN), segmentation, feature extraction and classification are the steps to categorize the cancer into benign and malignant. Nuclei segmentation is a crucial step that needs to be taken into account in order to establish malignancy. These are considered essential for early diagnosis of BRCA. A unique method proposed for BRCA prediction is put forward. To maximize the prediction accuracy, the suggested method is integrated with machine learning (ML) techniques and clinical data is used to evaluate the suggested approach. Results: This strategy is adaptable to many cancer types and imaging techniques. The suggested technique is applied to clinical data and is integrated with logistic regression and K-Nearest Neighbor resulting in accuracy of 92.10% and 86.89% respectively for BRCA histopathology images. Conclusion: The objective of this work is to validate the proposed model which takes input as feature pattern for a given label. For the collected clinical samples, the model is able to classify the input as benign or malignant. The proposed model worked efficiently for different BC datasets and performed classification task successfully. Integrating mathematical model (MM) with ML model for interpreting histopathology BRCA is a potential area of research in the field of digital pathology.

Unsupervised cervical cell instance segmentation method integrating cellular characteristics.

Cell instance segmentation is a key technology for cervical cancer auxiliary diagnosis systems. However, pixel-level annotation is time-consuming and labor-intensive, making it difficult to obtain a large amount of annotated data. This results in the model not being fully trained. In response to these problems, this paper proposes an unsupervised cervical cell instance segmentation method that integrates cell characteristics. Cervical cells have a clear corresponding structure between the nucleus and cytoplasm. This method fully takes this feature into account by building a dual-flow framework to locate the nucleus and cytoplasm and generate high-quality pseudo-labels. In the nucleus segmentation stage, the position and range of the nucleus are determined using the standard cell-restricted nucleus segmentation method. In the cytoplasm segmentation stage, a multi-angle collaborative segmentation method is used to achieve the positioning of the cytoplasm. First, taking advantage of the self-similarity characteristics of pixel blocks in cells, a cytoplasmic segmentation method based on self-similarity map iteration is proposed. The pixel blocks are mapped from the perspective of local details, and the iterative segmentation is repeated. Secondly, using low-level features such as cell color and shape, a self-supervised heatmap-aware cytoplasm segmentation method is proposed to obtain the activation map of the cytoplasm from the perspective of global attention. The two methods are fused to determine cytoplasmic regions, and combined with nuclear locations, high-quality pseudo-labels are generated. These pseudo-labels are used to train the model cyclically, and the loss strategy is used to encourage the model to discover new object masks, thereby obtaining a segmentation model with better performance. Experimental results show that this method achieves good results in cytoplasm segmentation. On the three datasets of ISBI, MS_CellSeg, and Cx22, 54.32%, 44.64%, and 66.52% AJI were obtained, respectively, which is better than other typical unsupervised methods selected in this article.

Segmentation and classification of white blood SMEAR images using modified CNN architecture

The classification and recognition of leukocytes or WBCs in blood smear images presents a key role in the corresponding diagnosis of specific diseases, such as leukemia, tumor, hematological disorders, etc. The computerized framework for automated segmentation & classification of WBCs nucleus contributes an important role for the recognition of WBCs related disorders. Therefore, this work emphasizes WBCs nucleus segmentation using modified U-Net architecture and the segmented WBCs nucleus are further classified into their subcategory i.e., basophil, eosinophil, neutrophil, monocyte and lymphocyte. The classification and nucleus characterization task has been performed using VGGNet and MobileNet V2 architecture. Initially, collected instances are passed to the preprocessing phase for image rescaling and normalization. The rescaled and normalized instances are passed to the U-Net model for nucleus segmentation. Extracted nucleus are forwarded to the classification phase for their class identifications. Furthermore, the functioning of the intended design will be compared with other modern methods. By the end of this study a successful model classifying various nucleus morphologies such as Basophil, Eosinophil, Lymphocyte, Monocyte and Neutrophil was obtained where overall test accuracy achieved was 97.0% for VGGNet classifier and 94.0% for MobileNet V2 classifier.

Keep It Accurate and Robust: An Enhanced Nuclei Analysis Framework

Accurate segmentation and classification of nuclei in histology images is critical but challenging due to nuclei heterogeneity, staining variations, and tissue complexity. Existing methods often struggle with limited dataset variability, with patches extracted from similar whole slide images (WSI), making models prone to falling into local optima. Here we propose a new framework to address this limitation and enable robust nuclear analysis. Our method leverages dual-level ensemble modeling to overcome issues stemming from limited dataset variation. Intra-ensembling applies diverse transformations to individual samples, while inter-ensembling combines networks of different scales. We also introduce enhancements to the HoVer-Net architecture, including updated encoders, nested dense decoding and model regularization strategy. We achieve state-of-the-art results on public benchmarks, including 1st place for nuclear composition prediction and 3rd place for segmentation/classification in the 2022 Colon Nuclei Identification and Counting (CoNIC) Challenge. This success validates our approach for accurate histological nuclei analysis. Extensive experiments and ablation studies provide insights into optimal network design choices and training techniques. In conclusion, this work proposes an improved framework advancing the state-of-the-art in nuclei analysis. We will release our code and models to serve as a toolkit for the community.

CSGO: A Deep Learning Pipeline for Whole-Cell Segmentation in Hematoxylin and Eosin Stained Tissues

Accurate whole-cell segmentation is essential in various biomedical applications, particularly in studying the tumor microenvironment (TME). Despite advancements in machine learning for nuclei segmentation in hematoxylin and eosin (H&E) stained images, there remains a need for effective whole-cell segmentation methods. This study aims to develop a deep learning-based pipeline to automatically segment cells in H&E-stained tissues, thereby advancing the capabilities of pathological image analysis. The Cell Segmentation with Globally Optimized boundaries (CSGO) framework integrates nuclei and membrane segmentation algorithms, followed by post-processing using an energy-based watershed method. Specifically, we employed the You Only Look Once (Yolo) object detection algorithm for nuclei segmentation and U-Net for membrane segmentation. The membrane detection model was trained on a dataset of 7 hepatocellular carcinomas and 11 normal liver tissue patches. The cell segmentation performance was extensively evaluated on five external datasets, including liver, lung, and oral disease cases. CSGO demonstrated superior performance over the state-of-the-art method Cellpose, achieving higher F1 scores ranging from 0.37 to 0.53 at an intersection over union (IoU) threshold of 0.5 in four of the five external datasets, compared to that of Cellpose from 0.21 to 0.36. These results underscore the robustness and accuracy of our approach in various tissue types. A web-based application is available at https://ai.swmed.edu/projects/csgo, providing a user-friendly platform for researchers to apply our method to their own datasets. Our method exhibits remarkable versatility in whole-cell segmentation across diverse cancer subtypes, serving as an accurate and reliable tool to facilitate TME studies. The advancements presented in this study have the potential to significantly enhance the precision and efficiency of pathological image analysis, contributing to better understanding and treatment of cancer.

A method of nucleus image segmentation and counting based on TC-UNet++ and distance watershed

Nucleus segmentation and counting play a crucial role in many cell analysis applications. However, the dense distribution and blurry boundaries of nucleus make nucleus segmentation tasks challenging. This paper proposes a novel segmentation and counting method. Firstly, TC-UNet++ is proposed to achieve a global segmentation. Then, the distance watershed method is used to finish local segmentation, which separate the adhesion and overlap part of the image. Finally, counting method is performed to obtain information on the counting number, area and center of mass of nucleus. TC-UNet++ achieved a Dice coefficient of 89.95% for cell instance segmentation on the Data Science Bowl dataset, surpassing the original U-Net++ by 0.23%. It also showed a 5.09% improvement in counting results compared to other methods. On the ALL-IDB dataset, TC-UNet++ reached a Dice coefficient of 83.97%, a 7.93% increase over the original U-Net++. Additionally, its counting results improved by 16.82% compared to other approaches. These results indicate that our method has a more complete and reasonable nucleus segmentation and counting scheme compared to other methods.

Histology-guided MRI segmentation of brainstem nuclei critical to consciousness.

While substantial progress has been made in mapping the connectivity of cortical networks responsible for conscious awareness, neuroimaging analysis of subcortical arousal networks that modulate arousal (i.e., wakefulness) has been limited by a lack of a robust segmentation procedures for brainstem arousal nuclei. Automated segmentation of brainstem arousal nuclei is an essential step toward elucidating the physiology of arousal in human consciousness and the pathophysiology of disorders of consciousness. We created a probabilistic atlas of brainstem arousal nuclei built on diffusion MRI scans of five ex vivo human brain specimens scanned at 750 μm isotropic resolution. Labels of arousal nuclei used to generate the probabilistic atlas were manually annotated with reference to nucleus-specific immunostaining in two of the five brain specimens. We then developed a Bayesian segmentation algorithm that utilizes the probabilistic atlas as a generative model and automatically identifies brainstem arousal nuclei in a resolution- and contrast-agnostic manner. The segmentation method displayed high accuracy in both healthy and lesioned in vivo T1 MRI scans and high test-retest reliability across both T1 and T2 MRI contrasts. Finally, we show that the segmentation algorithm can detect volumetric changes and differences in magnetic susceptibility within brainstem arousal nuclei in Alzheimer's disease and traumatic coma, respectively. We release the probabilistic atlas and Bayesian segmentation tool in FreeSurfer to advance the study of human consciousness and its disorders.

A roadmap towards standardized neuroimaging approaches for human thalamic nuclei.

The thalamus has a key role in mediating cortical-subcortical interactions but is often neglected in neuroimaging studies, which mostly focus on changes in cortical structure and activity. One of the main reasons for the thalamus being overlooked is that the delineation of individual thalamic nuclei via neuroimaging remains controversial. Indeed, neuroimaging atlases vary substantially regarding which thalamic nuclei are included and how their delineations were established. Here, we review current and emerging methods for thalamic nuclei segmentation in neuroimaging data and consider the limitations of existing techniques in terms of their research and clinical applicability. We address these challenges by proposing a roadmap to improve thalamic nuclei segmentation in human neuroimaging and, in turn, harmonize research approaches and advance clinical applications. We believe that a collective effort is required to achieve this. We hope that this will ultimately lead to the thalamic nuclei being regarded as key brain regions in their own right and not (as often currently assumed) as simply a gateway between cortical and subcortical regions.

Nuclear instance segmentation and tracking for preimplantation mouse embryos.

For investigations into fate specification and morphogenesis in time-lapse images of preimplantation embryos, automated 3D instance segmentation and tracking of nuclei are invaluable. Low signal-to-noise ratio, high voxel anisotropy, high nuclear density, and variable nuclear shapes can limit the performance of segmentation methods, while tracking is complicated by cell divisions, low frame rates, and sample movements. Supervised machine learning approaches can radically improve segmentation accuracy and enable easier tracking, but they often require large amounts of annotated 3D data. Here we first report a novel mouse line expressing near-infrared nuclear reporter H2B-miRFP720. We then generate a dataset (termed BlastoSPIM) of 3D images of H2B-miRFP720-expressing embryos with ground truth for nuclear instances. Using BlastoSPIM, we benchmark seven convolutional neural networks and identify Stardist-3D as the most accurate instance segmentation method. With our BlastoSPIM-trained Stardist-3D models, we construct a complete pipeline for nuclear instance segmentation and lineage tracking from the 8-cell stage to the end of preimplantation development (>100 nuclei). Finally, we demonstrate BlastoSPIM's usefulness as pre-train data for related problems, both for a different imaging modality and for different model systems.

Colorectal cancer histopathology image analysis: A comparative study of prognostic values of automatically extracted morphometric nuclear features in multispectral and red-blue-green imagery.

Multispectral imaging (MSI) has been utilized to predict the prognosis of colorectal cancer (CRC) patients, however, our understanding of the prognostic value of nuclear morphological parameters of bright-field MSI in CRC is still limited. This study was designed to compare the efficiency of MSI and standard red-green-blue (RGB) images in predicting the prognosis of CRC. We compared the efficiency of MS and conventional RGB images on the quantitative assessment of hematoxylin-eosin (HE) stained histopathology images. A pipeline was developed using a pixel-wise support vector machine (SVM) classifier for gland-stroma segmentation, and a marker-controlled watershed algorithm was used for nuclei segmentation. The correlation between extracted morphological parameters and the five-year disease-free survival (5-DFS) was analyzed. Forty-seven nuclear morphological parameters were extracted in total. Based on Kaplan-Meier analysis, eight features derived from MS images and seven featured derived from RGB images were significantly associated with 5-DFS, respectively. Compared with RGB images, MSI showed higher accuracy, precision, and Dice index in nuclei segmentation. Multivariate analysis indicated that both integrated parameters 1 (factors negatively correlated with CRC prognosis including nuclear number, circularity, eccentricity, major axis length) and 2 (factors positively correlated with CRC prognosis including nuclear average area, area perimeter, total area/total perimeter ratio, average area/perimeter ratio) in MS images were independent prognostic factors of 5-DFS, in contrast with only integrated parameter 1 (P<0.001) in RGB images. More importantly, the quantification of HE-stained MS images displayed higher accuracy in predicting 5-DFS compared with RGB images (76.9% vs 70.9%). Quantitative evaluation of HE-stained MS images could yield more information and better predictive performance for CRC prognosis than conventional RGB images, thereby contributing to precision oncology.

OrgaMapper: a robust and easy-to-use workflow for analyzing organelle positioning

BackgroundEukaryotic cells are highly compartmentalized by a variety of organelles that carry out specific cellular processes. The position of these organelles within the cell is elaborately regulated and vital for their function. For instance, the position of lysosomes relative to the nucleus controls their degradative capacity and is altered in pathophysiological conditions. The molecular components orchestrating the precise localization of organelles remain incompletely understood. A confounding factor in these studies is the fact that organelle positioning is surprisingly non-trivial to address e.g., perturbations that affect the localization of organelles often lead to secondary phenotypes such as changes in cell or organelle size. These phenotypes could potentially mask effects or lead to the identification of false positive hits. To uncover and test potential molecular components at scale, accurate and easy-to-use analysis tools are required that allow robust measurements of organelle positioning.ResultsHere, we present an analysis workflow for the faithful, robust, and quantitative analysis of organelle positioning phenotypes. Our workflow consists of an easy-to-use Fiji plugin and an R Shiny App. These tools enable users without background in image or data analysis to (1) segment single cells and nuclei and to detect organelles, (2) to measure cell size and the distance between detected organelles and the nucleus, (3) to measure intensities in the organelle channel plus one additional channel, (4) to measure radial intensity profiles of organellar markers, and (5) to plot the results in informative graphs. Using simulated data and immunofluorescent images of cells in which the function of known factors for lysosome positioning has been perturbed, we show that the workflow is robust against common problems for the accurate assessment of organelle positioning such as changes of cell shape and size, organelle size and background.ConclusionsOrgaMapper is a versatile, robust, and easy-to-use automated image analysis workflow that can be utilized in microscopy-based hypothesis testing and screens. It effectively allows for the mapping of the intracellular space and enables the discovery of novel regulators of organelle positioning.

Complex role of neutrophils in the tumor microenvironment: an avenue for novel immunotherapies.

Neutrophils, which originate from the bone marrow and are characterized by a segmented nucleus and a brief lifespan, have a crucial role in the body's defense against infections and acute inflammation. Recent research has uncovered the complex roles of neutrophils as regulators in tumorigenesis, during which neutrophils exhibit a dualistic nature that promotes or inhibits tumor progression. This adaptability is pivotal within the tumor microenvironment (TME). In this review, we provide a comprehensive characterization of neutrophil plasticity and heterogeneity, aiming to illuminate current research findings and discuss potential therapeutic avenues. By delineating the intricate interplay of neutrophils in the TME, this review further underscores the urgent need to understand the dual functions of neutrophils with particular emphasis on the anti-tumor effects to facilitate the development of effective therapeutic strategies against cancer.

Automated segmentation of midbrain nuclei using deep learning and multisequence MRI: a longitudinal study on iron accumulation with age

Abstract Elevated levels of brain iron, particularly within the basal ganglia, have been associated with cognitive and motor impairment in normal aging and neurodegenerative conditions. The subthalamic nucleus (STN), substantia nigra (SN), and red nucleus (RN), despite their high iron content and contribution to motor and cognitive processes, are less frequently studied. This oversight can largely be attributed to the challenges posed by in-vivo assessments of these small, deep-seated midbrain structures. We developed and validated an automated tool for the segmentation of the STN, SN, and RN. Multi-sequence magnetic resonance imaging (MRI) data including T1-weighted, FLAIR, Quantitative Susceptibility Mapping (QSM) and R2* alongside manual delineation on QSM images of 40 individuals, was used to train segmentation models based on nnU-Net deep-learning framework. A combination of QSM and FLAIR sequences was found to be optimal for structure segmentation (mean Dice scores of 0.84, 0.91 and 0.94 for STN, SN and RN, respectively). We next applied the automated segmentation method to an independent 3-year longitudinal dataset including 175 healthy adults (age range at baseline: 20-79 years old). Structural equation modelling was used to assess iron accumulation over time using age, sex, baseline iron and regional volume as factors of interest. Cross-sectionally, older age was linearly associated with higher iron load in SN and STN; the association was non-linear in RN. Longitudinally, results indicated significant iron accumulation in the STN (Mean increase = 0.02, p = 0.005) and SN (Mean increase = 0.035, p = 0.001), but not in the RN (Mean increase = 0.015, p = 0.2). Our findings demonstrated high performance of nnU-Net in automated segmentation, and advanced our understanding of iron accumulation in midbrain nuclei in aging.

Optimization of whole slide imaging scan settings for computer vision using human lung cancer tissue.

Digital pathology has become increasingly popular for research and clinical applications. Using high-quality microscopes to produce Whole Slide Images of tumor tissue enables the discovery of insights into biological aspects invisible to the human eye. These are acquired through downstream analyses using spatial statistics and artificial intelligence. Determination of the quality and consistency of these images is needed to ensure accurate outcomes when identifying clinical and subclinical image features. Additionally, the time-intensive process of generating high-volume images results in a trade-off that needs to be carefully balanced. This study aims to determine optimal instrument settings to generate representative images of pathological tissue using digital microscopy. Using various settings, an H&E stained sample was scanned using the ZEISS Axio Scan.Z1. Next, nucleus segmentation was performed on resulting images using StarDist. Subsequently, detections were compared between scans using a matching algorithm. Finally, nucleus-level information was compared between scans. Results indicated that while general matching percentages were high, similarity between information from replicates was relatively low. Additionally, settings resulting in longer scanning times and increased data volume did not increase similarity between replicates. In conclusion, the scan setting ultimately deemed optimal combined consistent and qualitative performance with low throughput time.

Label-free characterization of pathological changes in the portal area of liver fibrosis tissue using multiphoton imaging and quantitative image analysis

Abstract Liver fibrosis plays a crucial role in the progression of liver diseases and serves as a pivotal stage leading to the development of liver cirrhosis and cancer. It typically initiates from portal area with various pathological characteristics. In this article, we employed multiphoton microscopy (MPM) to characterize the pathological changes in the portal areas of liver fibrosis tissues, and subsequently, we used our developed image analysis method to extract eight collagen morphological features from MPM images and also combined a deep learning method with a cell nuclear feature extraction algorithm to perform automatic nuclei segmentation and quantitative analysis in the H&amp;E-stained histopathology images of portal areas. Our results demonstrate that MPM can effectively identify various pathological features in portal areas, and there are significant differences in four collagen features (collagen proportionate area, number, length and width) between normal and abnormal portal areas and in four nuclear features (mean axes distance ratio, disorder of distance to 3, 5 and 7 nearest neighbors) between normal portal area, bile duct hyperplasia and periductal fibrosis. Therefore, a combination of MPM and image-based quantitative analysis may be considered as a rapid and effective means to monitor histopathological changes in portal area and offer new insights into liver fibrosis.

Instance segmentation of cells and nuclei from multi-organ cross-protocol microscopic images.

Light microscopy is a widely used technique in cell biology due to its satisfactory resolution for cellular structure analysis, prevalent availability of fluorescent probes for staining, and compatibility for the dynamic analysis of live cells. However, the segmentation of cells and nuclei from microscopic images is not a straightforward process because it has several challenges such as high variation in morphology and shape, the presence of noise and diverse contrast in backgrounds, clustering or overlapping nature of cells. Dealing with these challenges and facilitating more reliable analysis necessitates the implementation of computer-aided methods that leverage image processing techniques and deep learning algorithms. The major goal of this study is to propose a model, for instance segmentation of cells and nuclei, applying the most cutting-edge deep learning techniques. A fine-tuned You Only Look at Once version 9 extended (YOLOv9-E) model is initially applied as a prompt generator to generate bounding box prompts. Using the generated prompts, a pre-trained segment anything model (SAM) is subsequently applied through zero-short inferencing to produce raw segmentation masks. These segmentation masks are then refined using non-max suppression and simple image processing methods such as image addition and morphological processing. The proposed method is developed and evaluated using an open-sourced dataset called Expert Visual Cell Annotation (EVICAN), which is relatively large and contains 4,738 microscopy images extracted from cross organs using different protocols. Based on the evaluation results on three different levels of EVICAN test sets, the proposed method demonstrates noticeable performances showing average mAP50 [mean average precision at intersection over union (IoU) =0.50] scores of 96.25, 95.05, and 94.18 for cell segmentation, and 68.04, 54.66, and 38.29 for nucleus segmentation on easy, medium, and difficult test sets, respectively. Our proposed method for instance segmentation of cells and nuclei provided favorable performance compared to the existing methods in literature, indicating its potential utility as an assistive tool for cell culture experts, facilitating prompt and reliable analysis.