Nuclear Segmentation Research Articles

AER-Net: Attention-Enhanced Residual Refinement Network for Nuclei Segmentation and Classification in Histology Images

The acurate segmentation and classification of nuclei in histological images are crucial for the diagnosis and treatment of colorectal cancer. However, the aggregation of nuclei and intra-class variability in histology images present significant challenges for nuclei segmentation and classification. In addition, the imbalance of various nuclei classes exacerbates the difficulty of nuclei classification and segmentation using deep learning models. To address these challenges, we present a novel attention-enhanced residual refinement network (AER-Net), which consists of one encoder and three decoder branches that have same network structure. In addition to the nuclei instance segmentation branch and nuclei classification branch, one branch is used to predict the vertical and horizontal distance from each pixel to its nuclear center, which is combined with output by the segmentation branch to improve the final segmentation results. The AER-Net utilizes an attention-enhanced encoder module to focus on more valuable features. To further refine predictions and achieve more accurate results, an attention-enhancing residual refinement module is employed at the end of each encoder branch. Moreover, the coarse predictions and refined predictions are combined by using a loss function that employs cross-entropy loss and generalized dice loss to efficiently tackle the challenge of class imbalance among nuclei in histology images. Compared with other state-of-the-art methods on two colorectal cancer datasets and a pan-cancer dataset, AER-Net demonstrates outstanding performance, validating its effectiveness in nuclear segmentation and classification.

Nuclear instance segmentation and tracking for preimplantation mouse embryos.

For investigations into fate specification and morphogenesis in time-lapse images of preimplantation embryos, automated 3D instance segmentation and tracking of nuclei are invaluable. Low signal-to-noise ratio, high voxel anisotropy, high nuclear density, and variable nuclear shapes can limit the performance of segmentation methods, while tracking is complicated by cell divisions, low frame rates, and sample movements. Supervised machine learning approaches can radically improve segmentation accuracy and enable easier tracking, but they often require large amounts of annotated 3D data. Here we first report a novel mouse line expressing near-infrared nuclear reporter H2B-miRFP720. We then generate a dataset (termed BlastoSPIM) of 3D images of H2B-miRFP720-expressing embryos with ground truth for nuclear instances. Using BlastoSPIM, we benchmark seven convolutional neural networks and identify Stardist-3D as the most accurate instance segmentation method. With our BlastoSPIM-trained Stardist-3D models, we construct a complete pipeline for nuclear instance segmentation and lineage tracking from the 8-cell stage to the end of preimplantation development (>100 nuclei). Finally, we demonstrate BlastoSPIM's usefulness as pre-train data for related problems, both for a different imaging modality and for different model systems.

HoverFast: an accurate, high-throughput, clinically deployable nuclear segmentation tool for brightfield digital pathology images

HoverFast: an accurate, high-throughput, clinically deployable nuclear segmentation tool for brightfield digital pathology images

Regionally Adaptive Active Learning Framework for Nuclear Segmentation in Microscopy Image

Regionally Adaptive Active Learning Framework for Nuclear Segmentation in Microscopy Image

Automated analysis of scattering-based light sheet microscopy images of anal squamous intraepithelial lesions.

We developed an algorithm for automatically analyzing scattering-based light sheet microscopy (sLSM) images of anal squamous intraepithelial lesions. We developed a method for automatically segmenting sLSM images for nuclei and calculating seven features: nuclear intensity, intensity slope as a function of depth, nuclear-to-nuclear distance, nuclear-to-cytoplasm ratio, cell density, nuclear area, and proportion of pixels corresponding to nuclei. 187 images from 80 anal biopsies were used for feature analysis and classifier development. The automated nuclear segmentation method provided reliable performance with the precision of 0.97 and recall of 0.91 when compared with the manual segmentation. Among the seven features, six showed statistically significant differences between high-grade squamous intraepithelial lesion (HSIL) and non-HSIL (non-dysplastic or low-grade squamous intraepithelial lesion, LSIL). A classifier using linear support vector machine (SVM) achieved promising performance in diagnosing HSIL versus non-HSIL: sensitivity of 90%, specificity of 70%, and area under the curve (AUC) of 0.89 for per-image diagnosis, and sensitivity of 90%, specificity of 80%, and AUC of 0.92 for per-biopsy diagnosis.

Deep learning-based aberration compensation improves contrast and resolution in fluorescence microscopy.

Optical aberrations hinder fluorescence microscopy of thick samples, reducing image signal, contrast, and resolution. Here we introduce a deep learning-based strategy for aberration compensation, improving image quality without slowing image acquisition, applying additional dose, or introducing more optics into the imaging path. Our method (i) introduces synthetic aberrations to images acquired on the shallow side of image stacks, making them resemble those acquired deeper into the volume and (ii) trains neural networks to reverse the effect of these aberrations. We use simulations and experiments to show that applying the trained 'de-aberration' networks outperforms alternative methods, providing restoration on par with adaptive optics techniques; and subsequently apply the networks to diverse datasets captured with confocal, light-sheet, multi-photon, and super-resolution microscopy. In all cases, the improved quality of the restored data facilitates qualitative image inspection and improves downstream image quantitation, including orientational analysis of blood vessels in mouse tissue and improved membrane and nuclear segmentation in C. elegans embryos.

A deep learning-based toolkit for 3D nuclei segmentation and quantitative analysis in cellular and tissue context.

We present a new set of computational tools that enable accurate and widely applicable 3D segmentation of nuclei in various 3D digital organs. We have developed an approach for ground truth generation and iterative training of 3D nuclear segmentation models, which we applied to popular CellPose, PlantSeg and StarDist algorithms. We provide two high-quality models trained on plant nuclei that enable 3D segmentation of nuclei in datasets obtained from fixed or live samples, acquired from different plant and animal tissues, and stained with various nuclear stains or fluorescent protein-based nuclear reporters. We also share a diverse high-quality training dataset of about 10,000 nuclei. Furthermore, we advanced the MorphoGraphX analysis and visualization software by, among other things, providing a method for linking 3D segmented nuclei to their surrounding cells in 3D digital organs. We found that the nuclear-to-cell volume ratio varies between different ovule tissues and during the development of a tissue. Finally, we extended the PlantSeg 3D segmentation pipeline with a proofreading tool that uses 3D segmented nuclei as seeds to correct cell segmentation errors in difficult-to-segment tissues.

ConfluentFUCCI for fully-automated analysis of cell-cycle progression in a highly dense collective of migrating cells.

Understanding mechanisms underlying various physiological and pathological processes often requires accurate and fully automated analysis of dense cell populations that collectively migrate. In such multicellular systems, there is a rising interest in the relations between biophysical and cell cycle progression aspects. A seminal tool that led to a leap in real-time study of cell cycle is the fluorescent ubiquitination-based cell cycle indicator (FUCCI). Here, we introduce ConfluentFUCCI, an open-source graphical user interface-based framework that is designed, unlike previous tools, for fully automated analysis of cell cycle progression, cellular dynamics, and cellular morphology, in highly dense migrating cell collectives. We integrated into ConfluentFUCCI's pipeline state-of-the-art tools such as Cellpose, TrackMate, and Napari, some of which incorporate deep learning, and we wrap the entire tool into an isolated computational environment termed container. This provides an easy installation and workflow that is independent of any specific operation system. ConfluentFUCCI offers accurate nuclear segmentation and tracking using FUCCI tags, enabling comprehensive investigation of cell cycle progression at both the tissue and single-cell levels. We compare ConfluentFUCCI to the most recent relevant tool, showcasing its accuracy and efficiency in handling large datasets. Furthermore, we demonstrate the ability of ConfluentFUCCI to monitor cell cycle transitions, dynamics, and morphology within densely packed epithelial cell populations, enabling insights into mechanotransductive regulation of cell cycle progression. The presented tool provides a robust approach for investigating cell cycle-related phenomena in complex biological systems, offering potential applications in cancer research and other fields.

Automated Nuclear Morphometry: A Deep Learning Approach for Prognostication in Canine Pulmonary Carcinoma to Enhance Reproducibility.

The integration of deep learning-based tools into diagnostic workflows is increasingly prevalent due to their efficiency and reproducibility in various settings. We investigated the utility of automated nuclear morphometry for assessing nuclear pleomorphism (NP), a criterion of malignancy in the current grading system in canine pulmonary carcinoma (cPC), and its prognostic implications. We developed a deep learning-based algorithm for evaluating NP (variation in size, i.e., anisokaryosis and/or shape) using a segmentation model. Its performance was evaluated on 46 cPC cases with comprehensive follow-up data regarding its accuracy in nuclear segmentation and its prognostic ability. Its assessment of NP was compared to manual morphometry and established prognostic tests (pathologists' NP estimates (n = 11), mitotic count, histological grading, and TNM-stage). The standard deviation (SD) of the nuclear area, indicative of anisokaryosis, exhibited good discriminatory ability for tumor-specific survival, with an area under the curve (AUC) of 0.80 and a hazard ratio (HR) of 3.38. The algorithm achieved values comparable to manual morphometry. In contrast, the pathologists' estimates of anisokaryosis resulted in HR values ranging from 0.86 to 34.8, with slight inter-observer reproducibility (k = 0.204). Other conventional tests had no significant prognostic value in our study cohort. Fully automated morphometry promises a time-efficient and reproducible assessment of NP with a high prognostic value. Further refinement of the algorithm, particularly to address undersegmentation, and application to a larger study population are required.

A Pipeline for Evaluation of Machine Learning/Artificial Intelligence Models to Quantify Programmed Death Ligand 1 Immunohistochemistry

Immunohistochemistry (IHC) is used to guide treatment decisions in multiple cancer types. For treatment with checkpoint inhibitors, programmed death ligand 1 (PD-L1) IHC is used as a companion diagnostic. However, the scoring of PD-L1 is complicated by its expression in cancer and immune cells. Separation of cancer and noncancer regions is needed to calculate tumor proportion scores (TPS) of PD-L1, which is based on the percentage of PD-L1-positive cancer cells. Evaluation of PD-L1 expression requires highly experienced pathologists and is often challenging and time-consuming. Here, we used a multi-institutional cohort of 77 lung cancer cases stained centrally with the PD-L1 22C3 clone. We developed a 4-step pipeline for measuring TPS that includes the coregistration of hematoxylin and eosin, PD-L1, and negative control (NC) digital slides for exclusion of necrosis, segmentation of cancer regions, and quantification of PD-L1+ cells. As cancer segmentation is a challenging step for TPS generation, we trained DeepLab V3 in the Visiopharm software package to outline cancer regions in PD-L1 and NC images and evaluated the model performance by mean intersection over union (mIoU) against manual outlines. Only 14 cases were required to accomplish a mIoU of 0.82 for cancer segmentation in hematoxylin-stained NC cases. For PD-L1-stained slides, a model trained on PD-L1 tiles augmented by registered NC tiles achieved a mIoU of 0.79. In segmented cancer regions from whole slide images, the digital TPS achieved an accuracy of 75% against the manual TPS scores from the pathology report. Major reasons for algorithmic inaccuracies include the inclusion of immune cells in cancer outlines and poor nuclear segmentation of cancer cells. Our transparent and stepwise approach and performance metrics can be applied to any IHC assay to provide pathologists with important insights on when to apply and how to evaluate commercial automated IHC scoring systems.

Abstract 5506: HOST-Factor: A comprehensive platform for single-cell highplex immunofluorescence staining, digital imaging, spatial mapping, and quantitative analysis of the tumor microenvironment

Abstract Solid tumor complexity extends beyond the genetic and functional landscapes of heterogenous cancer cells, encompassing the tumor microenvironment (TME). Elucidating the TME’s complexity requires a comprehensive assessment of its cellular composition, functional states, and spatial distributions. We developed the Harmonic Output of Stromal Traits (HOST) to identify TME cells, and the HOST-Factor to quantify their functional states. The HOST-Factor is a numerical value that reflects the relative contribution of cancer-associated fibroblasts (CAFs) to tumor-suppressive or tumor-promoting functions. Our workflow combines automated cycling highplex immunofluorescent microscopy with artificial intelligence (AI)-guided image analysis. This generates HOST-Factor values for each identified TME cell within selected regions of interest, providing spatial distribution data. The TME signature encompasses 15 immune cells and 14 CAF antibody-detected biomarkers. We applied our workflow to ten human pancreatic cancer specimens, generating OME-TIFF output images. This cancer model was used due to its significant TME makeup. The 29 highplex AI-based digital image analysis was conducted using the Phenoplex™ workflow from Visiopharm. The workflow included deep-learning-based tissue morphologic and cellular segmentations, cellular phenotyping, and integration of spatial/location data. Biomarker subsets were visualized, and a user-trained algorithm was used for tissue segmentation. Nuclear segmentation was done using a pre-trained algorithm on a DAPI-labeled DNA channel. Cellular phenotyping was performed using thresholds based on visual assessment and confirmation of positivity. Spatial neighborhood plots and metrics, heatmaps and partitioned t-SNE plots were generated for the dataset for downstream analysis. Importantly, the workflow's visualization templates, pre-trained nuclear/cytoplasm segmentation tools, and neighborhood plots and metrics, are reusable and fully customizable for new datasets. Using HOST-Factor values, we successfully classified cancer and TME cells, along with their functional states, and spatial distributions. This AI-based computational approach and user-friendly workflow provides a simple and effective way to obtain single-cell information from multiplex immunofluorescence images, making spatial phenotyping of several cell populations in tissues more accessible to researchers, providing a fully amendable means for future clinical translation. Citation Format: Janusz Franco-Barraza, Fabian Schneider, Rasmus Norré-Sørensen, Rasmus Ahrenkiel-Lyngby, Caneta Brown, Dan Winkowski, James Robert Mansfield, Edna Cukierman. HOST-Factor: A comprehensive platform for single-cell highplex immunofluorescence staining, digital imaging, spatial mapping, and quantitative analysis of the tumor microenvironment [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 5506.

Abstract 6178: Automating tumor purity assessment: Advanced AI-driven segmentation and classification of nuclei in H&E stained slides

Abstract Background Tumor purity, the proportion of cancerous cells within a tumor mass, is a pivotal factor in oncological research and clinical practice and plays a significant role in the molecular testing workflow, particularly when selecting samples for genomic analysis. Accurate assessment of tumor purity is crucial for understanding tumor behavior, guiding treatment decisions, and predicting patient outcomes. Traditional methods of analyzing hematoxylin and eosin (H&E) stained slides for tumor purity are predominantly pathologist manual readouts. This process is both time-consuming and prone to inter- and intra-observer variability, leading to inconsistencies in diagnosis and treatment planning. Here, we introduce an automated AI-based product designed to streamline tumor purity analysis. The RevealAI-HE digital assay leverages state-of-the-art AI algorithms to automate the segmentation and classification of nuclei in H&E slides, enhancing the accuracy, consistency, and speed of tumor purity assessment. RevealAI-HE aims to assist pathologists in decision-making, support oncology research, streamline molecular testing workflows, and improve personalized care. Methods H&E stained solid tumor slides were imaged using 3D-Histech Pannoramic 1000 and 250 scanners. From this dataset, 104 regions of interest (ROIs) were manually selected to cover a wide range of H&E stain intensities in 14 different indications. A nuclear segmentation model based on StarDist-H&E was deployed on these ROIs, and a pathology expert manually assigned one of three classifications to each nuclei: tumor, immune, and stroma. The resulting ground-truth dataset of over 26,000 classified nuclei was used for training a classifier model based on a ResNet architecture. For model evaluation, the nuclei dataset was randomly split into training (80%) and testing (20%) the model, and the training data was batched and randomized for each training epoch to ensure robustness and generalizability. The nuclear segmentation, nucleus classifier, and tumor purity statistics models were integrated into an end-to-end digital assay (RevealAI-HE) using Reveal Biosciences’ proprietary digital assay development pipeline. RevealAI-HE is integrated into Reveal Biosciences’ flagship whole slide image management system, ImageDx. Results The RevealAI-HE digital assay’s nucleus classification model achieved a high accuracy across all 14 indications. Compared to manual methods, the AI system significantly reduced the time required for tumor purity analysis. Furthermore, the AI model exhibited excellent consistency in its analyses, addressing the issue of variability seen in manual assessments. These results indicate that our AI product, in many aspects, surpasses the current gold standard of tumor purity analysis (pathologist readout), offering a more efficient, accurate, and reproducible approach. Citation Format: Nikolaus R. Wagner, Kaito Kikuchi, Kristen Fukuda, Nikola Spasic, Stacy Littlechild, Sinisa Todorovic, Dragana Stojnev, Masa Jovanovic, Claire Weston. Automating tumor purity assessment: Advanced AI-driven segmentation and classification of nuclei in H&E stained slides [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 6178.

Abstract 4896: A high throughput method of identifying cells with increased genomic content across diverse cancer tissues

Abstract Cancer cells with increased genomic content have been associated with recurrence and poor patient outcomes. The current manual identification of these rare cells by pathologists is time-consuming and labor intensive, given their scarcity within tissue samples. This study introduces a robust methodology for the automated identification of cancer cells with increased genomic content within prostate cancer tissues. Leveraging Qupath software integrated with the Stardist extension, our approach combines precise nuclear segmentation with threshold setting using a Python script. This methodology enables the specific identification of noncancerous small cells, cancerous cells, and those exhibiting genomic content significantly surpassing the average cancer cell (3x to 5x), complemented by thresholds based on hematoxylin intensity for enhanced result accuracy. Thus, our methodology offers an efficient solution to streamline the identification processes for these challenging cells. We have extensively validated our method across diverse models, including animal and human models, transgenic and subcutaneous models, and Human Tissue Microarrays (TMAs), to identify cancer cells with increased genomic content. The rarity of these cells within tissue samples underscores the practical significance of our research, providing a solution to the current difficulties faced by researchers and clinicians in their identification. The integration of this methodology aims to improve efficiency in cancer tissue analysis. Ongoing efforts are directed towards the evaluation of this high-throughput methodology using animal models and TMAs with known outcomes, further substantiating its applicability and potential clinical relevance. Citation Format: Shilpa Priyadarsini Nair, Kenneth J. Pienta, Sarah R. Amend. A high throughput method of identifying cells with increased genomic content across diverse cancer tissues [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4896.

Abstract 4725: Phenotypic single-cell analysis of nuclear stress responses in drug-treated prostate cancer cells

Abstract Chemotherapy with anti-cancer drugs aims to stop the growth of cancer cells either by killing them or by stopping them from dividing. The different specificities in inducing cellular damage responses are incompletely understood currently and may underlie the preferential effectiveness of anti-cancer drugs against different types of cancer cells. Better understanding and tools to measure drug-induced phenotypes in vitro may aid in assessing cellular heterogeneity of responses, determining potential effectiveness, estimating amount and types of adverse effects, and predicting likelihood of surviving populations for residual disease. To measure the types and extent of nuclear stress responses to cancer drugs, we studied nuclear organelle, namely nucleolar and Cajal body, phenotypes to drug-induced stress in prostate cancer cells. We utilized organelle-specific markers NPM, Fibrillarin, Coilin, and SMN1 in immunofluorescence staining to determine suitable markers for nuclear organelle stress. We used three different prostate cancer cells lines representing different biological types of advanced prostate cancer (AR positive, AR negative, and expressing AR transcript variant). Based on initial assessments, NPM and coilin immunostaining were selected for automated phenotypic analysis screening for drug effects. Using AI-assisted nuclear segmentation and feature-based single-cell phenotypic analysis, we demonstrated quantitative effects on nuclear morphology, nucleoli, and Cajal bodies with drug treatments. A proof-of-principle study with platinum-based chemotherapeutic agents in clinical use demonstrated heterologous responses to the platinum drugs that are associated with prostate cancer cell types with different AR status. Further, we have defined the corresponding quantitative responses for androgen deprivation therapy (ADT) drugs enzalutamide and bicalutamide in AR-positive prostate cancer cells. Our findings evidence variability of prostate cancer cells’ response to chemotherapy and have implications in future therapy regimen considerations. Furthermore, we have developed a fluorescence light microscopy-based method to assess nuclear stress response phenotypes in prostate cancer cells that can be used for high throughput drug screening. Citation Format: Enkhzaya Batnasan, Minttu Kärkkäinen, Sonja Koivukoski, Pekka Ruusuvuori, Leena Latonen. Phenotypic single-cell analysis of nuclear stress responses in drug-treated prostate cancer cells [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4725.

Targeting SRSF2 mutations in leukemia with RKI-1447: A strategy to impair cellular division and nuclear structure

Spliceosome machinery mutations are common early mutations in myeloid malignancies; however, effective targeted therapies against them are still lacking. In the current study, we used an invitro high-throughput drug screen among four different isogenic cell lines and identified RKI-1447, a Rho-associated protein kinase inhibitor, as selective cytotoxic effector of SRSF2 mutant cells. RKI-1447 targeted SRSF2 mutated primary human samples in xenografts models. RKI-1447 induced mitotic catastrophe and induced major reorganization of the microtubule system and severe nuclear deformation. Transmission electron microscopy and 3D light microscopy revealed that SRSF2 mutations induce deep nuclear indentation and segmentation that are apparently driven by microtubule-rich cytoplasmic intrusions, which are exacerbated by RKI-1447. The severe nuclear deformation in RKI-1447-treated SRSF2 mutant cells prevents cells from completing mitosis. These findings shed new light on the interplay between microtubules and the nucleus and offers new ways for targeting pre-leukemic SRSF2 mutant cells.

High-volume, label-free imaging for quantifying single-cell dynamics in induced pluripotent stem cell colonies.

To facilitate the characterization of unlabeled induced pluripotent stem cells (iPSCs) during culture and expansion, we developed an AI pipeline for nuclear segmentation and mitosis detection from phase contrast images of individual cells within iPSC colonies. The analysis uses a 2D convolutional neural network (U-Net) plus a 3D U-Net applied on time lapse images to detect and segment nuclei, mitotic events, and daughter nuclei to enable tracking of large numbers of individual cells over long times in culture. The analysis uses fluorescence data to train models for segmenting nuclei in phase contrast images. The use of classical image processing routines to segment fluorescent nuclei precludes the need for manual annotation. We optimize and evaluate the accuracy of automated annotation to assure the reliability of the training. The model is generalizable in that it performs well on different datasets with an average F1 score of 0.94, on cells at different densities, and on cells from different pluripotent cell lines. The method allows us to assess, in a non-invasive manner, rates of mitosis and cell division which serve as indicators of cell state and cell health. We assess these parameters in up to hundreds of thousands of cells in culture for more than 36 hours, at different locations in the colonies, and as a function of excitation light exposure.

HistoEM: A Pathologist-Guided and Explainable Workflow Using Histogram Embedding for Gland Classification

Pathologists have, over several decades, developed criteria for diagnosing and grading prostate cancer. However, this knowledge has not, so far, been included in the design of convolutional neural networks (CNN) for prostate cancer detection and grading. Further, it is not known whether the features learned by machine-learning algorithms coincide with diagnostic features used by pathologists. We propose a framework that enforces algorithms to learn the cellular and subcellular differences between benign and cancerous prostate glands in digital slides from hematoxylin and eosin–stained tissue sections. After accurate gland segmentation and exclusion of the stroma, the central component of the pipeline, named HistoEM, utilizes a histogram embedding of features from the latent space of the CNN encoder. Each gland is represented by 128 feature-wise histograms that provide the input into a second network for benign vs cancer classification of the whole gland. Cancer glands are further processed by a U-Net structured network to separate low-grade from high-grade cancer. Our model demonstrates similar performance compared with other state-of-the-art prostate cancer grading models with gland-level resolution. To understand the features learned by HistoEM, we first rank features based on the distance between benign and cancer histograms and visualize the tissue origins of the 2 most important features. A heatmap of pixel activation by each feature is generated using Grad-CAM and overlaid on nuclear segmentation outlines. We conclude that HistoEM, similar to pathologists, uses nuclear features for the detection of prostate cancer. Altogether, this novel approach can be broadly deployed to visualize computer-learned features in histopathology images.

Abstract A032: Cellos: High-throughput deconvolution of 3D organoid dynamics at cellular resolution for cancer pharmacology

Abstract Three-dimensional (3D) culture models, such as organoids, are flexible systems to interrogate cellular evolution, cellular growth and morphology, multicellular spatial architecture, and cell interactions in response to drug treatment. However, new computational methods to segment and analyze 3D models at cellular resolution with sufficiently high throughput are needed to realize these possibilities. Here we report Cellos (Cell and Organoid Segmentation), an accurate, high throughput image analysis pipeline for 3D organoid and nuclear segmentation analysis. Cellos segments organoids in 3D using classical algorithms and segments nuclei using a Stardist-3D convolutional neural network which we trained on manually annotated dataset of 3,862 cells. To evaluate the capabilities of Cellos we then analyzed 74,450 organoids with 1.65 million cells, from multiple experiments on triple negative breast cancer organoids containing clonal mixtures with complex cisplatin sensitivies. Cellos was able to accurately distinguish ratios of distinct fluorescently labeled cell populations in organoids, with < 3% deviation from seeding ratios in each well and was effective for both fluorescently labelled nuclei and independent Hoechst stained datasets. Cellos was able to recapitulate traditional luminescence-based drug response quantification by analyzing 3D images, including parallel analysis of multiple cancer clones in the same well. Moreover, Cellos was able to identify organoid and nuclei morphology features associated with treatment and unique to each of the clones. Finally, Cellos enables 3D analysis of cell spatial relationships, which we used to detect ecological affinity between cancer clones beyond what arises from local cell division and organoid composition. Cellos provides powerful tools to perform high throughput analysis for pharmacological testing and biological investigation of organoids based on 3D imaging. Citation Format: Patience Mukashyaka, Pooja Kumar, David J. Mellert, Shadae Nicholas, Javad Noorbakhsh, Mattia Brugiolo, Olga Anczukow, Edison T. Liu, Jeffrey H. Chuang. Cellos: High-throughput deconvolution of 3D organoid dynamics at cellular resolution for cancer pharmacology [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: Translating Cancer Evolution and Data Science: The Next Frontier; 2023 Dec 3-6; Boston, Massachusetts. Philadelphia (PA): AACR; Cancer Res 2024;84(3 Suppl_2):Abstract nr A032.

Attention guided multi-scale cluster refinement with extended field of view for amodal nuclei segmentation

Nuclei segmentation plays a crucial role in disease understanding and diagnosis. In whole slide images, cell nuclei often appear overlapping and densely packed with ambiguous boundaries due to the underlying 3D structure of histopathology samples. Instance segmentation via deep neural networks with object clustering is able to detect individual segments in crowded nuclei but suffers from a limited field of view, and does not support amodal segmentation. In this work, we introduce a dense feature pyramid network with a feature mixing module to increase the field of view of the segmentation model while keeping pixel-level details. We also improve the model output quality by adding a multi-scale self-attention guided refinement module that sequentially adjusts predictions as resolution increases. Finally, we enable clusters to share pixels by separating the instance clustering objective function from other pixel-related tasks, and introduce supervision to occluded areas to guide the learning process. For evaluation of amodal nuclear segmentation, we also update prior metrics used in common modal segmentation to allow the evaluation of overlapping masks and mitigate over-penalization issues via a novel unique matching algorithm. Our experiments demonstrate consistent performance across multiple datasets with significantly improved segmentation quality.

Content-aware frame interpolation (CAFI): deep learning-based temporal super-resolution for fast bioimaging

The development of high-resolution microscopes has made it possible to investigate cellular processes in 3D and over time. However, observing fast cellular dynamics remains challenging because of photobleaching and phototoxicity. Here we report the implementation of two content-aware frame interpolation (CAFI) deep learning networks, Zooming SlowMo and Depth-Aware Video Frame Interpolation, that are highly suited for accurately predicting images in between image pairs, therefore improving the temporal resolution of image series post-acquisition. We show that CAFI is capable of understanding the motion context of biological structures and can perform better than standard interpolation methods. We benchmark CAFI’s performance on 12 different datasets, obtained from four different microscopy modalities, and demonstrate its capabilities for single-particle tracking and nuclear segmentation. CAFI potentially allows for reduced light exposure and phototoxicity on the sample for improved long-term live-cell imaging. The models and the training and testing data are available via the ZeroCostDL4Mic platform.