Abstract PR-05: Leveraging graphs to do novel hypothesis and data-driven research using multiplex immunofluorescence images

Background: Eosinophils are innate immune granulocytes that migrate to areas of inflammation to combat against infection and disease. Best known for their detrimental role in asthma and allergic disorders, there is growing interest in the involvement of eosinophils in cancer. Eosinophils are routinely observed in the tumor microenvironment (TME) and, depending on the cancer type, have been shown to drive other immune cells to either suppress or promote tumor growth - In colorectal cancer (CRC), eosinophil infiltration into the TME has been linked to a favorable prognosis. However, the behavior of eosinophils and their effect on associated immune mediators in the TME remains poorly understood. Currently, eosinophils are primarily identified from H&E stained tissue sections based on morphological features by a pathologist, but it can be challenging to reliably and efficiently identify all eosinophils based on morphology alone. Here, we present a novel image analysis workflow that established an AI-based cell classifier which can accurately quantify eosinophils in H&E stained CRC tissue sections by leveraging biomarker staining of eosinophils using multiplex immunofluorescence (mIF) imaging in conjunction with the morphological characteristics of eosinophils observed by H&E. Methods: CRC tissue sections were labeled with a 5plex panel of eosinophil-specific markers and imaged by mIF using the PhenoImager HT platform (Akoya). Biomarker fluorescence was then quenched, sections stained by H&E, and reimaged using the PhenoImager. mIF and H&E images were then imported into the HALO® platform for algorithm development. Halo AI cell classification of eosinophils was trained on the morphological features of eosinophil staining in H&E images, guided by eosinophil-specific labeling in the 5plex mIF images. H&E staining was also performed on a serial section of each specimen. Morphological identification of eosinophils in H&E images was performed by a pathologist. Results: Correlation analysis was performed to evaluate the relationship, per high power field (HPF), between manual eosinophil counts by a pathologist and AI algorithm derived eosinophils counts. The results showed the two methods are highly correlated, demonstrating reliable algorithm performance. Conclusion: The AI-based eosinophil detection algorithm established here enables high-throughput analysis and quantification of eosinophils from H&E stained CRC tissue specimens and facilitates morphology diagnosis. Citation Format: Arezoo Hanifi, Elizabeth Blain, James Hargrove, Jeff Lock, Nam Tran, Vladislav Chizhevsky, Qingyan Au. Development of an AI-based algorithm to quantify eosinophils in H&E images from colorectal cancer (CRC) tissue sections guided by biomarker staining using multiplex immunofluorescence imaging [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 2352.

3020 Background: Understanding the underlying heterogeneity of the tumor microenvironment (TME) on a single-cell level is becoming increasingly important to predict a patient’s response to immunotherapy. Conventional imaging methods can help reveal tissue heterogeneity, but are not optimal for identifying multiple cellular subpopulations or cellular interactions from a single slide image, limiting their use in clinical settings. Here, we present a clinical artificial intelligence (AI)-driven multiplex immunofluorescence (MxIF) imaging pipeline based on novel cell segmentation and cell typing methods to evaluate tumor cellular heterogeneity, immune cell composition, and cell-to-cell interactions. Methods: A machine learning (ML)-based cell segmentation algorithm was trained on a manually annotated dataset created from 219 different regions of interest (ROIs) that contained 85,991 cells from various tissues (colon, kidney, lung, lymph node, tonsil, and ureter). A dataset containing 58,676 cells from 146 ROIs was used for validation and accuracy was determined between automated and manually annotated images; accuracy was further evaluated by calculating the f1-score using available methods (DeepCell and Stardist). Marker stains with a low signal-to-noise ratio were automatically enhanced, allowing for adequate cell-to-cell interaction analysis. Results: An automated MxIF image processing workflow was developed. Validation of the trained cell segmentation model showed high accuracy (0.80 f1-score), demonstrating superior performance compared to other methods (DeepCell and Stardist - 0.55 and 0.78 f1-score, respectively). The pathologist-determined accuracy (0.84 mean f1-score) indicated a near-human performance of the developed method. Normalized expression values obtained from the cell typing model allowed automated cell recognition. We analyzed cellular heterogeneity across 3 regions of colorectal cancer (CRC), gastric cancer (GC), and non-small cell lung cancer (NSCLC) samples. While proportions of immune cells varied, proportions of malignant epithelial cells were stable across all regions of each sample, as concordant percentages of Ki67+ cells were identified (CRC-19%; GC-21%; NSCLC-5%). Analysis of cell-to-cell interactions and immune communities identified tumor-, immune-, and stromal-enriched communities in all tumor samples that were stable across regions. Conclusions: By analyzing complex tumor tissue at single-cell resolution with high accuracy, this AI-driven MxIF imaging technology is able to characterize tumor and microenvironment heterogeneity across cancer types. This novel AI-based tool is currently being integrated into several ongoing prospective clinical studies to aid in the development of predictive and prognostic biomarkers.

Measuring CD8-Positive T-Cells in the Tumor Microenvironment in Classical Hodgkin Lymphoma Using Multiplex Immunofluorescence Staining and Image Analysis: A Possible Prognostic Factor

BackgroundCell morphology, the study of cellular form and structure, provides valuable insights into the diverse characteristics and functions of different cell types. Cell morphology is also indicative of cellular activities...

Development of an automated combined positive score prediction pipeline using artificial intelligence on multiplexed immunofluorescence images

Multiplexing in immunofluorescence imaging is important for the spatial profiling of cells and molecules in tumor tissue samples. Cyclic immunofluorescence (IF) methods using oxidants (e.g. hydrogen peroxide) and enzymes (e.g. DNase) localize a great number of cellular makers and proteins in a tissue section while repeating a process of IF staining, imaging, and fluorescence deactivation. However, the repeated use of chemicals and enzymes might cause artifacts in tissue and cell morphologies. Furthermore, these methods are restricted to thin tissue sections (~5 μm thick) which are inappropriate to provide comprehensive structural information on tissue samples. Although reconstruction of two-dimensional (2D) images from serial tissue sections can provide a certain volumetric tissue image, it takes a huge amount of time and effort. Here we introduce a three-dimensional (3D) multiplex IF imaging method using LED photobleaching. We built high-power LED illuminators with 100W warm (emission wavelength: 480-700 nm), green (430-520 nm), and red (600- 680 nm) LED chips, which can efficiently bleach a broad or selected wavelength of fluorescence signals in tissue samples. We integrated this LED photobleaching with the Transparent Tissue Tomography (T3) protocol and created a 3D cyclic IF method involving tissue macrosectioning (400 μm), three-color IF staining, D-fructose-based tissue clearing, 3D confocal fluorescence microscopy, LED photobleaching, tissue washing, and three-color IF staining for other biomarkers, and repeating the process. By applying this method to mouse mammary tumor tissues, we could perform 8-plex fluorescence microscopy for visualizing cell nuclei (DAPI), vascular (CD31, SMA) and structural (ER-TR7) cells, immune cells (CD3, CD8, CD45), and cancer cells (CK8) in the tumor macrosections in 3D at tissue and cellular resolution. To validate the method as an evaluation tool for immunotherapy, we treated the mouse mammary tumor with a STING agonist (DMAXX) intratumorally and collected the tumor tissue 1 day after the treatment, and processed it for the 3D cyclic IF protocol. The quantitative multiplex image data showed immune-driven-cancer eradication and high tumor infiltration of a large number of CD3+CD8+CD45+ cytotoxic T cells. We also examined that Red and Green LED illumination can selectively bleach fluorophores in tissues, which would be useful for patterning fluorescence in tissue as well as studying fluorescent drug-cell interaction in a tissue. In summary, this chemical and enzyme-free 3D cyclic IF imaging method will be a powerful tissue assay tool to provide comprehensive spatial information of tissue (tumor) samples including cell types, cellular and molecular location, and their 3D organization in a tissue sample. Citation Format: Jingtian Zheng, Evan Phillips, Yi-Chien Wu, Steve Seung-Young Lee, Vytautas Bindokas. LED photobleaching-based multiplex 3D microscopy of the tumor microenvironment. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 4707.

BackgroundMultiplexed imaging technologies enable the comprehensive examination of tumor tissue at the cellular level, while preserving spatial details. However, gaining a deep understanding of complex tumor tissue organization and dynamic...

BackgroundSingle-cell spatial phenotyping using multiplex immunofluorescence (MxIF) imaging has emerged as a promising method to evaluate the heterogeneity of the tumor microenvironment (TME) for immunotherapy response prediction. Here, we use...

Immune profiling is becoming a vital tool for identifying predictive and prognostic markers for translational studies. The study of the tumor microenvironment (TME) in paraffin tumor tissues such as malignant pleural mesothelioma (MPM) could yield insights to actionable targets to improve patient outcome. Here, we optimized and tested a new immune-profiling method to characterize immune cell phenotypes in paraffin tissues and explore the co-localization and spatial distribution between the immune cells within the TME and the stromal or tumor compartments. Tonsil tissues and tissue microarray (TMA) were used to optimize an automated nine-color multiplex immunofluorescence (mIF) panel to study the TME using eight antibodies: PD-L1, PD-1, CD3, CD8, Foxp3, CD68, KI67, and pancytokeratin. To explore the potential role of the cells into the TME with this mIF panel we applied this panel in twelve MPM cases to assess the multiple cell phenotypes obtained from the image analysis and well as their spatial distribution in this cohort. We successful optimized and applied an automated nine-color mIF panel to explore a small set of MPM cases. Image analysis showed a high degree of cell phenotype diversity with immunosuppression patterns in the TME of the MPM cases. Mapping the geographic cell phenotype distribution in the TME, we were able to identify two distinct, complex immune landscapes characterized by specific patterns of cellular distribution as well as cell phenotype interactions with malignant cells. Successful we showed the optimization and reproducibility of our mIF panel and their incorporation for comprehensive TME immune profiling into translational studies that could refine our ability to correlate immunologic phenotypes with specific patterns of cells distribution and distance analysis. Overall, this will improve our ability to understand the behavior of cells within the TME and predict new treatment strategies to improve patient outcome.

The immune checkpoint involving PD-1 and PD-L1 generates suppressive signals in T cells, which help to prevent autoimmunity by inducing a state of immune exhaustion. In the context of the tumor microenvironment (TME), however, cancer cells can manipulate these pathways to camouflage themselves from an immune attack. Blocking these checkpoints has thus emerged as a key immunotherapeutic tactic. Nonetheless, the success rate of immune checkpoint inhibitors (ICIs) has been mixed, even for patients who test positive for relevant diagnostic biomarkers. Patient stratification currently depends on immunohistochemical staining for checkpoint proteins like PD-L1, but these tests do not provide adequate stratification. A more comprehensive stratification, one that includes immune profiling of the TME plus an evaluation of immune checkpoint interactions, might provide better patient stratification and ICI responsiveness. Here we describe mIF that is augmented with a PD-1 and PD-L1 protein-protein interaction assay. Human formalin-fixed, paraffin-embedded (FFPE) tissue sections were subjected to standard histological processing and then incubated with primary antibodies against PD1 and PD-L1. Sections were then treated with oligonucleotide-modified secondary probes suitable for in situ Proximity Ligation Assay (isPLA), enabling the detection of PD1 and PD-L1 interactions. The PLA protocol utilized a fluorescent probe complementary to amplified nucleic acid to enhance the visibility of the protein interactions. The sample was then mounted in a closed microfluidic chamber and imaged on the CellScape™ platform for Precise Spatial Multiplexing. After imaging of the isPLA signal, the CellScape cyclic multiplex mIF staining approach was used to iteratively stain immune and structural markers on the tissue sample utilizing the VistaPlex™ Spatial Immune Profiling Assay Kit. This proof-of-concept study demonstrates that high-plex mIF can be augmented with an isPLA on human FFPE samples. The combined application of both methods allows visualization of PD1/PD-L1 protein-protein interactions and integrates this interaction within the spatial context of the surrounding cell populations. This approach allows for a more comprehensive insight into the interplay of different immune cell and non-immune cell populations during checkpoint activation processes in normal and neoplastic tissues. Citation Format: Arne Christians, Jannik Boog, Charles E. Jackson, Matthew H. Ingalls, J Spencer Schwarz, Sara Bodbin, Agata Zieba Wicher, Subham Basu, Oliver Braubach. High-resolution analysis of immune checkpoint activation utilizing a combined PD1/PD-L1 in situ proximity ligation assay (isPLA) and multiplex immunofluorescence (mIF) imaging approach [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 5354.

2580 Background: Multiplex staining and imaging, a state-of-the-art technology, has revolutionized the simultaneous visualization of multiple protein markers within a single tissue sample. Various techniques have emerged to capture multiplex images with up to one hundred markers, enabling a deeper understanding of complex biological processes. The increased marker count increased the likelihood of staining and imaging failure, leading to higher resource usage in multiplex staining and imaging. We address this challenge by proposing a deep learning method and leveraging latent biological relationships between markers to accurately impute unstained protein markers. Methods: A deep learning-based marker imputation model for multiplex images (MAXIM) was developed and trained. The model’s imputation ability is evaluated at pixel and cell levels across various cancer types. Additionally, we present a comparison between imputed and actual marker images within the context of a downstream cell classification task. The MAXIM model’s interpretability is enhanced by gaining insights into the contribution of individual markers in the imputation process. Results: MAXIM was successfully trained and evaluated on a whole slide multiplex immunofluorescence (mIF) imaging datasets (14,476 images), encompassing cases from four different cancer types: Urothelial, Anal, Cervical, Head and Neck Squamous Cell Carcinoma (HNSCC). A separate MAXIM model was trained for each marker in mIF images, using the remaining markers as input. MAXIM performance was evaluated using structural similarity index (SSIM) and mean absolute error (MAE) between the imputed marker images and corresponding real marker images. MAXIM achieved high median SSIM, and low median MAE scores as well as high precision scores (AUC 0.95-0.99). Conclusions: The MAXIM’s method provides a platform with multiple potentials. First, laboratories can seamlessly train an in-house MAXIM model using images devoid of staining issues. The trained model can then be employed to accurately impute markers in multiplexed images that are marred by staining problems. Second, MAXIM can serve as a valuable tool for quality control in newly generated multiplex images, aiding in the detection of staining failures. The strong correlation between imputed and real markers in new images will be an indicator of staining integrity. In practice, MAXIM can reduce the cost and time of multiplex staining and image acquisition by accurately imputing protein markers with less staining. Third the interpretability of MAXIM provides the opportunity to uncover previously unknown latent biological relationships between different protein markers, leading to new insights in the field. Finally, the method can be scaled up for discovery of novel and clinically relevant biomarkers beneficial for offering targeted treatments in different cancer types.

BackgroundThe identification of robust biomarkers that will allow for the selection of cancer patients who will benefit from a given therapeutic is critically needed for the design of late-stage clinical...

BackgroundMultiplexed immunofluorescence imaging is a powerful spatial biology tool that can produce rich marker expression data at single-cell resolution and whole-slide scales. High parameter images have been particularly useful for...

Risk stratification remains a critical challenge in non-small cell lung cancer patients for optimal therapy selection. In this study, we develop an artificial intelligence-powered spatial cellomics approach that combines histology, multiplex immunofluorescence imaging and multimodal machine learning to characterize the complex cellular relationships of 43 cell phenotypes in the tumor microenvironment in a real-world retrospective cohort of 1168 non-small cell lung cancer patients from two large German cancer centers. The model identifies cell niches associated with survival and achieves a 14% and 47% improvement in risk stratification in the two main non-small cell lung cancer subtypes, lung adenocarcinoma and squamous cell carcinoma, respectively, combining niche patterns with conventional cancer staging. Our results show that complex immune cell niche patterns identify potentially undertreated high-risk patients qualifying for adjuvant therapy. Our approach highlights the potential of artificial intelligence powered multiplex imaging analyses to better understand the contribution of the tumor microenvironment to cancer progression and to improve risk stratification and treatment selection in non-small cell lung cancer.

Multiplex immunofluorescence staining and image analysis assay for diffuse large B cell lymphoma