Slide Images Research Articles

Application of quantitative histomorphometric features in computational pathology

AbstractComputer vision has facilitated the execution of various computer‐aided diagnostic tasks. From a methodological perspective, these tasks are primarily implemented using two dominant strategies: end‐to‐end Deep learning (DL)‐based methods and traditional feature engineering‐based methods. DL methods are capable of automatically extracting, analyzing, and filtering features, leading to final decision‐making from whole slide images. However, these methods are often criticized for the “black box” issue, a significant limitation of DL. In contrast, traditional feature engineering‐based methods involve well‐defined quantitative input features. But it was considered as less potent than DL methods. Advances in segmentation technology and the development of quantitative histomorphometric (QH) feature representation have propelled the evolution of feature engineering‐based methods. This review contrasts the performance differences between the two methods and traces the development of QH feature representation. The conclusion is that, with the ongoing progress in QH feature representation and segmentation technology, methods based on QH features will leverage their advantages—such as explainability, reduced reliance on large training datasets, and lower computational resource requirements—to play a more significant role in some clinical tasks. They may even replace DL methods somewhat or be used in conjunction with them to achieve accurate and understandable results.

Improved DeTraC Binary Coyote Net-Based Multiple Instance Learning for Predicting Lymph Node Metastasis of Breast Cancer From Whole-Slide Pathological Images.

Early detection of lymph node metastasis in breast cancer is vital for improving treatment outcomes and prognosis. This study introduces an Improved Decompose, Transfer, and Compose Binary Coyote Net-based Multiple Instance Learning (ImDeTraC-BCNet-MIL) method for predicting lymph node metastasis from Whole Slide Images (WSIs) using multiple instance learning. The method involves segmenting WSIs into patches using Otsu and double-dimensional clustering techniques. The developed multiple instance learning approach introduces a paradigm into computational pathology by shaping pathological data and constructing features. ImDeTraC-BCNet-MIL was utilised for feature generation during both training and testing to differentiate lymph node metastasis in WSIs. The proposed model achieves the highest accuracy of 95.3% and 99.8%, precision values of 98% and 99.8%, and recall rates of 92.9% and 99.8% on the Camelyon16 and Camelyon17 datasets. These findings underscore the effectiveness of ImDeTraC-BCNet-MIL in enhancing the early detection of lymph node metastasis in breast cancer.

MOTH: Memory-Efficient On-the-Fly Tiling of Histological Image Annotations Using QuPath

The emerging usage of digitalized histopathological images is leading to a novel possibility for data analysis. With the help of artificial intelligence algorithms, it is now possible to detect certain structures and morphological features on whole slide images automatically. This enables algorithms to count, measure, or evaluate those areas when trained properly. To achieve suitable training, datasets must be annotated and curated by users in programs like QuPath. The extraction of this data for artificial intelligence algorithms is still rather tedious and needs to be saved on a local hard drive. We developed a toolkit for integration into existing pipelines and tools, like U-net, for the on-the-fly extraction of annotation tiles from existing QuPath projects. The tiles can be directly used as input for artificial intelligence algorithms, and the results are directly transferred back to QuPath for visual inspection. With the toolkit, we created a convenient way to incorporate QuPath into existing AI workflows.

Digital profiling of gene expression from histology images with linearized attention.

Cancer is a heterogeneous disease requiring costly genetic profiling for better understanding and management. Recent advances in deep learning have enabled cost-effective predictions of genetic alterations from whole slide images (WSIs). While transformers have driven significant progress in non-medical domains, their application to WSIs lags behind due to high model complexity and limited dataset sizes. Here, we introduce SEQUOIA, a linearized transformer model that predicts cancer transcriptomic profiles from WSIs. SEQUOIA is developed using 7584 tumor samples across 16 cancer types, with its generalization capacity validated on two independent cohorts comprising 1368 tumors. Accurately predicted genes are associated with key cancer processes, including inflammatory response, cell cycles and metabolism. Further, we demonstrate the value of SEQUOIA in stratifying the risk of breast cancer recurrence and in resolving spatial gene expression at loco-regional levels. SEQUOIA hence deciphers clinically relevant information from WSIs, opening avenues for personalized cancer management.

PATH-50. AI-POWERED AUTOMATED TISSUE SEGMENTATION IMPROVES OUTCOME STRATIFICATION IN GLIOBLASTOMA

Abstract Glioblastoma displays morphological heterogeneity on routine H&E whole slide images, with conflicting evidence on the reliability of morphological features in predicting tumor behavior and treatment response. However, current standards for assessing morphology and disease status rely on nonquantitative manual evaluation by pathologists, which are prone to inherent variation. We sought to (1) expand the tools available for quantitative assessment and (2) evaluate the utility of this approach in identifying valuable biomarkers. We trained a deep learning-based algorithm using DenseNet on HALO AI platform for automated segmentation of six tissue classes (heavy infiltration, mild infiltration, mesenchymal tissue, geographic necrosis, palisading necrosis, and hemorrhage) on 3232 labeled annotations from 100 whole slide images. Manual pathologists’ estimations from clinical records showed strong correlation with corresponding digital quantifications (p<0.001), and further pixel-wise validation on 828 separate annotations showed high overlap between predicted and ground truth segmentations (precision=0.88, recall= 0.85, F1-score=0.87). We then identified a longitudinal cohort of 115 newly diagnosed glioblastoma tissue paired with the recurrence post-treatment (RT/TMZ), including all 827 H&E whole slide images scanned at 40x. The quantification of longitudinally paired tissue showed a relative increase in geographic necrosis (+62%) and mild infiltration (+110%) and a relative decrease in palisading necrosis (-54%) and heavy infiltration (-35%) between the two timepoints. Patients with rapid clinical progression had more areas of large geographic necrosis (p=0.002), and mesenchymal tissue increase was associated with worse overall survival (p=0.016). Digitally quantified heavy infiltration as a predominant tissue on recurrence outperformed corresponding pathologist estimation of tumor as a predominant tissue on recurrence in stratifying overall-survival (p=0.005 vs p=0.640). Collectively, our AI-powered classifier enables an automated objective quantification of glioblastoma tissue, identifies markers of clinical response, and offers a potential adjunct to enhance routine pathologic reporting of glioblastoma.

PATH-39. AI-BASED IDENTIFICATION OF GLIOMA IDH MUTATIONAL STATUS FROM H&E-STAINED WHOLE SLIDE IMAGES

Abstract INTRODUCTION Isocitrate-dehydrogenase (IDH) mutational status is diagnostically critical in adult gliomas, with prognostic and therapeutic implications. IDH status cannot be determined by sole histologic assessment and requires molecular testing. We hypothesize that AI-based analysis can accurately predict IDH mutational status in glioma, by retrieving sub-visual cues in H&E-stained digitized tissue sections. MATERIALS AND METHODS A retrospective discovery cohort of 1,534 cases (756/778 IDH-wildtype/IDH-mutant, including all available grades) from the TCGA-LGG/TCGA-GBM collections was used for model development with 10-fold cross-validation (CV). A replication cohort of 114 cases (82/32 IDH-wildtype/IDH-mutant) from the University of Pennsylvania Health System (UPHS) was used for independent hold-out evaluation. Each 20X magnification whole slide image (WSI) underwent comprehensive curation for elimination of artifactual content, followed by partitioning into 256x256 patches, and feature extraction using pre-trained i) ImageNet weights (baseline), and ii) self-supervised vision transformer (ViT). A weakly-supervised attention-based multiple-instance-learning framework distinguished cases as IDH-wildtype or IDH-mutant, while generating visually interpretable attention heatmaps. RESULTS Evaluation on discovery and replication cohorts with the ViT yield accuracy of 90.84% (AUCDiscovery=97.16%) and 93.94% (AUCReplication=97.36%), respectively, with high sensitivity (i.e., high confidence in correctly predicting IDH-wildtype glioma). Comparison of the ViT with the baseline model on the replication cohort indicates the ViT superiority by improvements of 8% and 9.5% in accuracy and AUC, respectively. Histologic assessment of heatmaps indicates IDH-wildtype tumors exhibiting distinct regions of significant pleomorphism and microvascular proliferation, while IDH-mutant tumors exhibit dense nodular cell concentrations, microcystic architecture, and gemistocytic cells. CONCLUSION Our accurate H&E-based computational determination of glioma IDH status, with interpretations aligned with human-identifiable features, can contribute to obviating the need for molecular analyses and enable expedited diagnosis even in community healthcare settings. This study robustly identifies morpho-IDH correlates and could enable expedited patient prognostic stratification.

Deep-Learning-Based Approach in Cancer-Region Assessment from HER2-SISH Breast Histopathology Whole Slide Images

Fluorescence in situ hybridization (FISH) is widely regarded as the gold standard for evaluating human epidermal growth factor receptor 2 (HER2) status in breast cancer; however, it poses challenges such as the need for specialized training and issues related to signal degradation from dye quenching. Silver-enhanced in situ hybridization (SISH) serves as an automated alternative, employing permanent staining suitable for bright-field microscopy. Determining HER2 status involves distinguishing between “Amplified” and “Non-Amplified” regions by assessing HER2 and centromere 17 (CEN17) signals in SISH-stained slides. This study is the first to leverage deep learning for classifying Normal, Amplified, and Non-Amplified regions within HER2-SISH whole slide images (WSIs), which are notably more complex to analyze compared to hematoxylin and eosin (H&E)-stained slides. Our proposed approach consists of a two-stage process: first, we evaluate deep-learning models on annotated image regions, and then we apply the most effective model to WSIs for regional identification and localization. Subsequently, pseudo-color maps representing each class are overlaid, and the WSIs are reconstructed with these mapped regions. Using a private dataset of HER2-SISH breast cancer slides digitized at 40× magnification, we achieved a patch-level classification accuracy of 99.9% and a generalization accuracy of 78.8% by applying transfer learning with a Vision Transformer (ViT) model. The robustness of the model was further evaluated through k-fold cross-validation, yielding an average performance accuracy of 98%, with metrics reported alongside 95% confidence intervals to ensure statistical reliability. This method shows significant promise for clinical applications, particularly in assessing HER2 expression status in HER2-SISH histopathology images. It provides an automated solution that can aid pathologists in efficiently identifying HER2-amplified regions, thus enhancing diagnostic outcomes for breast cancer treatment.

NIMG-18. RAPID INTRAOPERATIVE DIFFERENTIATION OF PRIMARY CNS LYMPHOMA FROM COMMON CNS TUMORS USING STIMULATED RAMAN HISTOLOGY AND DEEP LEARNING

Abstract Accurate intraoperative diagnosis is crucial for decision-making during tumor surgery and stereotactic-guided biopsies. Differentiating between primary CNS lymphoma (PCNSL) and non-PCNSL entities such as gliomas and metastatic cancers presents significant challenges due to overlapping histomorphological features, time constraints, and the differing further therapy strategies required. Our study utilized deep learning and stimulated Raman histology (SRH) to address this challenge. We imaged unprocessed, label-free tissue samples intraoperatively using a portable Raman scattering microscope, generating virtual H&E-like images within less than five minutes. Our training set comprised over 54,000 SRH tumor patch images, including various brain tumors. We developed a deep learning pipeline utilizing a ResNet-50 feature extractor and the BYOL (Bootstrap Your Own Latent) self-supervised learning algorithm. We trained a binary linear classifier to distinguish between PCNSL and non-PCNSL entities. Data were collected from New York University (NYU), the University of Cologne (UKK), and the University of Michigan (UM), including 263 SRH whole slide images from 177 patients sourced from surgical resections and stereotactic-guided biopsies. In an international bicentric test cohort (N = 100, NYU and UKK), the model achieved an overall accuracy of 91.1%, an area under the receiver operating characteristic curve (AUROC) of 0.983, a sensitivity of 89.3%, and a specificity of 100% compared to the final formalin-fixed, paraffin-embedded-based diagnosis. Our algorithm can accurately extract typical histomorphological features to help differentiate between PCNSL and non-PCNSL tumors. These results underscore the potential of deep learning models to enhance the accuracy of intraoperative diagnoses for PCNSL and non-PCNSL entities, thereby improving intraoperative decision-making and further treatment strategies by supporting neuropathologists and surgeons.

PATH-60. AUTOMATED QUANTIFICATION OF KI-67 PROLIFERATION INDEX BASED ON WHOLE SLIDE IMAGES IN PEDIATRIC BRAIN TUMORS

Abstract The use of AI methods with histopathological images (pathomics) has strong potential to aid pathologists in pediatric brain cancer diagnosis and prognosis given the morphological and molecular heterogeneity of these tumors. However, use of pathomics in this context has been relatively limited. One example is the IHC-based tumor proliferation index (percentage of Ki-67 positive cells amongst all malignant cells) which is often routinely reviewed by pathologists clinically but remains to be objectively quantified or reported in a standardized manner. As a result, analysis of this marker across patients at-scale and in data-driven methods remains largely hindered. Here, we explore the extension of an automated deep learning AI tool (DeepLIIF), originally trained and validated with adult breast/prostate/liver data, for quantification of Ki-67 index from digitized whole slide images of a cohort of pediatric brain tumors from the Children’s Brain Tumor Network (N=834). We find that derived Ki-67 scores largely agreed with neuropathologist-reported Ki-67 activity in original clinical notes and could be used to successfully stratify glioma/astrocytoma patients (N=200) into low- and high-grade groups (LGG: M=11.6; SEM=1.2; HGG: M=18.7; SEM=2; p=0.001). Ki-67 scores were also related to the survival status of the patients (Alive: M=11.8; SEM=1.1; Deceased: M=18.8; SEM=2.2; p=0.003). Summary statistics across tumor types (glioma, astrocytoma, medulloblastoma, ATRT, ependymoma, etc.) and molecularly defined subtypes are also compared, providing direct insights into distributional properties between these groups that has not previously been feasible. Our results show the ability of AI-powered methods to quantify routine histopathology metrics with little human input, allowing objective comparisons between patients and opening the door for precision medicine approaches with predictive analytics in clinical contexts.

Artificial intelligence-based morphologic classification and molecular characterization of neuroblastic tumors from digital histopathology

A deep learning model using attention-based multiple instance learning (aMIL) and self-supervised learning (SSL) was developed to perform pathologic classification of neuroblastic tumors and assess MYCN-amplification status using H&E-stained whole slide images from the largest reported cohort to date. The model showed promising performance in identifying diagnostic category, grade, mitosis-karyorrhexis index (MKI), and MYCN-amplification with validation on an external test dataset, suggesting potential for AI-assisted neuroblastoma classification.

A Streamlined Workflow for Microscopy-Driven MALDI Imaging Mass Spectrometry Data Collection.

Matrix-assisted laser desorption/ionization imaging mass spectrometry (MALDI IMS) is a rapidly advancing technology for biomedical research. As spatial resolution increases, however, so do acquisition time, file size, and experimental cost, which increases the need to perform precise sampling of targeted tissue regions to optimize the biological information gleaned from an experiment and minimize wasted resources. The ability to define instrument measurement regions based on key tissue features and automatically sample these specific regions of interest (ROIs) addresses this challenge. Herein, we demonstrate a workflow using standard software that allows for direct sampling of microscopy-defined regions by MALDI IMS. Three case studies are included, highlighting different methods for defining features from common sample types─manual annotation of vasculature in human brain tissue, automated segmentation of renal functional tissue units across whole slide images using custom segmentation algorithms, and automated segmentation of dispersed HeLa cells using open-source software. Each case minimizes data acquisition from unnecessary sample regions and dramatically increases throughput while uncovering molecular heterogeneity within targeted ROIs. This workflow provides an approachable method for spatially targeted MALDI IMS driven by microscopy as part of multimodal molecular imaging studies.

Quantifying the recovery process of skeletal muscle on hematoxylin and eosin stained images via learning from label proportion

Visual observing muscle tissue regeneration is used to measure experimental effect size in biological research to discover the mechanism of muscle strength decline due to illness or aging. Quantitative computer imaging analysis for support evaluating the recovery phase has not been established because of the localized nature of recovery and the difficulty in selecting image features for cells in regeneration. We constructed MyoRegenTrack for segmenting cells and classifying their regeneration phase in hematoxylin–eosin (HE) stained images. A straightforward approach to classification is supervised learning. However, obtaining detailed annotations for each fiber in a whole slide image is impractical in terms of cost and accuracy. Thus, we propose to learn individual recovery phase classification utilizing the proportions of cell class depending on the days after muscle injection to induce regeneration. We extract implicit multidimensional features from the HE-stained tissue images and train a classifier using weakly supervised learning, guided by their class proportion for elapsed time on recovery. We confirmed the effectiveness of MyoRegenTrack by comparing its results with expert annotations. A comparative study of the recovery relation between two different muscle injections shows that the analysis result using MyoRegenTrack is consistent with findings from previous studies.

Classifying driver mutations of papillary thyroid carcinoma on whole slide image: an automated workflow applying deep convolutional neural network

BackgroundInformative biomarkers play a vital role in guiding clinical decisions regarding management of cancers. We have previously demonstrated the potential of a deep convolutional neural network (CNN) for predicting cancer driver gene mutations from expert-curated histopathologic images in papillary thyroid carcinomas (PTCs). Recognizing the importance of whole slide image (WSI) analysis for clinical application, we aimed to develop an automated image preprocessing workflow that uses WSI inputs to categorize PTCs based on driver mutations.MethodsHistopathology slides from The Cancer Genome Atlas (TCGA) repository were utilized for diagnostic purposes. These slides underwent an automated tile extraction and preprocessing pipeline to ensure analysis-ready quality. Next, the extracted image tiles were utilized to train a deep learning CNN model, specifically Google’s Inception v3, for the classification of PTCs. The model was trained to distinguish between different groups based on BRAFV600E or RAS mutations.ResultsThe newly developed pipeline performed equally well as the expert-curated image classifier. The best model achieved Area Under the Curve (AUC) values of 0.86 (ranging from 0.847 to 0.872) for validation and 0.865 (ranging from 0.854 to 0.876) for the final testing subsets. Notably, it accurately predicted 90% of tumors in the validation set and 84.2% in the final testing set. Furthermore, the performance of our new classifier showed a strong correlation with the expert-curated classifier (Spearman rho = 0.726, p = 5.28 e-08), and correlated with the molecular expression-based classifier, BRS (BRAF-RAS scores) (Spearman rho = 0.418, p = 1.92e-13).ConclusionsUtilizing WSIs, we implemented an automated workflow with deep CNN model that accurately classifies driver mutations in PTCs.

Current status and challenges in HER2 IHC assessment: scoring survey results in Japan.

This study aimed to assess the concordance of human epidermal growth factor receptor 2 (HER2) expression scoring by immunohistochemistry (IHC) among practicing pathologists in Japan, given the challenging nature of scoring and the critical role of HER2 status in breast cancer management. Whole slide images (WSI) from 20 invasive breast cancer cases (1 representative WSI per case) selected to represent a diverse IHC scores and staining patterns were used in an online survey involving seven reference pathologists who established consensus HER2 IHC scores (0 to 3 +) decided by majority interpretation. Participating pathologists nationwide scored the same 20 WSI cases online using the American Society of Clinical Oncology/College of American Pathologists (ASCO/CAP) 2018 guidelines. Deidentified case metadata were registered in the uPath system. A total of 144 participating pathologists responded. The scoring results of the participating pathologists most commonly agreed with the consensus IHC score, followed by a ± 1 point deviation and no survey responses with > 1 point deviation. The mean percentage of agreement with the consensus score for all 20 cases was 63.4%. In cases where the reference pathologists' scores were discordant, the participating pathologists also showed a lower concordance rate. This study highlighted the current status of HER2 expression scoring by IHC for breast cancer among pathologists in Japan. These findings underscore the challenges in HER2 IHC scoring cases and emphasize the need for improved standardization and training, especially in the evolving landscape of HER2-targeted therapies.

A deep learning approach for ovarian cancer detection and classification based on fuzzy deep learning

Different oncologists make their own decisions about the detection and classification of the type of ovarian cancer from histopathological whole slide images. However, it is necessary to have an automated system that is more accurate and standardized for decision-making, which is essential for early detection of ovarian cancer. To help doctors, an automated detection and classification of ovarian cancer system is proposed. This model starts by extracting the main features from the histopathology images based on the ResNet-50 model to detect and classify the cancer. Then, recursive feature elimination based on a decision tree is introduced to remove unnecessary features extracted during the feature extraction process. Adam optimizers were implemented to optimize the network’s weights during training data. Finally, the advantages of combining deep learning and fuzzy logic are combined to classify the images of ovarian cancer. The dataset consists of 288 hematoxylin and eosin (H&E) stained whole slides with clinical information from 78 patients. H&E-stained Whole Slide Images (WSIs), including 162 effective and 126 invalid WSIs were obtained from different tissue blocks of post-treatment specimens. Experimental results can diagnose ovarian cancer with a potential accuracy of 98.99%, sensitivity of 99%, specificity of 98.96%, and F1-score of 98.99%. The results show promising results indicating the potential of using fuzzy deep-learning classifiers for predicting ovarian cancer.

Positional encoding-guided transformer-based multiple instance learning for histopathology whole slide images classification

Background and objectives:Whole slide image (WSI) classification is of great clinical significance in computer-aided pathological diagnosis. Due to the high cost of manual annotation, weakly supervised WSI classification methods have gained more attention. As the most representative, multiple instance learning (MIL) generally aggregates the predictions or features of the patches within a WSI to achieve the slide-level classification under the weak supervision of WSI labels. However, most existing MIL methods ignore spatial position relationships of the patches, which is likely to strengthen the discriminative ability of WSI-level features. Methods:In this paper, we propose a novel positional encoding-guided transformer-based multiple instance learning (PEGTB-MIL) method for histopathology WSI classification. It aims to encode the spatial positional property of the patch into its corresponding semantic features and explore the potential correlation among the patches for improving the WSI classification performance. Concretely, the deep features of the patches in WSI are first extracted and simultaneously a position encoder is used to encode the spatial 2D positional information of the patches into the spatial-aware features. After incorporating the semantic features and spatial embeddings, multi-head self-attention (MHSA) is applied to explore the contextual and spatial dependencies of the fused features. Particularly, we introduce an auxiliary reconstruction task to enhance the spatial-semantic consistency and generalization ability of features. Results:The proposed method is evaluated on two public benchmark TCGA datasets (TCGA-LUNG and TCGA-BRCA) and two in-house clinical datasets (USTC-EGFR and USTC-GIST). Experimental results validate it is effective in the tasks of cancer subtyping and gene mutation status prediction. In the test stage, the proposed PEGTB-MIL outperforms the other state-of-the-art methods and respectively achieves 97.13±0.34%, 86.74±2.64%, 83.25±1.65%, and 72.52±1.63% of the area under the receiver operating characteristic (ROC) curve (AUC). Conclusion:PEGTB-MIL utilizes positional encoding to effectively guide and reinforce MIL, leading to enhanced performance on downstream WSI classification tasks. Specifically, the introduced auxiliary reconstruction module adeptly preserves the spatial-semantic consistency of patch features. More significantly, this study investigates the relationship between position information and disease diagnosis and presents a promising avenue for further research.

AI-based tumor-infiltrating lymphocytes scoring system for assessing HCC prognosis in patients undergoing liver resection

Background & AimsTumor-infiltrating lymphocytes (TILs), particularly CD8+ TILs, are key prognostic markers in many cancers. However, their prognostic value in hepatocellular carcinoma (HCC) remains controversial, with different evidence. Given the heterogeneous outcomes in HCC patients undergoing liver resection, this study aims to develop an AI-based system to quantify CD8+ TILs and assess their prognostic value for HCC patients. MethodsWe conducted a retrospective multicenter study on patients undergoing liver resection across three cohorts (N=514). We trained a deep neural network and a random forest model to segment tumor regions and locate CD8+ TILs in H&E and CD8-stained whole slide images (WSIs). We quantified CD8+ TIL density and established an Automated CD8+ Tumor-infiltrating Lymphocyte Scoring (ATLS-8) system to assess its prognostic value. ResultsIn discovery cohort, the 5-year overall survival (OS) rates were 34.05% for ATLS-8 low-score and 65.03% for ATLS-8 high-score groups (HR 2.40; 95% CI, 1.37–4.19; P = 0.015). These findings were confirmed in validation cohort 1, which had 5-year OS rates of 28.57% and 68.73% (HR 3.38; 95% CI, 1.27–9.02; P = 0.0098), and validation cohort 2, which had 59.26% and 81.48% (HR 2.74; 95% CI, 1.05–7.15; P = 0.031). ATLS-8 improved prognostic model based on clinical variables (C-index 0.770 vs. 0.757; 0.769 vs. 0.727; 0.712 vs. 0.642 in three cohorts). ConclusionWe developed an automated system using CD8-stained WSIs to assess immune infiltration (ATLS-8). In HCC patients undergoing resection, higher CD8+ TIL density correlates with better OS, as per ATLS-8 assessment. This system is a promising tool for advancing clinical immune microenvironment assessment and outcome prediction. Impact and implicationCD8+ TILs have been identified as a prognostic factor associated with many cancers. In this study, CD8+ TILs were identified as an independent prognostic factor for OS in patients with hepatocellular carcinoma who undergoing liver resection. Therefore, ATLS-8, a novel digital biomarker based on WSI-level CD8+ TILs, could play an important role in the prognostic assessment of HCC patients and could be integrated into clinicopathological models to participate in the decision-making and prognostic assessment of patients. The scoring system combined with artificial intelligence is essential for automated, quantitative, WSI- level assessment of TILs, which can be widely applied to quantify the immune profile of multi-cancer disease types with the discussion of subsequent immunotherapy. Clinical trial numberThis is a retrospective study and does not require a clinical trial number.

Breast cancer survival prediction using an automated mitosis detection pipeline.

Mitotic count (MC) is the most common measure to assess tumor proliferation in breast cancer patients and is highly predictive of patient outcomes. It is, however, subject to inter- and intraobserver variation and reproducibility challenges that may hamper its clinical utility. In past studies, artificial intelligence (AI)-supported MC has been shown to correlate well with traditional MC on glass slides. Considering the potential of AI to improve reproducibility of MC between pathologists, we undertook the next validation step by evaluating the prognostic value of a fully automatic method to detect and count mitoses on whole slide images using a deep learning model. The model was developed in the context of the Mitosis Domain Generalization Challenge 2021 (MIDOG21) grand challenge and was expanded by a novel automatic area selector method to find the optimal mitotic hotspot and calculate the MC per 2 mm2. We employed this method on a breast cancer cohort with long-term follow-up from the University Medical Centre Utrecht (N = 912) and compared predictive values for overall survival of AI-based MC and light-microscopic MC, previously assessed during routine diagnostics. The MIDOG21 model was prognostically comparable to the original MC from the pathology report in uni- and multivariate survival analysis. In conclusion, a fully automated MC AI algorithm was validated in a large cohort of breast cancer with regard to retained prognostic value compared with traditional light-microscopic MC.

Keep It Accurate and Robust: An Enhanced Nuclei Analysis Framework

Accurate segmentation and classification of nuclei in histology images is critical but challenging due to nuclei heterogeneity, staining variations, and tissue complexity. Existing methods often struggle with limited dataset variability, with patches extracted from similar whole slide images (WSI), making models prone to falling into local optima. Here we propose a new framework to address this limitation and enable robust nuclear analysis. Our method leverages dual-level ensemble modeling to overcome issues stemming from limited dataset variation. Intra-ensembling applies diverse transformations to individual samples, while inter-ensembling combines networks of different scales. We also introduce enhancements to the HoVer-Net architecture, including updated encoders, nested dense decoding and model regularization strategy. We achieve state-of-the-art results on public benchmarks, including 1st place for nuclear composition prediction and 3rd place for segmentation/classification in the 2022 Colon Nuclei Identification and Counting (CoNIC) Challenge. This success validates our approach for accurate histological nuclei analysis. Extensive experiments and ablation studies provide insights into optimal network design choices and training techniques. In conclusion, this work proposes an improved framework advancing the state-of-the-art in nuclei analysis. We will release our code and models to serve as a toolkit for the community.

Multimodal Co-Attention Fusion Network With Online Data Augmentation for Cancer Subtype Classification.

It is an essential task to accurately diagnose cancer subtypes in computational pathology for personalized cancer treatment. Recent studies have indicated that the combination of multimodal data, such as whole slide images (WSIs) and multi-omics data, could achieve more accurate diagnosis. However, robust cancer diagnosis remains challenging due to the heterogeneity among multimodal data, as well as the performance degradation caused by insufficient multimodal patient data. In this work, we propose a novel multimodal co-attention fusion network (MCFN) with online data augmentation (ODA) for cancer subtype classification. Specifically, a multimodal mutual-guided co-attention (MMC) module is proposed to effectively perform dense multimodal interactions. It enables multimodal data to mutually guide and calibrate each other during the integration process to alleviate inter- and intra-modal heterogeneities. Subsequently, a self-normalizing network (SNN)-Mixer is developed to allow information communication among different omics data and alleviate the high-dimensional small-sample size problem in multi-omics data. Most importantly, to compensate for insufficient multimodal samples for model training, we propose an ODA module in MCFN. The ODA module leverages the multimodal knowledge to guide the data augmentations of WSIs and maximize the data diversity during model training. Extensive experiments are conducted on the public TCGA dataset. The experimental results demonstrate that the proposed MCFN outperforms all the compared algorithms, suggesting its effectiveness.