Digital Pathology Images Research Articles

Breast Cancer Classification From Digital Pathology Images via Connectivity-Aware Graph Transformer.

Automated classification of breast cancer subtypes from digital pathology images has been an extremely challenging task due to the complicated spatial patterns of cells in the tissue micro-environment. While newly proposed graph transformers are able to capture more long-range dependencies to enhance accuracy, they largely ignore the topological connectivity between graph nodes, which is nevertheless critical to extract more representative features to address this difficult task. In this paper, we propose a novel connectivity-aware graph transformer (CGT) for phenotyping the topology connectivity of the tissue graph constructed from digital pathology images for breast cancer classification. Our CGT seamlessly integrates connectivity embedding to node feature at every graph transformer layer by using local connectivity aggregation, in order to yield more comprehensive graph representations to distinguish different breast cancer subtypes. In light of the realistic intercellular communication mode, we then encode the spatial distance between two arbitrary nodes as connectivity bias in self-attention calculation, thereby allowing the CGT to distinctively harness the connectivity embedding based on the distance of two nodes. We extensively evaluate the proposed CGT on a large cohort of breast carcinoma digital pathology images stained by Haematoxylin & Eosin. Experimental results demonstrate the effectiveness of our CGT, which outperforms state-of-the-art methods by a large margin. Codes are released on https://github.com/wang-kang-6/CGT.

Computer-aided diagnosis system for grading brain tumor using histopathology images based on color and texture features

BackgroundCancer pathology shows disease development and associated molecular features. It provides extensive phenotypic information that is cancer-predictive and has potential implications for planning treatment. Based on the exceptional performance of computational approaches in the field of digital pathogenic, the use of rich phenotypic information in digital pathology images has enabled us to identify low-level gliomas (LGG) from high-grade gliomas (HGG). Because the differences between the textures are so slight, utilizing just one feature or a small number of features produces poor categorization results.MethodsIn this work, multiple feature extraction methods that can extract distinct features from the texture of histopathology image data are used to compare the classification outcomes. The successful feature extraction algorithms GLCM, LBP, multi-LBGLCM, GLRLM, color moment features, and RSHD have been chosen in this paper. LBP and GLCM algorithms are combined to create LBGLCM. The LBGLCM feature extraction approach is extended in this study to multiple scales using an image pyramid, which is defined by sampling the image both in space and scale. The preprocessing stage is first used to enhance the contrast of the images and remove noise and illumination effects. The feature extraction stage is then carried out to extract several important features (texture and color) from histopathology images. Third, the feature fusion and reduction step is put into practice to decrease the number of features that are processed, reducing the computation time of the suggested system. The classification stage is created at the end to categorize various brain cancer grades. We performed our analysis on the 821 whole-slide pathology images from glioma patients in the Cancer Genome Atlas (TCGA) dataset. Two types of brain cancer are included in the dataset: GBM and LGG (grades II and III). 506 GBM images and 315 LGG images are included in our analysis, guaranteeing representation of various tumor grades and histopathological features.ResultsThe fusion of textural and color characteristics was validated in the glioma patients using the 10-fold cross-validation technique with an accuracy equals to 95.8%, sensitivity equals to 96.4%, DSC equals to 96.7%, and specificity equals to 97.1%. The combination of the color and texture characteristics produced significantly better accuracy, which supported their synergistic significance in the predictive model. The result indicates that the textural characteristics can be an objective, accurate, and comprehensive glioma prediction when paired with conventional imagery.ConclusionThe results outperform current approaches for identifying LGG from HGG and provide competitive performance in classifying four categories of glioma in the literature. The proposed model can help stratify patients in clinical studies, choose patients for targeted therapy, and customize specific treatment schedules.

Sequential Patching Lattice for Image Classification and Enquiry: Streamlining Digital Pathology Image Processing

Digital pathology and the integration of artificial intelligence (AI) models have revolutionized histopathology, opening new opportunities. With the increasing availability of Whole Slide Images (WSIs), there’s a growing demand for efficient retrieval, processing, and analysis of relevant images from vast biomedical archives. However, processing WSIs presents challenges due to their large size and content complexity. Full computer digestion of WSIs is impractical, and processing all patches individually is prohibitively expensive. In this paper, we propose an unsupervised patching algorithm, Sequential Patching Lattice for Image Classification and Enquiry (SPLICE). This novel approach condenses a histopathology WSI into a compact set of representative patches, forming a “collage” of WSI while minimizing redundancy. SPLICE prioritizes patch quality and uniqueness by sequentially analyzing a WSI and selecting nonredundant representative features. We evaluated SPLICE for search and match applications, demonstrating improved accuracy, reduced computation time, and storage requirements compared to existing state-of-the-art methods. As an unsupervised method, SPLICE effectively reduces storage requirements for representing tissue images by 50%. This reduction enables numerous algorithms in computational pathology to operate much more efficiently, paving the way for accelerated adoption of digital pathology.

Development of a multi-scanner facility for data acquisition for digital pathology artificial intelligence.

Whole slide imaging (WSI) of pathology glass slides using high-resolution scanners has enabled the large-scale application of artificial intelligence (AI) in pathology, to support the detection and diagnosis of disease, potentially increasing efficiency and accuracy in tissue diagnosis. Despite the promise of AI, it has limitations. 'Brittleness' or sensitivity to variation in inputs necessitates that large amounts of data are used for training. AI is often trained on data from different scanners but not usually by replicating the same slide across scanners. The utilisation of multiple WSI instruments to produce digital replicas of the same slides will make more comprehensive datasets and may improve the robustness and generalisability of AI algorithms as well as reduce the overall data requirements of AI training. To this end, the National Pathology Imaging Cooperative (NPIC) has built the AI FORGE (Facilitating Opportunities for Robust Generalisable data Emulation), a unique multi-scanner facility embedded in a clinical site in the NHS to (1) compare scanner performance, (2) replicate digital pathology image datasets across WSI systems, and (3) support the evaluation of clinical AI algorithms. The NPIC AI FORGE currently comprises 15 scanners from nine manufacturers. It can generate approximately 4,000 WSI images per day (approximately 7 TB of image data). This paper describes the process followed to plan and build such a facility. © 2024 The Author(s). The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Artificial intelligence facilitates clinical management of epithelial dysplasia in multiple organs

Epithelial dysplasia is a condition characterized by a spectrum of architectural and cytological alterations to the epithelium, resulting from the accumulation of genetic alterations. It is associated with an increased risk of cancer progression in a variety of organs. However, the variability of different grading systems, as well as inter- and intra-examiner variability, gives rise to concerns regarding the reliability of the results. Histopathology represents the gold standard for the diagnosis of epithelial dysplasia. The combination of big data in pathology and artificial intelligence (AI) will facilitate the achievement of accurate diagnoses and treatments, providing objective and efficient methods to integrate and refine diverse morphological, molecular, and multi-omics information. This perspective provides a summary of the existing research and prospects for the application of AI to epithelial dysplasia in multiple organs. A number of studies have been conducted with the aim of developing a grading system and prognostic identification method for epithelial dysplasia in the oral cavity, larynx, esophagus, and stomach. Digital pathology-based AI may prove useful in facilitating the clinical management of epithelial dysplasia in multiple organs. In summary, digital pathology images obtained by scanning hematoxylin & eosin-stained slides, identifying image features, and building AI models using deep learning combined with machine learning algorithms, validated with real-world data from multicenter cohorts could provide AI as a promising clinical application in the future.

Deep Residual Learning-Based Classification with Identification of Incorrect Predictions and Quantification of Cellularity and Nuclear Morphological Features in Digital Pathological Images of Common Astrocytic Tumors.

Interobserver variations in the pathology of common astrocytic tumors impact diagnosis and subsequent treatment decisions. This study leveraged a residual neural network-50 (ResNet-50) in digital pathological images of diffuse astrocytoma, anaplastic astrocytoma, and glioblastoma to recognize characteristic pathological features and perform classification at the patch and case levels with identification of incorrect predictions. In addition, cellularity and nuclear morphological features, including axis ratio, circularity, entropy, area, irregularity, and perimeter, were quantified via a hybrid task cascade (HTC) framework and compared between different characteristic pathological features with importance weighting. A total of 95 cases, including 15 cases of diffuse astrocytoma, 11 cases of anaplastic astrocytoma, and 69 cases of glioblastoma, were collected in Taiwan Hualien Tzu Chi Hospital from January 2000 to December 2021. The results revealed that an optimized ResNet-50 model could recognize characteristic pathological features at the patch level and assist in diagnosis at the case level with accuracies of 0.916 and 0.846, respectively. Incorrect predictions were mainly due to indistinguishable morphologic overlap between anaplastic astrocytoma and glioblastoma tumor cell area, zones of scant vascular lumen with compact endothelial cells in the glioblastoma microvascular proliferation area mimicking the glioblastoma tumor cell area, and certain regions in diffuse astrocytoma with too low cellularity being misrecognized as the glioblastoma necrosis area. Significant differences were observed in cellularity and each nuclear morphological feature among different characteristic pathological features. Furthermore, using the extreme gradient boosting (XGBoost) algorithm, we found that entropy was the most important feature for classification, followed by cellularity, area, circularity, axis ratio, perimeter, and irregularity. Identifying incorrect predictions provided valuable feedback to machine learning design to further enhance accuracy and reduce errors in classification. Moreover, quantifying cellularity and nuclear morphological features with importance weighting provided the basis for developing an innovative scoring system to achieve objective classification and precision diagnosis among common astrocytic tumors.

CohortFinder: an open-source tool for data-driven partitioning of digital pathology and imaging cohorts to yield robust machine-learning models

Batch effects (BEs) refer to systematic technical differences in data collection unrelated to biological variations whose noise is shown to negatively impact machine learning (ML) model generalizability. Here we release CohortFinder (http://cohortfinder.com), an open-source tool aimed at mitigating BEs via data-driven cohort partitioning. We demonstrate CohortFinder improves ML model performance in downstream digital pathology and medical image processing tasks. CohortFinder is freely available for download at cohortfinder.com.

Pixel-super-resolved lens-free quantitative phase microscopy with partially coherent illumination

Lens-free on-chip microscopy (LFOCM) has been widely utilized in digital pathology, drug screening, point-of-care testing (POCT), and quantitative phase imaging (QPI) due to its high throughput imaging capability and compactness. Initially, coherent laser sources were used in LFOCM to generate interference fringes to reconstruct the intensity and phase information of an object. The use of partially coherent light-emitting diodes (LEDs) in LFOCM offers a more portable and cost-effective alternative to conventional coherent illumination sources. However, the coherence-gating effect from a relatively low degree of coherence may cause a blur of high-frequency information in holograms, leading to an inaccurate object recovery. Thus, we present a pixel-super-resolved lens-free quantitative phase microscopy (PSR-LFQPM) with partially coherent illumination, which not only compensates for the impact of low coherence without increasing the volume of the system but also suppresses the theoretical Nyquist-Shannon sampling resolution limit imposed by the sensor pixel size (0.9 μm). Based on the partially coherent imaging model, we integrate the spatial coherence transfer function (SCTF) obtained from the pre-calibrated LED source distribution during the iteration process to obtain an accurate high-resolution recovery. Applying PSR-LFQPM to image living HeLa cells in vitro, we achieve real-time dynamic high-throughput QPI performance (half-pitch resolution of 780 nm with a 1.41-fold improvement compared to results without considering the effect of coherence) across a wide FOV (19.53 mm2). The proposed method provides a compact, low-cost, and high-throughput lens-free on-chip microscopy system for biomedical and POCT applications.

Artificial Intelligence Could be the Personalized Treatment Strategy for Cancer

Cancer is one of the world’s most serious medical challenges. Because of its great heterogeneity, persons with similar tumors could react in a different way to the same drugs or surgical methods, prompting the development of more accurate tumor treatment approaches as well as patient-specific tailored treatments. To establish targeted therapy choices for patients, it is critical to have a full acceptance of the changes that tumors endure, counting modifications in their genetic factors, proteins, and cancer cell behaviors. Tumor treatment requires precise targeting. Big data-driven artificial intelligence (AI) may reveal patterns, insights, and related information hidden inside huge volumes of data. To identify exact data from transcriptomics, radiomics, genomes, proteomics, digital pathological images, etc., subsets of AI’s machine learning capability may be explored. This may help clinicians get a better and more comprehensive knowledge of malignancies. In addition, to provides the optimal therapy for each patient and improves clinical outcomes.

Prognostic value of the international association for the study of lung cancer grading system and its association with the tumor microenvironment in stage I EGFR-muted lung adenocarcinoma

IntroductionThe International Association for the Study of Lung Cancer (IASLC) grading system predicts early lung adenocarcinoma outcomes. MethodsThe purpose of this study is to examine prognostic value of the IASLC grading system and its association with the tumor microenvironment (TME) in Stage I EGFR-muted lung adenocarcinoma. Based on the IASLC grading system, we compared the clinicopathological characteristics of EGFR-mutated lung adenocarcinoma (n = 296). In addition, we examined the expression level of E-cadherin in tumor cells and counted the number of tumor-infiltrating lymphocytes (TILs; CD8, CD20, CD138, and Foxp3), tumor-associated macrophages (TAMs; CD204), and cancer-associated fibroblasts (CAFs; podoplanin) using semi-automatic digital pathology image analysis. ResultsRecurrence-free survival (RFS) curve showed that survival of grade 3 was significantly shorter than that of grade 1 (P < 0.01) and grade 2 (P = 0.03). Multivariate analysis of RFS revealed the invasive size, lymphatic permeation, and grade 3 (P < 0.01) as independent poor prognostic factors. The number of CD204 +TAMs and PDPN+CAFs was significantly higher in grade 3 than in grade 1 or 2 (all P < 0.01). Among the intermediate grade by the predominant subtype based classification, cases classified as grade 3 by the new classification had higher number of CD204 +TAMs (P < 0.01) and PDPN+CAFs (P = 0.02) than those classified as grade 2. ConclusionThe IASLC grading system correlated with the outcomes of EGFR-mutated lung adenocarcinoma. Grade 3 was found to have the TME that most contributes to tumor progression, which probably explained their poor prognosis.

PatchSorter: a high throughput deep learning digital pathology tool for object labeling

The discovery of patterns associated with diagnosis, prognosis, and therapy response in digital pathology images often requires intractable labeling of large quantities of histological objects. Here we release an open-source labeling tool, PatchSorter, which integrates deep learning with an intuitive web interface. Using >100,000 objects, we demonstrate a >7x improvement in labels per second over unaided labeling, with minimal impact on labeling accuracy, thus enabling high-throughput labeling of large datasets.

Prediction of Tumor-Associated Macrophages and Immunotherapy Benefits Using Weakly Supervised Contrastive Learning in Breast Cancer Pathology Images.

The efficacy of immune checkpoint inhibitors is significantly influenced by the tumor immune microenvironment (TIME). RNA sequencing of tumor tissue can offer valuable insights into TIME, but its high cost and long turnaround time seriously restrict its utility in routine clinical examinations. Several recent studies have suggested that ultrahigh-resolution pathology images can infer cellular and molecular characteristics. However, few study pay attention to the quantitative estimation of various tumor infiltration immune cells from pathology images. In this paper, we integrated contrastive learning and weakly supervised learning to infer tumor-associated macrophages and potential immunotherapy benefit from whole slide images (WSIs) of H &E stained pathological sections. We split the high-resolution WSIs into tiles and then apply contrastive learning to extract features of each tile. After aggregating the features at the tile level, we employ weak supervisory signals to fine-tune the encoder for various downstream tasks. Comprehensive experiments on two independent breast cancer cohorts and spatial transcriptomics data demonstrate that the computational pathological features accurately predict the proportion of tumor-infiltrating immune cells, particularly the infiltration level of macrophages, as well as the immune subtypes and potential immunotherapy benefit. These findings demonstrate that our model effectively captures pathological features beyond human vision, establishing a mapping relationship between cellular compositions and histological morphology, thus expanding the clinical applications of digital pathology images.

Artificial intelligence multiprocessing scheme for pathology images based on transformer for nuclei segmentation

Malignant tumors are a common cytopathologic disease. Pathological tissue examination is a key tool for diagnosing malignant tumors. Doctors need to manually analyze the images of pathological tissue sections, which is not only time-consuming but also highly subjective, easily leading to misdiagnosis. Most of the existing computer-aided diagnostic techniques focus too much on accuracy when processing pathological tissue images, and do not take into account the problems of insufficient resources in developing countries to meet the training of large models and the difficulty of obtaining medical annotation data. Based on this, this study proposes an artificial intelligence multiprocessing scheme (MSPInet) for digital pathology images of malignant tumors. We use techniques such as data expansion and noise reduction to enhance the dataset. Then we design a coarse segmentation method for cell nuclei of pathology images based on Transformer for Semantic Segmentation and further optimize the segmentation of tumor edges using conditional random fields. Finally, we improve the training strategy for knowledge distillation. As a medical assistive system, the method can quantify and convert complex pathology images into analyzable image information. Experimental results show that our method performs well in terms of segmentation accuracy and also has advantages in terms of time and space efficiency. This makes our technology available to developing countries that are not as well resourced, and equipped in terms of medical care. The teacher model and lightweight student model included in our method achieve 71.6% and 66.1% Intersection over Union (IoU) in cell segmentation respectively, outperforming Swin-unet and CSWin Transformer.

AnVILWorkflow: A runnable workflow package for Cloud-implemented bioinformatics analysis pipelines.

Advancements in sequencing technologies and the development of new data collection methods produce large volumes of biological data. The Genomic Data Science Analysis, Visualization, and Informatics Lab-space (AnVIL) provides a cloud-based platform for democratizing access to large-scale genomics data and analysis tools. However, utilizing the full capabilities of AnVIL can be challenging for researchers without extensive bioinformatics expertise, especially for executing complex workflows. Here we present the AnVILWorkflow R package, which enables the convenient execution of bioinformatics workflows hosted on AnVIL directly from an R environment. AnVILWorkflowsimplifies the setup of the cloud computing environment, input data formatting, workflow submission, and retrieval of results through intuitive functions. We demonstrate the utility of AnVILWorkflowfor three use cases: bulk RNA-seq analysis with Salmon, metagenomics analysis with bioBakery, and digital pathology image processing with PathML. The key features of AnVILWorkflow include user-friendly browsing of available data and workflows, seamless integration of R and non-R tools within a reproducible analysis pipeline, and accessibility to scalable computing resources without direct management overhead. While some limitations exist around workflow customization, AnVILWorkflowlowers the barrier to taking advantage of AnVIL's resources, especially for exploratory analyses or bulk processing with established workflows. This empowers a broader community of researchers to leverage the latest genomics tools and datasets using familiar R syntax. This package is distributed through the Bioconductor project (https://bioconductor.org/packages/AnVILWorkflow), and the source code is available through GitHub (https://github.com/shbrief/AnVILWorkflow).

Digital pathology structure and deployment in Veneto: a proof-of-concept study

Nowadays pathology laboratories are worldwide facing a digital revolution, with an increasing number of institutions adopting digital pathology (DP) and whole slide imaging solutions. Despite indeed providing novel and helpful advantages, embracing a whole DP workflow is still challenging, especially for wide healthcare networks. The Azienda Zero of the Veneto Italian region has begun a process of a fully digital transformation of an integrated network of 12 hospitals producing nearly 3 million slides per year. In the present article, we describe the planning stages and the operative phases needed to support such a disruptive transition, along with the initial preliminary results emerging from the project. The ultimate goal of the DP program in the Veneto Italian region is to improve patients’ clinical care through a safe and standardized process, encompassing a total digital management of pathology samples, easy file sharing with experienced colleagues, and automatic support by artificial intelligence tools.

Lung and Colon Cancer Histopathological Image Classification Using 1D Convolutional Channel-based Attention Networks

Lung and Colon cancer are the leading diseases of death and disability in humans caused by a combination of genetic diseases and biochemical abnormalities. If these are diagnosed in their early stages, they can not be spread in organs and negatively impact human life. Many deep-learning networks have recently been proposed to detect and classify these malignancies. However, incorrect detection or misclassification of these fatal diseases can significantly affect an individual's health and well-being. This paper introduces a novel, cost-effective, and mobile-embedded architecture to diagnose and classify Lung squamous cell carcinomas and adenocarcinomas of the lung and colon from digital pathology images. Extensive experiment shows that our proposed modifications achieve 100% testing results for lung, colon, and lung-and-colon cancer detection. Our novel architecture takes around 0.65 million trainable parameters and around 6.4 million flops to achieve the best lung and colon cancer detection performance. Compared with the other results, our proposed architecture shows state-of-the-art performance.

Construction and validation of a deep learning prognostic model based on digital pathology images of stage III colorectal cancer

BackgroundTNM staging is the main reference standard for prognostic prediction of colorectal cancer (CRC), but the prognosis heterogeneity of patients with the same stage is still large. This study aimed to classify the tumor microenvironment of patients with stage III CRC and quantify the classified tumor tissues based on deep learning to explore the prognostic value of the developed tumor risk signature (TRS). MethodsA tissue classification model was developed to identify nine tissues (adipose, background, debris, lymphocytes, mucus, smooth muscle, normal mucosa, stroma, and tumor) in whole-slide images (WSIs) of stage III CRC patients. This model was used to extract tumor tissues from WSIs of 265 stage III CRC patients from The Cancer Genome Atlas and 70 stage III CRC patients from the Sixth Affiliated Hospital of Sun Yat-sen University. We used three different deep learning models for tumor feature extraction and applied a Cox model to establish the TRS. Survival analysis was conducted to explore the prognostic performance of TRS. ResultsThe tissue classification model achieved 94.4 % accuracy in identifying nine tissue types. The TRS showed a Harrell's concordance index of 0.736, 0.716, and 0.711 in the internal training, internal validation, and external validation sets. Survival analysis showed that TRS had significant predictive ability (hazard ratio: 3.632, p = 0.03) for prognostic prediction. ConclusionThe TRS is an independent and significant prognostic factor for PFS of stage III CRC patients and it contributes to risk stratification of patients with different clinical stages.

Global contextual representation via graph-transformer fusion for hepatocellular carcinoma prognosis in whole-slide images

Current methods of digital pathological images typically employ small image patches to learn local representative features to overcome the issues of computationally heavy and memory limitations. However, the global contextual features are not fully considered in whole-slide images (WSIs). Here, we designed a hybrid model that utilizes Graph Neural Network (GNN) module and Transformer module for the representation of global contextual features, called TransGNN. GNN module built a WSI-Graph for the foreground area of a WSI for explicitly capturing structural features, and the Transformer module through the self-attention mechanism implicitly learned the global context information. The prognostic markers of hepatocellular carcinoma (HCC) prognostic biomarkers were used to illustrate the importance of global contextual information in cancer histopathological analysis. Our model was validated using 362 WSIs from 355 HCC patients diagnosed from The Cancer Genome Atlas (TCGA). It showed impressive performance with a Concordance Index (C-Index) of 0.7308 (95% Confidence Interval (CI): (0.6283–0.8333)) for overall survival prediction and achieved the best performance among all models. Additionally, our model achieved an area under curve of 0.7904, 0.8087, and 0.8004 for 1-year, 3-year, and 5-year survival predictions, respectively. We further verified the superior performance of our model in HCC risk stratification and its clinical value through Kaplan–Meier curve and univariate and multivariate COX regression analysis. Our research demonstrated that TransGNN effectively utilized the context information of WSIs and contributed to the clinical prognostic evaluation of HCC.

Predicting rectal cancer prognosis from histopathological images and clinical information using multi-modal deep learning.

The objective of this study was to provide a multi-modal deep learning framework for forecasting the survival of rectal cancer patients by utilizing both digital pathological images data and non-imaging clinical data. The research included patients diagnosed with rectal cancer by pathological confirmation from January 2015 to December 2016. Patients were allocated to training and testing sets in a randomized manner, with a ratio of 4:1. The tissue microarrays (TMAs) and clinical indicators were obtained. Subsequently, we selected distinct deep learning models to individually forecast patient survival. We conducted a scanning procedure on the TMAs in order to transform them into digital pathology pictures. Additionally, we performed pre-processing on the clinical data of the patients. Subsequently, we selected distinct deep learning algorithms to conduct survival prediction analysis using patients' pathological images and clinical data, respectively. A total of 292 patients with rectal cancer were randomly allocated into two groups: a training set consisting of 234 cases, and a testing set consisting of 58 instances. Initially, we make direct predictions about the survival status by using pre-processed Hematoxylin and Eosin (H&E) pathological images of rectal cancer. We utilized the ResNest model to extract data from histopathological images of patients, resulting in a survival status prediction with an AUC (Area Under the Curve) of 0.797. Furthermore, we employ a multi-head attention fusion (MHAF) model to combine image features and clinical features in order to accurately forecast the survival rate of rectal cancer patients. The findings of our experiment show that the multi-modal structure works better than directly predicting from histopathological images. It achieves an AUC of 0.837 in predicting overall survival (OS). Our study highlights the potential of multi-modal deep learning models in predicting survival status from histopathological images and clinical information, thus offering valuable insights for clinical applications.

Prediction of immunochemotherapy response for diffuse large B-cell lymphoma using artificial intelligence digital pathology.

Diffuse large B-cell lymphoma (DLBCL) is a heterogeneous and prevalent subtype of aggressive non-Hodgkin lymphoma that poses diagnostic and prognostic challenges, particularly in predicting drug responsiveness. In this study, we used digital pathology and deep learning to predict responses to immunochemotherapy in patients with DLBCL. We retrospectively collected 251 slide images from 216 DLBCL patients treated with rituximab, cyclophosphamide, doxorubicin, vincristine, and prednisone (R-CHOP), with their immunochemotherapy response labels. The digital pathology images were processed using contrastive learning for feature extraction. A multi-modal prediction model was developed by integrating clinical data and pathology image features. Knowledge distillation was employed to mitigate overfitting on gigapixel histopathology images to create a model that predicts responses based solely on pathology images. Based on the importance derived from the attention mechanism of the model, we extracted histological features that were considered key textures associated with drug responsiveness. The multi-modal prediction model achieved an impressive area under the ROC curve of 0.856, demonstrating significant associations with clinical variables such as Ann Arbor stage, International Prognostic Index, and bulky disease. Survival analyses indicated their effectiveness in predicting relapse-free survival. External validation using TCGA datasets supported the model's ability to predict survival differences. Additionally, pathology-based predictions show promise as independent prognostic indicators. Histopathological analysis identified centroblastic and immunoblastic features to be associated with treatment response, aligning with previous morphological classifications and highlighting the objectivity and reproducibility of artificial intelligence-based diagnosis. This study introduces a novel approach that combines digital pathology and clinical data to predict the response to immunochemotherapy in patients with DLBCL. This model shows great promise as a diagnostic and prognostic tool for clinical management of DLBCL. Further research and genomic data integration hold the potential to enhance its impact on clinical practice, ultimately improving patient outcomes.