Whole-slide Images Research Articles

Classifying driver mutations of papillary thyroid carcinoma on whole slide image: an automated workflow applying deep convolutional neural network

BackgroundInformative biomarkers play a vital role in guiding clinical decisions regarding management of cancers. We have previously demonstrated the potential of a deep convolutional neural network (CNN) for predicting cancer driver gene mutations from expert-curated histopathologic images in papillary thyroid carcinomas (PTCs). Recognizing the importance of whole slide image (WSI) analysis for clinical application, we aimed to develop an automated image preprocessing workflow that uses WSI inputs to categorize PTCs based on driver mutations.MethodsHistopathology slides from The Cancer Genome Atlas (TCGA) repository were utilized for diagnostic purposes. These slides underwent an automated tile extraction and preprocessing pipeline to ensure analysis-ready quality. Next, the extracted image tiles were utilized to train a deep learning CNN model, specifically Google’s Inception v3, for the classification of PTCs. The model was trained to distinguish between different groups based on BRAFV600E or RAS mutations.ResultsThe newly developed pipeline performed equally well as the expert-curated image classifier. The best model achieved Area Under the Curve (AUC) values of 0.86 (ranging from 0.847 to 0.872) for validation and 0.865 (ranging from 0.854 to 0.876) for the final testing subsets. Notably, it accurately predicted 90% of tumors in the validation set and 84.2% in the final testing set. Furthermore, the performance of our new classifier showed a strong correlation with the expert-curated classifier (Spearman rho = 0.726, p = 5.28 e-08), and correlated with the molecular expression-based classifier, BRS (BRAF-RAS scores) (Spearman rho = 0.418, p = 1.92e-13).ConclusionsUtilizing WSIs, we implemented an automated workflow with deep CNN model that accurately classifies driver mutations in PTCs.

Association of artificial intelligence-based immunoscore with the efficacy of chemoimmunotherapy in patients with advanced non-squamous non-small cell lung cancer: a multicentre retrospective study

PurposeCurrently, chemoimmunotherapy is effective only in a subset of patients with advanced non-squamous non-small cell lung cancer. Robust biomarkers for predicting the efficacy of chemoimmunotherapy would be useful to identify patients who would benefit from chemoimmunotherapy. The primary objective of our study was to develop an artificial intelligence-based immunoscore and to evaluate the value of patho-immunoscore in predicting clinical outcomes in patients with advanced non-squamous non-small cell lung cancer (NSCLC).MethodsWe have developed an artificial intelligence–powered immunoscore analyzer based on 1,333 whole-slide images from TCGA-LUAD. The predictive efficacy of the model was further validated in the CPTAC-LUAD cohort and the biomarker cohort of the ORIENT-11 study, a randomized, double-blind, phase 3 study. Finally, the clinical significance of the patho-immunoscore was evaluated using the ORIENT-11 study cohort.ResultsOur immunoscore analyzer achieved good accuracy in all the three cohort mentioned above (TCGA-LUAD, mean AUC: 0.783; ORIENT-11 cohort, AUC: 0.741; CPTAC-LUAD cohort, AUC: 0.769). In the 259 patients treated with chemoimmunotherapy, those with high patho-immunoscore (n = 146) showed significantly longer median progression-free survival than those with low patho-immunoscore (n = 113) (13.8 months vs 7.13 months, hazard ratio [HR]: 0.53, 95% confidence interval [CI]: 0.38 – 0.73; p < 0.001). In contrast, no significant difference was observed in patients who were treated with chemotherapy only (5.07 months vs 5.07 months, HR: 1.04, 95% CI: 0.71 – 1.54; p = 0.83). Similar trends were observed in overall survival.ConclusionOur study indicates that AI-powered immunoscore applied on LUAD digital slides can serve as a biomarker for survival outcomes in patients with advanced non-squamous NSCLC who received chemoimmunotherapy. This methodology could be applied to other cancers and facilitate cancer immunotherapy.

Current status and challenges in HER2 IHC assessment: scoring survey results in Japan.

This study aimed to assess the concordance of human epidermal growth factor receptor 2 (HER2) expression scoring by immunohistochemistry (IHC) among practicing pathologists in Japan, given the challenging nature of scoring and the critical role of HER2 status in breast cancer management. Whole slide images (WSI) from 20 invasive breast cancer cases (1 representative WSI per case) selected to represent a diverse IHC scores and staining patterns were used in an online survey involving seven reference pathologists who established consensus HER2 IHC scores (0 to 3 +) decided by majority interpretation. Participating pathologists nationwide scored the same 20 WSI cases online using the American Society of Clinical Oncology/College of American Pathologists (ASCO/CAP) 2018 guidelines. Deidentified case metadata were registered in the uPath system. A total of 144 participating pathologists responded. The scoring results of the participating pathologists most commonly agreed with the consensus IHC score, followed by a ± 1 point deviation and no survey responses with > 1 point deviation. The mean percentage of agreement with the consensus score for all 20 cases was 63.4%. In cases where the reference pathologists' scores were discordant, the participating pathologists also showed a lower concordance rate. This study highlighted the current status of HER2 expression scoring by IHC for breast cancer among pathologists in Japan. These findings underscore the challenges in HER2 IHC scoring cases and emphasize the need for improved standardization and training, especially in the evolving landscape of HER2-targeted therapies.

A deep learning approach for ovarian cancer detection and classification based on fuzzy deep learning

Different oncologists make their own decisions about the detection and classification of the type of ovarian cancer from histopathological whole slide images. However, it is necessary to have an automated system that is more accurate and standardized for decision-making, which is essential for early detection of ovarian cancer. To help doctors, an automated detection and classification of ovarian cancer system is proposed. This model starts by extracting the main features from the histopathology images based on the ResNet-50 model to detect and classify the cancer. Then, recursive feature elimination based on a decision tree is introduced to remove unnecessary features extracted during the feature extraction process. Adam optimizers were implemented to optimize the network’s weights during training data. Finally, the advantages of combining deep learning and fuzzy logic are combined to classify the images of ovarian cancer. The dataset consists of 288 hematoxylin and eosin (H&E) stained whole slides with clinical information from 78 patients. H&E-stained Whole Slide Images (WSIs), including 162 effective and 126 invalid WSIs were obtained from different tissue blocks of post-treatment specimens. Experimental results can diagnose ovarian cancer with a potential accuracy of 98.99%, sensitivity of 99%, specificity of 98.96%, and F1-score of 98.99%. The results show promising results indicating the potential of using fuzzy deep-learning classifiers for predicting ovarian cancer.

Multimodal Co-Attention Fusion Network With Online Data Augmentation for Cancer Subtype Classification.

It is an essential task to accurately diagnose cancer subtypes in computational pathology for personalized cancer treatment. Recent studies have indicated that the combination of multimodal data, such as whole slide images (WSIs) and multi-omics data, could achieve more accurate diagnosis. However, robust cancer diagnosis remains challenging due to the heterogeneity among multimodal data, as well as the performance degradation caused by insufficient multimodal patient data. In this work, we propose a novel multimodal co-attention fusion network (MCFN) with online data augmentation (ODA) for cancer subtype classification. Specifically, a multimodal mutual-guided co-attention (MMC) module is proposed to effectively perform dense multimodal interactions. It enables multimodal data to mutually guide and calibrate each other during the integration process to alleviate inter- and intra-modal heterogeneities. Subsequently, a self-normalizing network (SNN)-Mixer is developed to allow information communication among different omics data and alleviate the high-dimensional small-sample size problem in multi-omics data. Most importantly, to compensate for insufficient multimodal samples for model training, we propose an ODA module in MCFN. The ODA module leverages the multimodal knowledge to guide the data augmentations of WSIs and maximize the data diversity during model training. Extensive experiments are conducted on the public TCGA dataset. The experimental results demonstrate that the proposed MCFN outperforms all the compared algorithms, suggesting its effectiveness.

MTCSNet: One-Stage Learning and Two-Point Labeling are Sufficient for Cell Segmentation.

Deep convolution neural networks have been widely used in medical image analysis, such as lesion identification in whole-slide images, cancer detection, and cell segmentation, etc. However, it is often inevitable that researchers try their best to refine annotations so as to enhance the model performance, especially for cell segmentation task. Weakly supervised learning can greatly reduce the workload of annotations, while there is still a huge performance gap between the weakly and fully supervised learning approaches. In this work, we propose a weakly-supervised cell segmentation method, namely Multi-Task Cell Segmentation Network (MTCSNet), for multi-modal medical images, including pathological, brightfield, fluorescent, phase-contrast and differential interference contrast images. MTCSNet is learnt in a single-stage training manner, where only two annotated points for each cell provide supervision information, and the first one is the centroid, the second one is its boundary. Additionally, five auxiliary tasks are elaborately designed to train the network, including two pixel-level classifications, a pixel-level regression, a local temperature scaling and an instance-level distance regression task, which is proposed to regress the distances between the cell centroid and its boundaries in eight orientations. The experimental results indicate that our method outperforms all state-of-the-art weakly-supervised cell segmentation approaches on public multi-modal medical image datasets. The promising performance also shows that a single-stage learning with two-point labeling approach are sufficient for cell segmentation, instead of fine contour delineation. The codes are available at: https://github.com/binging512/MTCSNet.

Real-time segmentation and classification of whole-slide images for tumor biomarker scoring

Histopathology image segmentation and classification are essential for diagnosing and treating breast cancer. This study introduced a highly accurate segmentation and classification for histopathology images using a single architecture. We utilized the famous segmentation architectures, SegNet and U-Net, and modified the decoder to attach ResNet, VGG and DenseNet to perform classification tasks. These hybrid models are integrated with Stardist as the backbone, and implemented in a real-time pathologist workflow with a graphical user interface. These models were trained and tested offline using the ER-IHC-stained private and H&E-stained public datasets (MoNuSeg). For real-time evaluation, the proposed model was evaluated using PR-IHC-stained glass slides. It achieved the highest segmentation pixel-based F1-score of 0.902 and 0.903 for private and public datasets respectively, and a classification-based F1-score of 0.833 for private dataset. The experiment shows the robustness of our method where a model trained on ER-IHC dataset able to perform well on real-time microscopy of PR-IHC slides on both 20x and 40x magnification. This will help the pathologists with a quick decision-making process.

Utility of glass slide morphology (GSM) and whole slide image (WSI) in the diagnosis of acute leukemia (AL) by types

Utility of glass slide morphology (GSM) and whole slide image (WSI) in the diagnosis of acute leukemia (AL) by types

Artificial intelligence-based automated determination in breast and colon cancer and distinction between atypical and typical mitosis using a cloud-based platform

Artificial intelligence (AI) technology in pathology has been utilized in many areas and requires supervised machine learning. Notably, the annotations that define the ground truth for the identification of different confusing process pathologies, vary from study to study. In this study, we present our findings in the detection of invasive breast cancer for the IHC/ISH assessment system, along with the automated analysis of each tissue layer, cancer type, etc. in colorectal specimens. Additionally, models for the detection of atypical and typical mitosis in several organs were developed using existing whole-slide image (WSI) sets from other AI projects. All H&E slides were scanned by different scanners with a resolution of 0.12–0.50 μm/pixel, and then uploaded to a cloud-based AI platform. Convolutional neural networks (CNN) training sets consisted of invasive carcinoma, atypical and typical mitosis, and colonic tissue elements (mucosa-epithelium, lamina propria, muscularis mucosa, submucosa, muscularis propria, subserosa, vessels, and lymph nodes). In total, 59 WSIs from 59 breast cases, 217 WSIs from 54 colon cases, and 28 WSIs from 23 different types of tumor cases with relatively higher amounts of mitosis were annotated for the training. The harmonic average of precision and sensitivity was scored as F1 by AI. The final AI models of the Breast Project showed an F1 score of 94.49% for Invasive carcinoma. The mitosis project showed F1 scores of 80.18%, 97.40%, and 97.68% for mitosis, atypical, and typical mitosis layers, respectively. Overall F1 scores for the current results of the colon project were 90.02% for invasive carcinoma, 94.81% for the submucosa layer, and 98.02% for vessels and lymph nodes. After the training and optimization of the AI models and validation of each model, external validators evaluated the results of the AI models via blind-reader tasks. The AI models developed in this study were able to identify tumor foci, distinguish in situ areas, define colonic layers, detect vessels and lymph nodes, and catch the difference between atypical and typical mitosis. All results were exported for integration into our in-house applications for breast cancer and AI model development for both whole-block and whole-slide image-based 3D imaging assessment.

Classifying histopathological growth patterns for resected colorectal liver metastasis with a deep learning analysis.

Histopathological growth patterns are one of the strongest prognostic factors in patients with resected colorectal liver metastases. Development of an efficient, objective and ideally automated histopathological growth pattern scoring method can substantially help the implementation of histopathological growth pattern assessment in daily practice and research. This study aimed to develop and validate a deep-learning algorithm, namely neural image compression, to distinguish desmoplastic from non-desmoplastic histopathological growth patterns of colorectal liver metastases based on digital haematoxylin and eosin-stained slides. The algorithm was developed using digitalized whole-slide images obtained in a single-centre (Erasmus MC Cancer Institute, the Netherlands) cohort of patients who underwent first curative intent resection for colorectal liver metastases between January 2000 and February 2019. External validation was performed on whole-slide images of patients resected between October 2004 and December 2017 in another institution (Radboud University Medical Center, the Netherlands). The outcomes of interest were the automated classification of dichotomous hepatic growth patterns, distinguishing between desmoplastic hepatic growth pattern and non-desmoplatic growth pattern by a deep-learning model; secondary outcome was the correlation of these classifications with overall survival in the histopathology manual-assessed histopathological growth pattern and those assessed using neural image compression. Nine hundred and thirty-two patients, corresponding to 3.641 whole-slide images, were reviewed to develop the algorithm and 870 whole-slide images were used for external validation. Median follow-up for the development and the validation cohorts was 43 and 29 months respectively. The neural image compression approach achieved significant discriminatory power to classify 100% desmoplastic histopathological growth pattern with an area under the curve of 0.93 in the development cohort and 0.95 upon external validation. Both the histopathology manual-scored histopathological growth pattern and neural image compression-classified histopathological growth pattern achieved a similar multivariable hazard ratio for desmoplastic versus non-desmoplastic growth pattern in the development cohort (histopathology manual score: 0.63 versus neural image compression: 0.64) and in the validation cohort (histopathology manual score: 0.40 versus neural image compression: 0.48). The neural image compression approach is suitable for pathology-based classification tasks of colorectal liver metastases.

Deep Learning Predicts Lymphovascular Invasion Status in Muscle Invasive Bladder Cancer Histopathology.

Lymphovascular invasion (LVI) is linked to poor prognosis in patients with muscle-invasive bladder cancer (MIBC). Accurately identifying the LVI status in MIBC patients is crucial for effective risk stratification and precision treatment. We aim to develop a deep learning model to identify the LVI status in whole-slide images (WSIs) of MIBC patients. A cohort from The Cancer Genome Atlas (TCGA) database was used to train a deep learning model, slide-based lymphovascular invasion predictor (SBLVIP), based on multiple-instance learning. This model was externally validated using the Renmin Hospital of Wuhan University (RHWU) and People's Hospital of Hanchuan City (PHHC) cohorts. Kaplan-Meier curves, along with univariate and multivariate Cox models, were employed to evaluate the association between the LVI status predicted by SBLVIP and the survival outcomes of MIBC patients. In the TCGA cohort, the SBLVIP model achieved an average accuracy of 0.804 [95% confidence interval (CI) 0.712-0.895] and an area under the receiver operating characteristic curve (AUC) of 0.77 (95% CI 0.63-0.84) in the training set. In the internal validation set, the model's average accuracy and AUC were 0.774 (95% CI, 0.701-0.846) and 0.76 (95% CI, 0.60-0.83), respectively. In the RHWU cohort, the SBLVIP model achieved an average accuracy of 0.807 (95% CI 0.734-0.880) and an AUC of 0.74 (95% CI 0.55-0.83). In the PHHC cohort, SBLVIP demonstrated an average accuracy of 0.821 (95% CI 0.737-0.909) and an AUC of 0.74 (95% CI 0.58-0.89). Moreover, the LVI status predicted by SBLVIP showed significant independent prognostic value (P = 1 × 10-6). We developed a deep learning model named SBLVIP to predict the LVI status in routine WSIs of MIBC patients.

CytoNet: an efficient dual attention based automatic prediction of cancer sub types in cytology studies

Computer-assisted diagnosis (CAD) plays a key role in cancer diagnosis or screening. Whereas, current CAD performs poorly on whole slide image (WSI) analysis, and thus fails to generalize well. This research aims to develop an automatic classification system to distinguish between different types of carcinomas. Obtaining rich deep features in multi-class classification while achieving high accuracy is still a challenging problem. The detection and classification of cancerous cells in WSI are quite challenging due to the misclassification of normal lumps and cancerous cells. This is due to cluttering, occlusion, and irregular cell distribution. Researchers in the past mostly obtained the hand-crafted features while neglecting the above-mentioned challenges which led to a reduction of the classification accuracy. To mitigate this problem we proposed an efficient dual attention-based network (CytoNet). The proposed network is composed of two main modules (i) Efficient-Net and (ii) Dual Attention Module (DAM). Efficient-Net is capable of obtaining higher accuracy and enhancing efficiency as compared to existing Convolutional Neural Networks (CNNs). It is also useful to obtain the most generic features as it has been trained on ImageNet. Whereas DAM is very robust in obtaining attention and targeted features while negating the background. In this way, the combination of an efficient and attention module is useful to obtain the robust, and intrinsic features to obtain comparable performance. Further, we evaluated the proposed network on two well-known datasets (i) Our generated thyroid dataset (ii) Mendeley Cervical dataset (Hussain in Data Brief, 2019) with enhanced performance compared to their counterparts. CytoNet demonstrated a 99% accuracy rate on the thyroid dataset in comparison to its counterpart. The precision, recall, and F1-score values achieved on the Mendeley Cervical dataset are 0.992, 0.985, and 0.977, respectively. The code implementation is available on GitHub. https://github.com/naveedilyas/CytoNet-An-Efficient-Dual-Attention-based-Automatic-Prediction-of-Cancer-Sub-types-in-Cytol.

Deep Convolutional Neural Network for Accurate Classification of Myofibroblastic Lesions on Patch-Based Images.

This study aimed to implement and evaluate a Deep Convolutional Neural Network for classifying myofibroblastic lesions into benign and malignant categories based on patch-based images. A Residual Neural Network (ResNet50) model, pre-trained with weights from ImageNet, was fine-tuned to classify a cohort of 20 patients (11 benign and 9 malignant cases). Following annotation of tumor regions, the whole-slide images (WSIs) were fragmented into smaller patches (224 × 224 pixels). These patches were non-randomly divided into training (308,843 patches), validation (43,268 patches), and test (42,061 patches) subsets, maintaining a 78:11:11 ratio. The CNN training was caried out for 75 epochs utilizing a batch size of 4, the Adam optimizer, and a learning rate of 0.00001. ResNet50 achieved an accuracy of 98.97%, precision of 99.91%, sensitivity of 97.98%, specificity of 99.91%, F1 score of 98.94%, and AUC of 0.99. The ResNet50 model developed exhibited high accuracy during training and robust generalization capabilities in unseen data, indicating nearly flawless performance in distinguishing between benign and malignant myofibroblastic tumors, despite the small sample size. The excellent performance of the AI model in separating such histologically similar classes could be attributed to its ability to identify hidden discriminative features, as well as to use a wide range of features and benefit from proper data preprocessing.

Artificial Intelligence in Head and Neck Cancer Diagnosis: A Comprehensive Review with Emphasis on Radiomics, Histopathological, and Molecular Applications

The present review discusses the transformative role of AI in the diagnosis and management of head and neck cancers (HNCs). Methods: It explores how AI technologies, including ML, DL, and CNNs, are applied in various diagnostic tasks, such as medical imaging, molecular profiling, and predictive modeling. Results: This review highlights AI’s ability to improve diagnostic accuracy and efficiency, particularly in analyzing medical images like CT, MRI, and PET scans, where AI sometimes outperforms human radiologists. This paper also emphasizes AI’s application in histopathology, where algorithms assist in whole-slide image (WSI) analysis, tumor-infiltrating lymphocytes (TILs) quantification, and tumor segmentation. AI shows promise in identifying subtle or rare histopathological patterns and enhancing the precision of tumor grading and treatment planning. Furthermore, the integration of AI with molecular and genomic data aids in mutation analysis, prognosis, and personalized treatment strategies. Conclusions: Despite these advancements, the review identifies challenges in AI adoption, such as data standardization and model interpretability, and calls for further research to fully integrate AI into clinical practice for improved patient outcomes.

Development and Evaluation of a Convolutional Neural Network for Microscopic Diagnosis Between Pleomorphic Adenoma and Carcinoma Ex-Pleomorphic Adenoma.

To develop a model capable of distinguishing carcinoma ex-pleomorphic adenoma from pleomorphic adenoma using a convolutional neural network architecture. A cohort of 83 Brazilian patients, divided into carcinoma ex-pleomorphic adenoma (n = 42) and pleomorphic adenoma (n = 41), was used for training a convolutional neural network. The whole-slide images were annotated and fragmented into 743 869 (carcinoma ex-pleomorphic adenomas) and 211 714 (pleomorphic adenomas) patches, measuring 224 × 224 pixels. Training (80%), validation (10%), and test (10%) subsets were established. The Residual Neural Network (ResNet)-50 was chosen for its recognition and classification capabilities. The training and validation graphs, and parameters derived from the confusion matrix, were evaluated. The loss curve recorded 0.63, and the accuracy reached 0.93. Evaluated parameters included specificity (0.88), sensitivity (0.94), precision (0.96), F1 score (0.95), and area under the curve (0.97). The study underscores the potential of ResNet-50 in the microscopic diagnosis of carcinoma ex-pleomorphic adenoma. The developed model demonstrated strong learning potential, but exhibited partial limitations in generalization, as indicated by the validation curve. In summary, the study established a promising baseline despite limitations in model generalization. This indicates the need to refine methodologies, investigate new models, incorporate larger datasets, and encourage inter-institutional collaboration for comprehensive studies in salivary gland tumors.

Artificial Intelligence-Based Sentinel Lymph Node Metastasis Detection in Cervical Cancer

Background/objectives: Pathological ultrastaging, an essential part of sentinel lymph node (SLN) mapping, involves serial sectioning and immunohistochemical (IHC) staining in order to reliably detect clinically relevant metastases. However, ultrastaging is labor-intensive, time-consuming, and costly. Deep learning algorithms offer a potential solution by assisting pathologists in efficiently assessing serial sections for metastases, reducing workload and costs while enhancing accuracy. This proof-of-principle study evaluated the effectiveness of a deep learning algorithm for SLN metastasis detection in early-stage cervical cancer. Methods: We retrospectively analyzed whole slide images (WSIs) of hematoxylin and eosin (H&E)-stained SLNs from early-stage cervical cancer patients diagnosed with an SLN metastasis with either H&E or IHC. A CE-IVD certified commercially available deep learning algorithm, initially developed for detection of breast and colon cancer lymph node metastases, was employed off-label to assess its sensitivity in cervical cancer. Results: This study included 21 patients with early-stage cervical cancer, comprising 15 with squamous cell carcinoma, five with adenocarcinoma, and one with clear cell carcinoma. Among these patients, 10 had macrometastases and 11 had micrometastases in at least one SLN. The algorithm was applied to evaluate H&E WSIs of 47 SLN specimens, including 22 that were negative for metastasis, 13 with macrometastases, and 12 with micrometastases in the H&E slides. The algorithm detected all H&E macro- and micrometastases with 100% sensitivity. Conclusions: This proof-of-principle study demonstrated high sensitivity of a deep learning algorithm for detection of clinically relevant SLN metastasis in early-stage cervical cancer, despite being originally developed for adenocarcinomas of the breast and colon. Our findings highlight the potential of leveraging an existing algorithm for use in cervical cancer, warranting further prospective validation in a larger population.

Computational approach for counting of SISH amplification signals for HER2 status assessment

The human epidermal growth factor receptor 2 (HER2) gene is a critical biomarker for determining amplification status and targeting clinical therapies in breast cancer treatment. This study introduces a computer-aided method that automatically measures and scores HER2 gene status from invasive tissue regions of breast cancer using whole slide images (WSI) through silver in situ hybridization (SISH) staining. Image processing and deep learning techniques are employed to isolate untruncated and non-overlapping single nuclei from cancer regions. The Stardist deep learning model is fine-tuned on our HER2-SISH data to identify nuclei regions, followed by post-processing based on identified HER2 and CEP17 signals. Conventional thresholding techniques are used to segment HER2 and CEP17 signals. HER2 amplification status is determined by calculating the HER2-to-CEP17 signal ratio, in accordance with ASCO/CAP 2018 standards. The proposed method significantly reduces the effort and time required for quantification. Experimental results demonstrate a 0.91% correlation coefficient between pathologists manual enumeration and the proposed automatic SISH quantification approach. A one-sided paired t-test confirmed that the differences between the outcomes of the proposed method and the reference standard are statistically insignificant, with p-values exceeding 0.05. This study illustrates how deep learning can effectively automate HER2 status determination, demonstrating improvements over current manual methods and offering a robust, reproducible alternative for clinical practice.

Whole slide cervical cancer classification via graph attention networks and contrastive learning

Cervical cancer is one of the most common cancers among women, which seriously threatens women’s health. Early screening can reduce the incidence rate and mortality. Thinprep cytologic test (TCT) is one of the important means of cytological screening, which has high sensitivity and specificity, and has been widely used in the early screening of cervical cancer. The automatic diagnosis of whole slide images (WSIs) by computers can effectively improve the efficiency and accuracy of doctors’ diagnoses. However, current methods ignore the intrinsic relationships between cervical cells in WSIs and neglect contextual information from the surrounding suspicious areas, and therefore limit their robustness and generalizability. In this paper, we propose a novel two-stage method to implement the automatic diagnosis of WSIs, which constructs Graph Attention Networks (GAT) based on local and global fields respectively to capture their contextual information in a hierarchical manner. In the first stage, we extract representative patches from each WSI through suspicious cell detection, and then employ a Local GAT to classify cervical cells by capturing correlations between suspicious cells in image tiles. This classification process provides the confidence and feature vectors for each suspicious cell. In the second stage, we perform WSI classification using a Global GAT model. We construct graphs corresponding to top-Kg and bottom-Kg cells for each WSI based on results from Local GAT, and introduce a supervised contrastive learning strategy to enhance the discriminative power of the extracted features. Experimental results demonstrate that our proposed method outperforms conventional approaches and effectively showcases the benefits of supervised contrastive learning. Our source code and example data are available at https://github.com/feimanman/Whole-Slide-Cervical-Cancer-Classification.

A multi-model approach integrating whole-slide imaging and clinicopathologic features to predict breast cancer recurrence risk.

Breast cancer is the most common malignancy affecting women worldwide and is notable for its morphologic and biologic diversity, with varying risks of recurrence following treatment. The Oncotype DX Breast Recurrence Score test is an important predictive and prognostic genomic assay for estrogen receptor positive/HER2 negative breast cancer that guides therapeutic strategies; however, such tests can be expensive, delay care, and are not widely available. The aim of this study was to develop a multi-model approach integrating the analysis of whole-slide images and clinicopathologic data to predict their associated breast cancer recurrence risks and categorize these patients into two risk groups according to the predicted score: low-risk and high-risk. The proposed novel methodology uses convolutional neural networks for feature extraction and vision transformers for contextual aggregation, complemented by a logistic regression model that analyzes clinicopathologic data for classification into two risk categories. This method was trained and tested on 956 hematoxylin and eosin-stained whole-slide images of 950 ER+/HER2- breast cancer patients with corresponding clinicopathological features that had prior Oncotype DX testing. The model's performance was evaluated using an internal test set of 192 patients from Dartmouth Health and an external test set of 405 patients from the University of Chicago. The multi-model approach achieved an AUC of 0.91 (95% CI: 0.87-0.95) on the internal set and an AUC of 0.84 (95% CI: 0.78-0.89) on the external cohort for predicting low- and high-breast cancer recurrence risk categories based on the Oncotype DX recurrence score. With further validation, the proposed methodology could provide an alternative to assist clinicians in personalizing treatment for breast cancer patients and potentially improving their outcomes.

Comparative analysis of tertiary lymphoid structures for predicting survival of colorectal cancer: A whole-slide images-based study

Abstract Background Tertiary lymphoid structures (TLS) are major component in immune microenvironment, correlating with a favorable prognosis in colorectal cancer. However, the methods used to define and characterize TLS were not united, hindering its's clinical application. This study aims to seek a more stable method to characterize TLS and clarify its prognostic value in larger multicenter cohorts. Methods A total of 1609 patients from four hospitals and the TCGA database were analyzed. We quantified the number and maximum length of TLS along the invasive margin of tumor using hematoxylin and eosin (HE)-stained whole-slide images (WSIs). Additionally, the length of the invasive margin was determined to calculate the TLS density. The prognostic value of the TLS for overall survival was evaluated. In addition, we examined the association between TLS density and immune cell infiltration using immunohistochemistry-stained WSIs. The performance for predicting overall survival was measured using hazard ratios (HR) with 95% confidence intervals (CI). Results Among the three TLS quantification methods, TLS density has the strongest discriminative performance. Survival analysis indicated that higher TLS density correlated with better overall survival (HR for high vs. low 0.57 [95% CI 0.42–0.78] in primary cohort; 0.49 [0.35–0.69] in validation cohort; 0.35 [0.18–0.67] in TCGA cohort). A high TLS density was associated with a high level of CD3+ T cells infiltration. Conclusions Based on this comparative multicenter analysis, TLS density was identified as a simple, robust, and effective immune prognostic index for colorectal cancer.