Slide Images Research Articles

CAMIL: Channel attention based multiple instance learning for whole slide image classification

Abstract Motivation The classification task based on whole-slide images (WSIs) is a classic problem in computational pathology. Multiple Instance Learning (MIL) provides a robust framework for analyzing whole slide images with slide-level labels at gigapixel resolution. However, existing MIL models typically focus on modeling the relationships between instances while neglecting the variability across the channel dimensions of instances, which prevents the model from fully capturing critical information in the channel dimension. Results To address this issue, we propose a plug-and-play module called Multi-scale Channel Attention Block (MCAB), which models the interdependencies between channels by leveraging local features with different receptive fields. By alternately stacking four layers of Transformer and MCAB, we designed a channel attention based MIL model (CAMIL) capable of simultaneously modeling both inter-instance relationships and intra-channel dependencies. To verify the performance of the proposed CAMIL in classification tasks, several comprehensive experiments were conducted across three datasets: Camelyon16, TCGA-NSCLC, and TCGA-RCC. Empirical results demonstrate that, whether the feature extractor is pretrained on natural images or on WSIs, our CAMIL surpasses current state-of-the-art MIL models across multiple evaluation metrics. Availability All implementation code is available at https://github.com/maojy0914/CAMIL Supplementary information Supplementary data are available at Bioinformatics online.

Weakly supervised pathological differentiation of primary central nervous system lymphoma and glioblastoma on multi-site whole slide images.

Differentiating primary central nervous system lymphoma (PCNSL) and glioblastoma (GBM) is crucial because their prognosis and treatment differ substantially. Manual examination of their histological characteristics is considered the golden standard in clinical diagnosis. However, this process is tedious and time-consuming and might lead to misdiagnosis caused by morphological similarity between their histology and tumor heterogeneity. Existing research focuses on radiological differentiation, which mostly uses multi-parametric magnetic resonance imaging. By contrast, we investigate the pathological differentiation between the two types of tumors using whole slide images (WSIs) of postoperative formalin-fixed paraffin-embedded samples. To learn the specific and intrinsic histological feature representations from the WSI patches, a self-supervised feature extractor is trained. Then, the patch representations are fused by feeding into a weakly supervised multiple-instance learning model for the WSI classification. We validate our approach on 134 PCNSL and 526 GBM cases collected from three hospitals. We also investigate the effect of feature extraction on the final prediction by comparing the performance of applying the feature extractors trained on the PCNSL/GBM slides from specific institutions, multi-site PCNSL/GBM slides, and large-scale histopathological images. Different feature extractors perform comparably with the overall area under the receiver operating characteristic curve value exceeding 85% for each dataset and close to 95% for the combined multi-site dataset. Using the institution-specific feature extractors generally obtains the best overall prediction with both of the PCNSL and GBM classification accuracies reaching 80% for each dataset. The excellent classification performance suggests that our approach can be used as an assistant tool to reduce the pathologists' workload by providing an accurate and objective second diagnosis. Moreover, the discriminant regions indicated by the generated attention heatmap improve the model interpretability and provide additional diagnostic information.

Not seeing the trees for the forest. The impact of neighbours on graph-based configurations in histopathology

BackgroundDeep learning (DL) has set new standards in cancer diagnosis, significantly enhancing the accuracy of automated classification of whole slide images (WSIs) derived from biopsied tissue samples. To enable DL models to process these large images, WSIs are typically divided into thousands of smaller tiles, each containing 10–50 cells. Multiple Instance Learning (MIL) is a commonly used approach, where WSIs are treated as bags comprising numerous tiles (instances) and only bag-level labels are provided during training. The model learns from these broad labels to extract more detailed, instance-level insights. However, biopsied sections often exhibit high intra- and inter-phenotypic heterogeneity, presenting a significant challenge for classification. To address this, many graph-based methods have been proposed, where each WSI is represented as a graph with tiles as nodes and edges defined by specific spatial relationships.ResultsIn this study, we investigate how different graph configurations, varying in connectivity and neighborhood structure, affect the performance of MIL models. We developed a novel pipeline, K-MIL, to evaluate the impact of contextual information on cell classification performance. By incorporating neighboring tiles into the analysis, we examined whether contextual information improves or impairs the network’s ability to identify patterns and features critical for accurate classification. Our experiments were conducted on two datasets: COLON cancer and UCSB datasets.ConclusionsOur results indicate that while incorporating more spatial context information generally improves model accuracy at both the bag and tile levels, the improvement at the tile level is not linear. In some instances, increasing spatial context leads to misclassification, suggesting that more context is not always beneficial. This finding highlights the need for careful consideration when incorporating spatial context information in digital pathology classification tasks.

Leveraging explainable AI and large-scale datasets for comprehensive classification of renal histologic types

Recently, as the number of cancer patients has increased, much research is being conducted for efficient treatment, including the use of artificial intelligence in genitourinary pathology. Recent research has focused largely on the classification of renal cell carcinoma subtypes. Nonetheless, the broader categorization of renal tissue into non-neoplastic normal tissue, benign tumor and malignant tumor remains understudied. This gap in research can primarily be attributed to the limited availability of extensive datasets including benign tumor and normal tissue in addition to specific type of renal cell carcinoma, which hampers the ability to conduct comprehensive studies in these broader categories. This research introduces a model aimed at classifying renal tissue into three primary categories: normal (non-neoplastic), benign tumor, and malignant tumor. Utilizing digital pathology while slide images (WSIs) from nephrectomy specimens of 2,535 patients from multiple institutions, the model provides a foundational approach for distinguishing these key tissue types. The study utilized a dataset of 12,223 WSIs comprising 1,300 WSIs of normal tissue, 700 WSIs of benign tumors, and 10,223 WSIs of malignant tumors. Employing the ResNet-18 architecture and a Multiple Instance Learning approach, the model demonstrated high accuracy, with F1-scores of 0.934 (CI: 0.933–0.934) for normal tissue, 0.684 (CI: 0.682–0.687) for benign tumors, and 0.878 (CI: 0.877–0.879) for malignant tumors. The overall performance was also notable, achieving a weighted average F1-score of 0.879 (CI: 0.879–0.880) and a weighted average area under the receiver operating characteristic curve of 0.969 (CI: 0.969–0.969). This model significantly aids in the swift and accurate diagnosis of renal tissue, encompassing non-neoplastic normal tissue, benign tumor, and malignant tumor.

Colorectal cancer classification using weakly annotated whole slide images: Multiple instance learning optimization study.

Colorectal cancer (CRC) is considered one of the most deadly cancer types nowadays. It is rapidly increasing due to many factors, such as unhealthy lifestyles, water and food pollution, aging, and medical diagnosis development. Detecting CRC in its early stages can help stop its growth by providing the necessary treatments, thereby saving many people's lives. There are various tests that doctors can perform to diagnose CRC; however, biopsy using histopathological images is considered the "gold standard" for CRC diagnosis. Deep learning techniques can now be leveraged to build computer-aided diagnosis (CAD) systems that can affirm if an input sample shows any symptoms of cancer and determine its stage and location with an acceptable degree of confidence. In this research, we utilize deep learning to study the CRC classification problem using weakly annotated histopathological whole slide images (WSIs). We relax the constraints of the multiple instance learning (MIL) algorithm and primarily propose WSI-label prediction functions to be integrated with MIL, which significantly enhances the performance of WSI-level classification. We also applied efficient preprocessing techniques that output a computationally power-efficient dataset representation and performed multiple experiments to compose the most efficient CAD system. Our study introduces a notable improvement over the results obtained by the baseline research where we achieved an accuracy of 93.05% compared to 84.17%. Furthermore, our results using only the weakly annotated WSIs outperformed the baseline results that are based on performing initial pre-training using a strongly annotated part of the dataset.

Deep Learning for Classification of Inflammatory Bowel Disease Activity in Whole Slide Images of Colonic Histopathology.

Grading activity of inflammatory bowel disease (IBD) using standardized histopathological scoring systems remains challenging due to limited availability of pathologists with IBD expertise and inter-observer variability. In this study, a deep learning model was developed to classify activity grades in hematoxylin and eosin-stained whole slide images (WSIs) from patients with IBD, offering a robust approach for general pathologists. This study utilized 2,077 WSIs from 636 patients who visited Dartmouth-Hitchcock Medical Center in 2018 and 2019, scanned at 40× magnification (0.25 μm/pixel). Board-certified gastrointestinal pathologists categorized the WSIs into four activity classes: inactive, mildly active, moderately active, and severely active. A transformer-based model was developed and validated using five-fold cross-validation to classify IBD activity. Using HoVerNet, neutrophil distribution across activity grades was examined. Attention maps from the model highlighted areas contributing to its prediction. The model classified IBD activity with weighted averages of 0.871 [95% Confidence Interval (CI): 0.860-0.883] for the area under the curve, 0.695 [95% CI: 0.674-0.715] for precision, 0.697 [95% CI: 0.678-0.716] for recall, and 0.695 [95% CI: 0.674-0.714] for F1-score. Neutrophil distribution was significantly different across activity classes. Qualitative evaluation of attention maps by a gastrointestinal pathologist suggested their potential for improved interpretability. The model demonstrates robust diagnostic performance and could enhance consistency and efficiency in IBD activity assessment.

Artificial Intelligence for Predicting HER2 Status of Gastric Cancer Based on Whole-Slide Histopathology Images: A Retrospective Multicenter Study.

Human epidermal growth factor receptor 2 (HER2) positive gastric cancer (GC) shows a robust response to the combined therapy based HER2-targeted therapy. The application of these therapies is highly dependent on the evaluation of tumor HER2 status. However, there are many risks and challenges in HER2 assessment in GC. Therefore, an economically viable and readily available instrument is requisite for distinguishing HER2 status among patients diagnosed with GC. The study has innovatively developed a deep learning model, HER2Net, which can predict the HER2 status by quantitatively calculating the proportion of HER2 high-expression regions. The HER2Net is trained on an internal training set derived from 531 hematoxylin & eosin (H&E) whole slide images (WSI) of 520 patients. Subsequently, the performance of HER2Net is validated on an internal test set from 115 H&E WSI of 111 patients and an external multi-center test set from 102 H&E WSI of 101 patients. The HER2Net achieves an accuracy of 0.9043 on the internal test set, and an accuracy of 0.8922 on an external test set from multiple institutes. This discovery indicates that the HER2Net can potentially offer a novel methodology for the identification of HER2-positive GC.

Tissue concepts: Supervised foundation models in computational pathology.

Due to the increasing workload of pathologists, the need for automation to support diagnostic tasks and quantitative biomarker evaluation is becoming more and more apparent. Foundation models have the potential to improve generalizability within and across centers and serve as starting points for data efficient development of specialized yet robust AI models. However, the training of foundation models themselves is usually very expensive in terms of data, computation, and time. This paper proposes a supervised training method that drastically reduces these expenses. The proposed method is based on multi-task learning to train a joint encoder, by combining 16 different classification, segmentation, and detection tasks on a total of 912,000 patches. Since the encoder is capable of capturing the properties of the samples, we term it the Tissue Concepts encoder. To evaluate the performance and generalizability of the Tissue Concepts encoder across centers, classification of whole slide images from four of the most prevalent solid cancers - breast, colon, lung, and prostate - was used. The experiments show that the Tissue Concepts model achieve comparable performance to models trained with self-supervision, while requiring only 6% of the amount of training patches. Furthermore, the Tissue Concepts encoder outperforms an ImageNet pre-trained encoder on both in-domain and out-of-domain data. The pre-trained models and will be made available under https://github.com/FraunhoferMEVIS/MedicalMultitaskModeling.

Diagnosing Helicobacter pylori using autoencoders and limited annotations through anomalous staining patterns in IHC whole slide images.

This work addresses the detection of Helicobacter pylori (H. pylori) in histological images with immunohistochemical staining. This analysis is a time-demanding task, currently done by an expert pathologist that visually inspects the samples. Given the effort required to localize the pathogen in images, a limited number of annotations might be available in an initial setting. Our goal is to design an approach that, using a limited set of annotations, is capable of obtaining results good enough to be used as a support tool. We propose to use autoencoders to learn the latent patterns of healthy patches and formulate a specific measure of the reconstruction error of the image in HSV space. ROC analysis is used to set the optimal threshold of this measure and the percentage of positive patches in a sample that determines the presence of H. pylori. Our method has been tested on an own database of 245 whole slide images (WSI) having 117 cases without H. pylori and different density of the bacteria in the remaining ones. The database has 1211 annotated patches, with only 163 positive patches. This dataset of positive annotations was used to train a baseline thresholding and an SVM using the features of a pre-trained RedNet-18 and ViT models. A 10-fold cross-validation shows that our method has better performance with 91% accuracy, 86% sensitivity, 96% specificity and 0.97 AUC in the diagnosis of H. pylori . Unlike classification approaches, our shallow autoencoder with threshold adaptation for the detection of anomalous staining is able to achieve competitive results with a limited set of annotated data. This initial approach is good enough to be used as a guide for fast annotation of infected patches.

Study on the Transformation Process of Thyroid Fine-Needle Aspiration Liquid-Based Cytology to Whole-Slide Image.

Analyse and summarise the reasons for failure in the digital acquisition of thyroid liquid-based cytology (LBC) slides and the technical challenges, and explore methods to obtain reliable and reproducible whole digital slide images for clinical thyroid cytology. Use the glass slide scanning imaging system to acquire whole-slide image (WSI) of thyroid LBC in sdpc format through different. Statistical analysis was conducted on the different acquisition methods, the quality of the glass slides, clinical and pathological characteristics of the case, TBSRTC grading and the quality of WSI. The WSI obtained by different scanning methods showed a high level of consistency in quality (W = 0.325, p < 0.001), especially between fully automatic scanning with different focus densities (W = 0.9, p < 0.001). A total of 2114 images were obtained through different methods of multi-layer fusion and multi-point focusing scanning, with scan success rates of 100.0%, 100.0%, 100.0% and 23.6%, respectively. The correlation between the quality of thyroid LBC glass slides and the image quality of thyroid LBC WSI was statistically significant (p < 0.001). The correlation between TBSRTC grading and the quality of thyroid LBC digital WSI was statistically significant (p < 0.001). Although the quality of glass slides has a significant impact, the success rate and image quality of malignant tumour scanning are both high. Overall, the risk of missed diagnosis of malignant tumours is low. In the future, we also need to improve the performance and algorithm of the scanner in cases of sparse cells.

Neuromuscular junction visualization in paraffin-embedded thyroarytenoid muscle sections: Expanding options beyond frozen section analysis.

The current gold standard for immunofluorescent (IF) visualization of neuromuscular junctions (NMJs) in muscle utilizes frozen tissue sections with fluorescent conjugated antibodies to demarcate neurons and IF alpha-bungarotoxin (α-BTX) to demarcate motor endplates. Frozen tissue sectioning comes with inherent inescapable limitations, including cryosectioning artifact and limited sample shelf-life. However, a parallel approach to identify NMJs in paraffin-embedded tissue sections has not been previously described. Yucatan minipig thyroarytenoid (TA) muscle was harvested and prepared as 5-μm thick paraffin-embedded tissue sections. A variety of antibodies at various concentrations were selected to target nicotinic acetylcholine receptors, synaptic vesicles, and neurons. Neurofilament (NEFL, Invitrogen, 1:500) and synaptic vesicle glycoprotein (SV2, DSHB, 1:230) bound and demarcated the neurons and synaptic vesicles, respectively. Following consistent visualization of nerve tissue, rabbit anti-nicotinic acetylcholine receptor alpha-1 subunit (CHRNα1, Abcam, 1:500) was used to identify the acetylcholine receptors within motor endplates. Complete NMJ visualization was then achieved with an optimized protocol using primary antibodies to the neurofilament light chain, nerve synaptic vesicle glycoprotein 2, and the alpha 1 subunit of the nicotinic acetylcholine receptor. Slide imaging was performed with the Echo Revolve Microscope (40×). Herein, we describe a new methodology to visualize NMJs within paraffin-embedded TA muscle sections. Our protocol avoids the known limitations associated with cryosectioned samples and introduces a new neurolaryngologic research tool that utilizes the advantageous ability of paraffin-embedded sectioning to preserve tissue morphology. In conjunction with standard cryosectioned methods, the described paraffin-embedded protocol serves to enhance histological analysis of NMJs. NA.

Using digital pathology to standardize and automate histological evaluations of environmental samples

Abstract Histological evaluations of tissues are commonly used in environmental monitoring studies to assess the health and fitness status of populations or even whole ecosystems. Although traditional histology can be cost-effective, there is a shortage of proficient histopathologists and results can often be subjective between operators, leading to variance. Digital pathology is a powerful diagnostic tool that has already significantly transformed research in human health but has rarely been applied to environmental studies. Digital analyses of whole slide images introduce possibilities of highly standardized histopathological evaluations, as well as the use of artificial intelligence for novel analyses. Furthermore, incorporation of digital pathology into environmental monitoring studies using standardized bioindicator species or groups such as bivalves and fish can greatly improve the accuracy, reproducibility, and efficiency of the studies. This review aims to introduce readers to digital pathology and how it can be applied to environmental studies. This includes guidelines for sample preparation, potential sources of error, and comparisons to traditional histopathological analyses.

Weakly supervised deep learning-based classification for histopathology of gliomas: a single center experience

Multiple artificial intelligence systems have been created to facilitate accurate and prompt histopathological diagnosis of tumors using hematoxylin-eosin-stained slides. We aimed to investigate whether weakly supervised deep learning can aid in glioma diagnosis. We analyzed 472 whole slide images (WSIs) from 226 patients in West China Hospital (WCH) and 1604 WSIs from 880 patients in The Cancer Genome Atlas (TCGA). We utilized the OpenSlide library to load WSIs, segmented them into small patches using the DeepZoom module, and then normalized the color using the Reinhard method. A weakly supervised deep learning model was developed using ResNet-50 combined with an attention mechanism. We investigated the performance of the model by calculating area under the curve (AUC) in a ten-fold cross-validation setting. Heatmap visualizations showed the prediction mechanism of the model. The results were promising, with high AUC values for differentiating grades of astrocytomas, oligodendrogliomas, all gliomas, and glioma types in the TCGA dataset (0.9419, 0.8659, 0.9904, and 0.9298, respectively), and in the WCH cohort (0.9048, 0.7423, 0.9510, and 0.7098, respectively). The model demonstrated a strong ability to infer IDH status in the TCGA dataset (AUC = 0.9488). The weakly supervised deep learning model proved to be an effective and reliable tool for neuropathological diagnosis, making it an attractive auxiliary tool.

Prospective and retrospective analysis of whole slide images of sentinel and targeted lymph node frozen sections in breast cancer.

Different types of digital modalities are currently available for frozen section (FS) evaluation in surgical pathology practice. However, there are limited studies that demonstrate the potential of whole slide imaging (WSI) as a robust digital pathology option for FS FS diagnosis. In the current study, we compared the diagnostic accuracy achieved with WSI to that achieved with Light Microscopy (LM) for evaluating FSs of axillary sentinel lymph nodes (SLNs) and clipped lymph nodes (LNs) from breast cancer patients using two modalities. Initially, a retrospective analysis evaluated hematoxylin and eosin (H&E)-stained FSs of 109 SLNs using WSI followed by LM after a wash-out period ranging from 2 to 6 weeks. Subsequently, a prospective analysis assessed FSs of 132 SLNs using LM by first pathologist, then H&E-stained FSs were scanned and interpreted remotely in real-time by a different pathologist. In the retrospective analysis, the diagnostic accuracy utilizing WSI, ranged from 96% to 99%, exhibited similarity to those achieved with LM, ranging from 94% to 99%. Similarly, the prospective analysis also demonstrated comparable diagnostic accuracy between WSI (96.2%) and LM (97%). Pathologists in the retrospective study required an additional 0.8-5.4 minutes to render diagnoses using WSI compared to LM (p<0.0001). In the prospective study conducted two years later, pathologists only took slightly longer to provide WSI FS diagnoses (3.95 min) compared to LM (3.51 min) (P>0.05). In conclusion, our study indicated that WSI-based evaluation showed comparable diagnostic accuracy to LM for assessing LN FSs. Furthermore, the prospective study demonstrated the feasibility of real-time acquisition of high-quality WSIs for remote FS diagnosis of SLNs. These findings substantiate the promising potential of using WSIs of SLNs and clipped LNs in real time FS evaluation of breast cancer patients as a standard of care in surgical pathology practice.

GNCnn: A QuPath extension for glomerulosclerosis and glomerulonephritis characterization based on deep learning.

The digitalization of traditional glass slide microscopy into whole slide images has opened up new opportunities for pathology, such as the application of artificial intelligence techniques. Specialized software is necessary to visualize and analyze these images. One of these applications is QuPath, a popular bioimage analysis tool. This study proposes GNCnn, the first open-source QuPath extension specifically designed for nephropathology. It integrates deep learning models to provide nephropathologists with an accessible, automatic detector and classifier of glomeruli, the basic filtering units of the kidneys. The aim is to offer nephropathologists a freely available application to measure and analyze glomeruli to identify conditions such as glomerulosclerosis and glomerulonephritis. GNCnn offers a user-friendly interface that enables nephropathologists to detect glomeruli with high accuracy (Dice coefficient of 0.807) and categorize them as either sclerotic or non-sclerotic, achieving a balanced accuracy of 98.46%. Furthermore, it facilitates the classification of non-sclerotic glomeruli into 12 commonly diagnosed types of glomerulonephritis, with a top-3 balanced accuracy of 84.41%. GNCnn provides real-time updates of results, which are available at both the glomerulus and slide levels. This allows users to complete a typical analysis task without leaving the main application, QuPath. This tool is the first to integrate the entire workflow for the assessment of glomerulonephritis directly into the nephropathologists' workspace, accelerating and supporting their diagnosis.

Graph neural networks in histopathology: Emerging trends and future directions.

Histopathological analysis of whole slide images (WSIs) has seen a surge in the utilization of deep learning methods, particularly Convolutional Neural Networks (CNNs). However, CNNs often fail to capture the intricate spatial dependencies inherent in WSIs. Graph Neural Networks (GNNs) present a promising alternative, adept at directly modeling pairwise interactions and effectively discerning the topological tissue and cellular structures within WSIs. Recognizing the pressing need for deep learning techniques that harness the topological structure of WSIs, the application of GNNs in histopathology has experienced rapid growth. In this comprehensive review, we survey GNNs in histopathology, discuss their applications, and explore emerging trends that pave the way for future advancements in the field. We begin by elucidating the fundamentals of GNNs and their potential applications in histopathology. Leveraging quantitative literature analysis, we explore four emerging trends: Hierarchical GNNs, Adaptive Graph Structure Learning, Multimodal GNNs, and Higher-order GNNs. Through an in-depth exploration of these trends, we offer insights into the evolving landscape of GNNs in histopathological analysis. Based on our findings, we propose future directions to propel the field forward. Our analysis serves to guide researchers and practitioners towards innovative approaches and methodologies, fostering advancements in histopathological analysis through the lens of graph neural networks.

Inspect quantitative signals in placental histopathology: Computer-assisted multiple functional tissues identification through multi-model fusion and distillation framework.

Pathological analysis of placenta is currently a valuable tool for gaining insights into pregnancy outcomes. In placental histopathology, multiple functional tissues can be inspected as potential signals reflecting the transfer functionality between fetal and maternal circulations. However, the identification of multiple functional tissues is challenging due to (1) severe heterogeneity in texture, size and shape, (2) distribution across different scales and (3) the need for comprehensive assessment at the whole slide image (WSI) level. To solve aforementioned problems, we establish a brand new dataset and propose a computer-aided segmentation framework through multi-model fusion and distillation to identify multiple functional tissues in placental histopathologic images, including villi, capillaries, fibrin deposits and trophoblast aggregations. Specifically, we propose a two-stage Multi-model Fusion and Distillation (MMFD) framework. Considering the multi-scale distribution and heterogeneity of multiple functional tissues, we enhance the visual representation in the first stage by fusing feature from multiple models to boost the effectiveness of the network. However, the multi-model fusion stage contributes to extra parameters and a significant computational burden, which is impractical for recognizing gigapixels of WSIs within clinical practice. In the second stage, we propose straightforward plug-in feature distillation method that transfers knowledge from the large fused model to a compact student model. In self-collected placental dataset, our proposed MMFD framework demonstrates an improvement of 4.3% in mean Intersection over Union (mIoU) while achieving an approximate 50% increase in inference speed and utilizing only 10% of parameters and computational resources, compared to the parameter-efficient fine-tuned Segment Anything Model (SAM) baseline. Visualization of segmentation results across entire WSIs on unseen cases demonstrates the generalizability of our proposed MMFD framework. Besides, experimental results on a public dataset further prove the effectiveness of MMFD framework on other tasks. Our work can present a fundamental method to expedite quantitative analysis of placental histopathology.

CTUSurv: A Cell-aware Transformer-based Network with Uncertainty for Survival Prediction Using Whole Slide Images

CTUSurv: A Cell-aware Transformer-based Network with Uncertainty for Survival Prediction Using Whole Slide Images

Evaluating the pathological and clinical implications of errors made by an artificial intelligence colon biopsy screening tool

ObjectiveArtificial intelligence (AI) tools for histological diagnosis offer great potential to healthcare, yet failure to understand their clinical context is delaying adoption. IGUANA (Interpretable Gland-Graphs using a Neural Aggregator) is...

Self-interactive learning: Fusion and evolution of multi-scale histomorphology features for molecular traits prediction in computational pathology.

Predicting disease-related molecular traits from histomorphology brings great opportunities for precision medicine. Despite the rich information present in histopathological images, extracting fine-grained molecular features from standard whole slide images (WSI) is non-trivial. The task is further complicated by the lack of annotations for subtyping and contextual histomorphological features that might span multiple scales. This work proposes a novel multiple-instance learning (MIL) framework capable of WSI-based cancer morpho-molecular subtyping by fusion of different-scale features. Our method, debuting as Inter-MIL, follows a weakly-supervised scheme. It enables the training of the patch-level encoder for WSI in a task-aware optimisation procedure, a step normally not modelled in most existing MIL-based WSI analysis frameworks. We demonstrate that optimising the patch-level encoder is crucial to achieving high-quality fine-grained and tissue-level subtyping results and offers a significant improvement over task-agnostic encoders. Our approach deploys a pseudo-label propagation strategy to update the patch encoder iteratively, allowing discriminative subtype features to be learned. This mechanism also empowers extracting fine-grained attention within image tiles (the small patches), a task largely ignored in most existing weakly supervised-based frameworks. With Inter-MIL, we carried out four challenging cancer molecular subtyping tasks in the context of ovarian, colorectal, lung, and breast cancer. Extensive evaluation results show that Inter-MIL is a robust framework for cancer morpho-molecular subtyping with superior performance compared to several recently proposed methods, in small dataset scenarios where the number of available training slides is less than 100. The iterative optimisation mechanism of Inter-MIL significantly improves the quality of the image features learned by the patch embedded and generally directs the attention map to areas that better align with experts' interpretation, leading to the identification of more reliable histopathology biomarkers. Moreover, an external validation cohort is used to verify the robustness of Inter-MIL on molecular trait prediction.