Multiple Instance Learning Framework Research Articles

Efficient online surface defect detection using multiple instance learning

Artificial intelligence (AI)-empowered defect detection has emerged as a promising solution for enhancing quality control in manufacturing. While prevalent object detection-based methods have achieved competitive performance, they do carry inherent limitations that necessitate further refinement prior to their practical application in online surface defect detection. This study introduces an efficient online surface defect detection method that makes predictions on the presence of defects based on image-level labels. The method leverages the multiple instance learning (MIL) framework, and utilizes convolutional neural network (CNN) as feature extractor. Extensive experiments are conducted on two real-world datasets to evaluate the method with a custom CNN and Resnet50 as feature extractors (referred to as MIL-CNN and MIL-Resnet50). The results demonstrate the superiority of the proposed method compared with the well-established benchmark methods, especially highlighting the advantage of MIL-Resnet50. Without requiring fine-grained labeling, MIL-Resnet50 enhances F1-macro by 2.5% and 1.5% within the two datasets compared to the second-ranking. Notably, it excels in detecting small-object defects. It also exhibits advantages in terms of detection speed, and are lightweight, making it easy to deploy even in resource-limited scenarios. Additionally, MIL-Resnet50 exhibits the capability to provide approximate defect localization through feature maps. These findings highlight the significant potential of the proposed method within industrial applications

CaMIL: Causal Multiple Instance Learning for Whole Slide Image Classification

Whole slide image (WSI) classification is a crucial component in automated pathology analysis. Due to the inherent challenges of high-resolution WSIs and the absence of patch-level labels, most of the proposed methods follow the multiple instance learning (MIL) formulation. While MIL has been equipped with excellent instance feature extractors and aggregators, it is prone to learn spurious associations that undermine the performance of the model. For example, relying solely on color features may lead to erroneous diagnoses due to spurious associations between the disease and the color of patches. To address this issue, we develop a causal MIL framework for WSI classification, effectively distinguishing between causal and spurious associations. Specifically, we use the expectation of the intervention P(Y | do(X)) for bag prediction rather than the traditional likelihood P(Y | X). By applying the front-door adjustment, the spurious association is effectively blocked, where the intervened mediator is aggregated from patch-level features. We evaluate our proposed method on two publicly available WSI datasets, Camelyon16 and TCGA-NSCLC. Our causal MIL framework shows outstanding performance and is plug-and-play, seamlessly integrating with various feature extractors and aggregators.

Multiple instance learning framework can facilitate explainability in murmur detection.

Cardiovascular diseases (CVDs) account for a high fatality rate worldwide. Heart murmurs can be detected from phonocardiograms (PCGs) and may indicate CVDs. Still, they are often overlooked as their detection and correct clinical interpretation require expert skills. In this work, we aim to predict the presence of murmurs and clinical outcomes from multiple PCG recordings employing an explainable multitask model. Our approach consists of a two-stage multitask model. In the first stage, we predict the murmur presence in single PCGs using a multiple instance learning (MIL) framework. MIL also allows us to derive sample-wise classifications (i.e. murmur locations) while only needing one annotation per recording ("weak label") during training. In the second stage, we fuse explainable hand-crafted features with features from a pooling-based artificial neural network (PANN) derived from the MIL framework. Finally, we predict the presence of murmurs and the clinical outcome for a single patient based on multiple recordings using a simple feed-forward neural network. We show qualitatively and quantitatively that the MIL approach yields useful features and can be used to detect murmurs on multiple time instances and may thus guide a practitioner through PCGs. We analyze the second stage of the model in terms of murmur classification and clinical outcome. We achieved a weighted accuracy of 0.714 and an outcome cost of 13612 when using the PANN model and demographic features on the CirCor dataset (hidden test set of the George B. Moody PhysioNet challenge 2022, team "Heart2Beat", rank 12 / 40). To the best of our knowledge, we are the first to demonstrate the usefulness of MIL in PCG classification. Also, we showcase how the explainability of the model can be analyzed quantitatively, thus avoiding confirmation bias inherent to many post-hoc methods. Finally, our overall results demonstrate the merit of employing MIL combined with handcrafted features for the generation of explainable features as well as for a competitive classification performance.

Dy-MIL: dynamic multiple-instance learning framework for video anomaly detection

Dy-MIL: dynamic multiple-instance learning framework for video anomaly detection

Cognitive Refined Augmentation for Video Anomaly Detection in Weak Supervision.

Weakly supervised video anomaly detection is a methodology that assesses anomaly levels in individual frames based on labeled video data. Anomaly scores are computed by evaluating the deviation of distances derived from frames in an unbiased state. Weakly supervised video anomaly detection encounters the formidable challenge of false alarms, stemming from various sources, with a major contributor being the inadequate reflection of frame labels during the learning process. Multiple instance learning has been a pivotal solution to this issue in previous studies, necessitating the identification of discernible features between abnormal and normal segments. Simultaneously, it is imperative to identify shared biases within the feature space and cultivate a representative model. In this study, we introduce a novel multiple instance learning framework anchored on a memory unit, which augments features based on memory and effectively bridges the gap between normal and abnormal instances. This augmentation is facilitated through the integration of an multi-head attention feature augmentation module and loss function with a KL divergence and a Gaussian distribution estimation-based approach. The method identifies distinguishable features and secures the inter-instance distance, thus fortifying the distance metrics between abnormal and normal instances approximated by distribution. The contribution of this research involves proposing a novel framework based on MIL for performing WSVAD and presenting an efficient integration strategy during the augmentation process. Extensive experiments were conducted on benchmark datasets XD-Violence and UCF-Crime to substantiate the effectiveness of the proposed model.

Instance-level loss based multiple-instance learning framework for acoustic scene classification

An acoustic scene is inferred by detecting properties combining diverse sounds and acoustic environments. This study is intended to discover these properties effectively using multiple-instance learning (MIL). MIL, also known as a weakly supervised learning approach, is a strategy for extracting an instance vector from an audio chunk that composes an audio clip and utilizing these unlabeled instances to infer a scene corresponding to the input data. However, many studies pointed out an underestimation problem of MIL. In this study, we propose an enhanced MIL framework more suitable for ASC systems by defining instance-level labels and loss to extract and cluster instances effectively. Furthermore, we design a lightweight convolutional neural network named FUSE comprising frequency-, temporal-sided depthwise, and pointwise convolutional filters. Experimental results show that the confidence and proportion of positive instances significantly increase compared to vanilla MIL, overcoming the underestimation problem and improving the classification accuracy even higher than the supervised learning. The proposed system achieved a performance of 81.1%, 72.3%, and 58.3% on the TAU urban acoustic scenes 2019, 2020 mobile, and 2022 mobile datasets with 139 K parameters, respectively. In particular, it achieves the highest performance among the systems having under the 1 M parameters for the TAU urban acoustic scenes 2019 dataset.

PhenoSV: interpretable phenotype-aware model for the prioritization of genes affected by structural variants

Structural variants (SVs) represent a major source of genetic variation associated with phenotypic diversity and disease susceptibility. While long-read sequencing can discover over 20,000 SVs per human genome, interpreting their functional consequences remains challenging. Existing methods for identifying disease-related SVs focus on deletion/duplication only and cannot prioritize individual genes affected by SVs, especially for noncoding SVs. Here, we introduce PhenoSV, a phenotype-aware machine-learning model that interprets all major types of SVs and genes affected. PhenoSV segments and annotates SVs with diverse genomic features and employs a transformer-based architecture to predict their impacts under a multiple-instance learning framework. With phenotype information, PhenoSV further utilizes gene-phenotype associations to prioritize phenotype-related SVs. Evaluation on extensive human SV datasets covering all SV types demonstrates PhenoSV’s superior performance over competing methods. Applications in diseases suggest that PhenoSV can determine disease-related genes from SVs. A web server and a command-line tool for PhenoSV are available at https://phenosv.wglab.org.

AdvMIL: Adversarial multiple instance learning for the survival analysis on whole-slide images

The survival analysis on histological whole-slide images (WSIs) is one of the most important means to estimate patient prognosis. Although many weakly-supervised deep learning models have been developed for gigapixel WSIs, their potential is generally restricted by classical survival analysis rules and fully-supervised learning requirements. As a result, these models provide patients only with a completely-certain point estimation of time-to-event, and they could only learn from the labeled WSI data currently at a small scale. To tackle these problems, we propose a novel adversarial multiple instance learning (AdvMIL) framework. This framework is based on adversarial time-to-event modeling, and integrates the multiple instance learning (MIL) that is much necessary for WSI representation learning. It is a plug-and-play one, so that most existing MIL-based end-to-end methods can be easily upgraded by applying this framework, gaining the improved abilities of survival distribution estimation and semi-supervised learning. Our extensive experiments show that AdvMIL not only could often bring performance improvement to mainstream WSI survival analysis methods at a relatively low computational cost, but also enables these methods to effectively utilize unlabeled data via semi-supervised learning. Moreover, it is observed that AdvMIL could help improving the robustness of models against patch occlusion and two representative image noises. The proposed AdvMIL framework could promote the research of survival analysis in computational pathology with its novel adversarial MIL paradigm.

A Structure-aware Hierarchical Graph-based Multiple Instance Learning Framework for pT Staging in Histopathological Image.

Pathological primary tumor (pT) stage focuses on the infiltration degree of the primary tumor to surrounding tissues, which relates to the prognosis and treatment choices. The pT staging relies on the field-of-views from multiple magnifications in the gigapixel images, which makes pixel-level annotation difficult. Therefore, this task is usually formulated as a weakly supervised whole slide image (WSI) classification task with the slide-level label. Existing weakly-supervised classification methods mainly follow the multiple instance learning paradigm, which takes the patches from single magnification as the instances and extracts their morphological features independently. However, they cannot progressively represent the contextual information from multiple magnifications, which is critical for pT staging. Therefore, we propose a structure-aware hierarchical graph-based multi-instance learning framework (SGMF) inspired by the diagnostic process of pathologists. Specifically, a novel graph-based instance organization method is proposed, namely structure-aware hierarchical graph (SAHG), to represent the WSI. Based on that, we design a novel hierarchical attention-based graph representation (HAGR) network to capture the critical patterns for pT staging by learning cross-scale spatial features. Finally, the top nodes of SAHG are aggregated by a global attention layer for bag-level representation. Extensive studies on three large-scale multi-center pT staging datasets with two different cancer types demonstrate the effectiveness of SGMF, which outperforms state-of-the-art up to 5.6% in the F1 score.

Local Self-Attention based Hybrid Multiple Instance Learning for Partial Spoof Speech Detection

The development of speech synthesis technology has increased the attention towards the threat of spoofed speech. Although various high-performance spoofing countermeasures (CMs) have been proposed in recent years, a particular scenario is overlooked: partially-spoofed audio, where spoofed utterances may contain both spoofed and bona fide segments. Currently, the research on partially-spoofed speech detection is lacking. The existing methods either train with partially-spoofed speech at utterance level, resulting in gradient conflicting at the segment level, or directly train with segment level data, which requires segment labels that are difficult to obtain in practice. In this study, to better detect partially-spoofed speech when only utterance labels are available, we formulate partially-spoofed speech detection into a multiple instance learning (MIL) problem. The typical MIL uses a pooling layer to fuse patch scores as a whole, and we propose a hybrid MIL (H-MIL) framework based on max and log-sum-exp (LSE) pooling methods, which can learn better segment representations to improve partially-spoofed speech detection performance. Theoretical and experimental verification shows that H-MIL can effectively relieve the gradient conflicting and gradient vanishing problems. Besides, we analyze the local correlations between segments and introduce a local self-attention mechanism to enhance segment features, which further promotes the detection performance. In our experiments, we provide not only detection results at the segment and utterance levels, but also some detailed visualization analysis, including the effect of spoof ratio and cross-dataset detection. The experimental results demonstrate the effective detection performance of our method at both the utterance and segment levels, especially when dealing with low spoof ratio attacks. The results confirm that our approach can better deal with partially-spoofed speech detection than previous methods.

Multiresolution Self-Supervised Feature Integration via Attention Multiple Instance Learning for Histopathology Analysis.

Digital histopathology image analysis of tumor tissue sections has seen great research interest for automating standard diagnostic tasks, but also for developing novel prognostic biomarkers. However, research has mainly been focused on developing uniresolution models, capturing either high-resolution cellular features or low-resolution tissue architectural features. In addition, in the patch-based weakly-supervised training of deep learning models, the features which represent the intratumoral heterogeneity are lost. In this study, we propose a multiresolution attention-based multiple instance learning framework that can capture cellular and contextual features from the whole tissue for predicting patient-level outcomes. Several basic mathematical operations were examined for integrating multiresolution features, i.e. addition, mean, multiplication and concatenation. The proposed multiplication-based multiresolution model performed the best (AUC=0.864), while all multiresolution models outperformed the uniresolution baseline models (AUC=0.669, 0.713) for breast-cancer grading. (Implementation: https://github.com/tsikup/multiresolution-clam).

RLogist: Fast Observation Strategy on Whole-Slide Images with Deep Reinforcement Learning

Whole-slide images (WSI) in computational pathology have high resolution with gigapixel size, but are generally with sparse regions of interest, which leads to weak diagnostic relevance and data inefficiency for each area in the slide. Most of the existing methods rely on a multiple instance learning framework that requires densely sampling local patches at high magnification. The limitation is evident in the application stage as the heavy computation for extracting patch-level features is inevitable. In this paper, we develop RLogist, a benchmarking deep reinforcement learning (DRL) method for fast observation strategy on WSIs. Imitating the diagnostic logic of human pathologists, our RL agent learns how to find regions of observation value and obtain representative features across multiple resolution levels, without having to analyze each part of the WSI at the high magnification. We benchmark our method on two whole-slide level classification tasks, including detection of metastases in WSIs of lymph node sections, and subtyping of lung cancer. Experimental results demonstrate that RLogist achieves competitive classification performance compared to typical multiple instance learning algorithms, while having a significantly short observation path. In addition, the observation path given by RLogist provides good decision-making interpretability, and its ability of reading path navigation can potentially be used by pathologists for educational/assistive purposes. Our code is available at: https://github.com/tencent-ailab/RLogist.

Multi-modal deep learning improves grain yield prediction in wheat breeding by fusing genomics and phenomics.

Developing new crop varieties with superior performance is highly important to ensure robust and sustainable global food security. The speed of variety development is limited by long field cycles and advanced generation selections in plant breeding programs. While methods to predict yield from genotype or phenotype data have been proposed, improved performance and integrated models are needed. We propose a machine learning model that leverages both genotype and phenotype measurements by fusing genetic variants with multiple data sources collected by unmanned aerial systems. We use a deep multiple instance learning framework with an attention mechanism that sheds light on the importance given to each input during prediction, enhancing interpretability. Our model reaches 0.754 ± 0.024 Pearson correlation coefficient when predicting yield in similar environmental conditions; a 34.8% improvement over the genotype-only linear baseline (0.559 ± 0.050). We further predict yield on new lines in an unseen environment using only genotypes, obtaining a prediction accuracy of 0.386 ± 0.010, a 13.5% improvement over the linear baseline. Our multi-modal deep learning architecture efficiently accounts for plant health and environment, distilling the genetic contribution and providing excellent predictions. Yield prediction algorithms leveraging phenotypic observations during training therefore promise to improve breeding programs, ultimately speeding up delivery of improved varieties. Available at https://github.com/BorgwardtLab/PheGeMIL (code) and https://doi.org/doi:10.5061/dryad.kprr4xh5p (data).

Generalized pancreatic cancer diagnosis via multiple instance learning and anatomically-guided shape normalization.

Pancreatic cancer is a highly malignant cancer type with a high mortality rate. As no obvious symptoms are associated with this cancer type, most of the diagnoses are made when the patients are already in a late stage. In this work, we propose an automated method for effective early diagnosis of pancreatic cancer based on multiple instance learning with contrast-enhanced CT images. In this method, diagnosis stability and generalizability were improved through shape normalization based on anatomical structures as well as instance-level contrastive learning. Specifically, anatomically-guided shape normalization were developed to reconstruct the pancreatic regions of interest by spatial transformations, account for larger tumor parts in these regions, and hence enhance the extraction of pancreatic features. Moreover, instance-level contrastive learning was employed to aggregate different types of tumor features within the multiple instance learning framework. This learning approach can maintain the tumor feature integrity and enhance the diagnosis stability. Finally, a balance-adjustment strategy was designed to alleviate the class imbalance problem caused by the scarcity of tumor samples. Extensive experimental results demonstrated remarkable performance of our method when conducted cross-validation on an in-house dataset with 310 patients and independent test on two unseen datasets (a private test set with 316 and a publicly-available test set with 281). The proposed strategies also led to significant improvements in generalizability. Besides, the clinical significance of the proposed method was further verified through two independent test results in which tumors smaller than 2cm in diameter were identified at accuracies of 80.9% and 90.1%, respectively. Overall, our method provides a potentially successful tool for early diagnosis of pancreatic cancer. Our source codes will be released at https://github.com/SJTUBME-QianLab/MIL_PAdiagnosis.

Weakly supervised histopathology image segmentation with self-attention.

Accurate segmentation in histopathology images at pixel-level plays a critical role in the digital pathology workflow. The development of weakly supervised methods for histopathology image segmentation liberates pathologists from time-consuming and labor-intensive works, opening up possibilities of further automated quantitative analysis of whole-slide histopathology images. As an effective subgroup of weakly supervised methods, multiple instance learning (MIL) has achieved great success in histopathology images. In this paper, we specially treat pixels as instances so that the histopathology image segmentation task is transformed into an instance prediction task in MIL. However, the lack of relations between instances in MIL limits the further improvement of segmentation performance. Therefore, we propose a novel weakly supervised method called SA-MIL for pixel-level segmentation in histopathology images. SA-MIL introduces a self-attention mechanism into the MIL framework, which captures global correlation among all instances. In addition, we use deep supervision to make the best use of information from limited annotations in the weakly supervised method. Our approach makes up for the shortcoming that instances are independent of each other in MIL by aggregating global contextual information. We demonstrate state-of-the-art results compared to other weakly supervised methods on two histopathology image datasets. It is evident that our approach has generalization ability for the high performance on both tissue and cell histopathology datasets. There is potential in our approach for various applications in medical images.

Histopathological auxiliary system for brain tumour (HAS-Bt) based on weakly supervised learning using a WHO CNS5-style pipeline.

Classification and grading of central nervous system (CNS) tumours play a critical role in the clinic. When WHO CNS5 simplifies the histopathology diagnosis and places greater emphasis on molecular pathology, artificial intelligence (AI) has been widely used to meet the increased need for an automatic histopathology scheme that could liberate pathologists from laborious work. This study was to explore the diagnosis scope and practicality of AI. A one-stop Histopathology Auxiliary System for Brain tumours (HAS-Bt) is introduced based on a pipeline-structured multiple instance learning (pMIL) framework developed with 1,385,163 patches from 1038 hematoxylin and eosin (H&E) slides. The system provides a streamlined service including slide scanning, whole-slide image (WSI) analysis and information management. A logical algorithm is used when molecular profiles are available. The pMIL achieved an accuracy of 0.94 in a 9-type classification task on an independent dataset composed of 268 H&E slides. Three auxiliary functions are developed and a built-in decision tree with multiple molecular markers is used to automatically formed integrated diagnosis. The processing efficiency was 443.0s per slide. HAS-Bt shows outstanding performance and provides a novel aid for the integrated neuropathological diagnostic workflow of brain tumours using CNS 5 pipeline.

Robust fall detection in video surveillance based on weakly supervised learning

Fall event detection has been a research hotspot in recent years in the fields of medicine and health. Currently, vision-based fall detection methods have been considered the most promising methods due to their advantages of a non-contact characteristic and easy deployment. However, the existing vision-based fall detection methods mainly use supervised learning in model training and require much time and energy for data annotations. To address these limitations, this work proposes a detection method that uses a weakly supervised learning-based dual-modal network. The proposed method adopts a deep multiple instance learning framework to learn the fall events using weak labels. As a result, the proposed method does not require time-consuming fine-grained annotations. The final detection result of each video is obtained by integrating the information obtained from two streams of the dual-modal network using the proposed dual-modal fusion strategy. Experimental results on two public benchmark datasets and a proposed dataset demonstrate the superiority of the proposed method over the current state-of-the-art methods.

Abstract 5414: Automated detection of high-grade urothelial carcinoma from urine cytology slides using attention-based deep learning

Abstract Urine cytology has long been an effective and non-invasive test for the detection of bladder urothelial carcinomas (UC) routinely performed in cases of unexplained hematuria or for monitoring patients with UC. In cytopathology practice, urine cytology specimens are examined manually with a light microscope to identify morphologic features associated with different diagnostic categories based on the Paris System (TPS) for Reporting Urinary Cytology. Specifically, the diagnosis of high-grade urothelial carcinoma (HGUC) requires the identification of > 5-10 cells with a nuclear/cytoplasm ratio of 0.7 or greater and hyperchromasia together with coarse chromatin or irregular nuclear membranes. However, the task of identifying HGUC involves a substantial degree of manual review and is often associated with intra-and inter-observer variability. To address this, we have designed an accurate and efficient deep learning system capable of automatically distinguishing between HGUC and non-HGUC using digitized cytology slides. Our model has been developed using a retrospective cohort of 158 digitized urine ThinPrep cytology slides consisting of HGUC (n=98) and negative for HGUC (n=60). The model was then prospectively validated on a cohort of 105 urine cytology slides that were also independently reviewed prospectively in a blinded manner by a cytopathologist and cytotechnologist. Our system uses Otsu’s method for automatic image thresholding followed by dividing images into non-overlapping tiles of 500 × 500 pixels at the highest magnification. Subsequently, we use a pre-trained ResNet50 model to extract features which are used for training our attention-based multiple instance learning framework. For the training task, our retrospective cohort (158 slides) has been divided into 10 different splits each consisting of training (70%), validation (15%), and testing (15%) sets. The training and validation sets were used for the model training and optimalization, respectively, while the testing set was used for assessing the performance. This process yielded 10 different models with an average Area Under the ROC Curve (AUC) of 0.80 in the testing set. The best performing model had an AUC of 0.90 and an accuracy of 0.88. This model was subsequently validated prospectively in an independent testing cohort with 105 slides. In the prospective testing cohort, the model was able to accurately distinguish between HGUC and non-HGUC with an AUC of 0.83, accuracy of 0.76, sensitivity of 0.89, and specificity of 0.62. Additionally, our system can detect slide regions with high attention score for HGUC which are enriched in atypical urothelial cells. These findings show that our system can be utilized to assist cytopathologists in assessing urine cytology slides and to detect regions with high-diagnostic relevance for further assessment which is expected to reduce the time needed for manual review. Citation Format: Mohamed Omar, David Kim, Luigi Marchionni, Momin T. Siddiqui. Automated detection of high-grade urothelial carcinoma from urine cytology slides using attention-based deep learning. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 5414.

Abstract 5369: Detection of ERG:TMPRSS2 gene fusion in prostate cancer from histopathology slides using attention-based deep learning

Abstract ERG:TMPRSS2 fusion is present in almost 50% of prostate cancer (PCa) cases of European ascent and plays an important role in carcinogenesis and disease progression. ERG status is detected using fluorescence in situ hybridization (FISH) or reverse transcription-polymerase chain reaction (RT-PCR), and since these tests are costly and require special training, there is need for innovative tools to decrease the cost and streamline the diagnostic process. For this reason, we have developed a deep learning (DL) system capable of inferring ERG fusion status using only digitized hematoxylin and eosin (H&E)-stained slides from PCa patients, and detecting tissue regions of high diagnostic relevance. To develop the model, we used the PCa TCGA dataset which includes 436 formalin-fixed paraffin-embedded (FFPE) H&E-stained whole slide images (WSIs) from 393 PCa patients who underwent radical prostatectomy. Subsequently, we evaluated the model’s performance on an independent cohort of 314 WSIs provided by the Johns Hopkins University (natural history cohort). Slides were divided into tiles of 500 × 500 px from which feature extraction was performed using a pre-trained ResNet50 model. Feature vectors were then used to train attention-based multiple instance learning framework to predict the slide-level label as either ERG-positive or negative, and to score slide regions based on their contribution to the slide-level representation. Our model can predict the ERG fusion status with an Area Under the ROC Curve (AUC) of 0.84 and 0.73 in the training data and the independent testing cohort, respectively. Also, the model detects tissue regions with high attention score for each class. To decipher the cellular composition in these highly relevant regions for the cases predicted as ERG-positive or negative, we used HoVer-Net model to perform nuclear segmentation and classification into five categories: benign epithelial, tumor, stroma, immune, and necrotic. Notably, we found that the cellular composition of the highly relevant patches can capture prognostic information. Specifically, In the TCGA dataset, a high ratio of neoplastic cells in the relevant patches was significantly associated with worse progression free survival (PFS), while high ratios of necrotic, stromal, and stromal to neoplastic cells were significantly associated with better PFS. Similar findings were also obtained in the natural history cohort in which a high ratio of neoplastic cells was significantly associated with worse overall survival (OS) and metastasis free survival (MFS), while high ratios of immune, stromal, and stromal to neoplastic cells were significantly associated with longer OS and MFS. These results show that ERG fusion status can be inferred from H&E-stained WSIs, ultimately demonstrating the benefit of DL systems in extracting tissue morphological features of high diagnostic and prognostic relevance. Citation Format: Mohamed Omar, Zhuoran Xu, Sophie B. Rand, Daniela C. Salles, Edward M. Schaeffer, Tamara L. Lotan, Massimo Loda, Luigi Marchionni. Detection of ERG:TMPRSS2 gene fusion in prostate cancer from histopathology slides using attention-based deep learning. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 5369.

Convolutional Multiple Instance Learning for Sleep Spindle Detection With Label Refinement

Sleep spindles are closely associated with cognitive functions and neurological disorders; thus, spindle detection has been an important topic in sleep medicine. Recently, machine learning approaches have shown the potential in automatic sleep spindle detection by learning optimized features in a data-driven way, while they highly rely on labeled data, and the performance can be degraded when labels are inaccurate. However, accurate annotation of the spindle is usually difficult to obtain and high intraexpert and interexpert variance exist. In this work, we propose a convolutional neural network (CNN) with a label refinement component to learn an effective spindle detector with imperfect labels. Our approach consists of two stages: 1) a feature learning stage and 2) a label refinement stage. In the feature learning stage, a CNN-based multiple instance learning framework (CNN-MIL) is built for spindle feature learning. By assuming only parts of each labeled spindle segment contain true spindle patterns, the CNN-MIL model can learn most-likely spindle-related features from ambiguous labels. In the label refinement stage, we adjust the spindle labels by merging the original labels and labels predicted by CNN-MIL, and the modified labels are then used in the next round CNN-MIL feature learning. The two stages perform alternately for detector optimization. Extensive experiments demonstrated that our approach achieved the state-of-the-art performance.