Multiple Instance Learning Methods Research Articles

Transformer based multiple superpixel-instance learning for weakly supervised segmenting lesions of interstitial lung disease

Automatic and accurate segmentation of lesions from high-resolution computed tomography (HRCT) images play a critical role in diagnosis of interstitial lung disease (ILD). Fully supervised methods require considerable expertise annotations, resulting in time-consuming and labor-intensive cost. Contrastively, weakly supervised segmentation (WSS) algorithms could alleviate greatly this issue of medical image annotation. Multiple-instance learning (MIL), a class of weakly supervised learning (WSL), has proven to be effective in image segmentation tasks. However, most existing MIL methods based on patches or pixels lack meaningful edge information of the region of interests (ROIs) and leave out of considering the dependence relations of inter-instances. In this study, we propose a novel Transformer based multiple superpixel-instance learning (MSIL) for pixel-level lesions segmentation of ILD. This method introduces superpixels into a MIL framework, which solves the problem of inconsistency in lesion boundary segmentation caused by using patches as instances. In addition, the proposed method introduces Transformer into the MIL framework to capture global or long-range dependencies and establish the relationship between superpixel instances based on the multi-head self-attention, which solves the shortcoming that instances are independent of each other in MIL. Experimental results on private HRCT images confirm that our proposed method can achieve state-of-the-art (SOTA) performance compared to other WSS algorithms, MIL-based WSS algorithms, CAMs-based classification algorithms (CA) and fully supervised segmentation algorithms (FSS).

Unsupervised mutual transformer learning for multi-gigapixel Whole Slide Image classification

The classification of gigapixel Whole Slide Images (WSIs) is an important task in the emerging area of computational pathology. There has been a surge of interest in deep learning models for WSI classification with clinical applications such as cancer detection or prediction of cellular mutations. Most supervised methods require expensive and labor-intensive manual annotations by expert pathologists. Weakly supervised Multiple Instance Learning (MIL) methods have recently demonstrated excellent performance; however, they still require large-scale slide-level labeled training datasets that require a careful inspection of each slide by an expert pathologist. In this work, we propose a fully unsupervised WSI classification algorithm based on mutual transformer learning. The instances (i.e., patches) from gigapixel WSIs are transformed into a latent space and then inverse-transformed to the original space. Using the transformation loss, pseudo labels are generated and cleaned using a transformer label cleaner. The proposed transformer-based pseudo-label generator and cleaner modules mutually train each other iteratively in an unsupervised manner. A discriminative learning mechanism is introduced to improve normal versus cancerous instance labeling. In addition to the unsupervised learning, we demonstrate the effectiveness of the proposed framework for weakly supervised learning and cancer subtype classification as downstream analysis. Extensive experiments on four publicly available datasets show better performance of the proposed algorithm compared to the existing state-of-the-art methods.

ProDiv: Prototype-driven consistent pseudo-bag division for whole-slide image classification

Background and ObjectivePathology image classification is one of the most essential auxiliary processes in cancer diagnosis. To overcome the problem of inadequate Whole-Slide Image (WSI) samples with weak labels, pseudo-bag-based multiple instance learning (MIL) methods have attracted wide attention in pathology image classification. In this type of method, the division scheme of pseudo-bags is usually a primary factor affecting classification performance. In order to improve the division of WSI pseudo-bags on existing random/clustering approaches, this paper proposes a new Prototype-driven Division (ProDiv) scheme for the pseudo-bag-based MIL classification framework on pathology images. MethodsThis scheme first designs an attention-based method to generate a bag prototype for each slide. On this basis, it further groups WSI patch instances into a series of instance clusters according to the feature similarities between the prototype and patches. Finally, pseudo-bags are obtained by randomly combining the non-overlapping patch instances of different instance clusters. Moreover, the design scheme of our ProDiv considers practicality, and it could be smoothly assembled with almost all the MIL-based WSI classification methods in recent years. ResultsEmpirical results show that our ProDiv, when integrated with several existing methods, can deliver classification AUC improvements of up to 7.3% and 10.3%, respectively on two public WSI datasets. ConclusionsProDiv could almost always bring obvious performance improvements to compared MIL models on typical metrics, which suggests the effectiveness of our scheme. Experimental visualization also visually interprets the correctness of the proposed ProDiv.

Boosting Multiple Instance Learning Models for Whole Slide Image Classification: A Model-Agnostic Framework Based on Counterfactual Inference

Multiple instance learning is an effective paradigm for whole slide image (WSI) classification, where labels are only provided at the bag level. However, instance-level prediction is also crucial as it offers insights into fine-grained regions of interest. Existing multiple instance learning methods either solely focus on training a bag classifier or have the insufficient capability of exploring instance prediction. In this work, we propose a novel model-agnostic framework to boost existing multiple instance learning models, to improve the WSI classification performance in both bag and instance levels. Specifically, we propose a counterfactual inference-based sub-bag assessment method and a hierarchical instance searching strategy to help to search reliable instances and obtain their accurate pseudo labels. Furthermore, an instance classifier is well-trained to produce accurate predictions. The instance embedding it generates is treated as a prompt to refine the instance feature for bag prediction. This framework is model-agnostic, capable of adapting to existing multiple instance learning models, including those without specific mechanisms like attention. Extensive experiments on three datasets demonstrate the competitive performance of our method. Code will be available at https://github.com/centurion-crawler/CIMIL.

Hyperbolic Secant representation of the logistic function: Application to probabilistic Multiple Instance Learning for CT intracranial hemorrhage detection

Multiple Instance Learning (MIL) is a weakly supervised paradigm that has been successfully applied to many different scientific areas and is particularly well suited to medical imaging. Probabilistic MIL methods, and more specifically Gaussian Processes (GPs), have achieved excellent results due to their high expressiveness and uncertainty quantification capabilities. One of the most successful GP-based MIL methods, VGPMIL, resorts to a variational bound to handle the intractability of the logistic function. Here, we formulate VGPMIL using Pólya-Gamma random variables. This approach yields the same variational posterior approximations as the original VGPMIL, which is a consequence of the two representations that the Hyperbolic Secant distribution admits. This leads us to propose a general GP-based MIL method that takes different forms by simply leveraging distributions other than the Hyperbolic Secant one. Using the Gamma distribution we arrive at a new approach that obtains competitive or superior predictive performance and efficiency. This is validated in a comprehensive experimental study including one synthetic MIL dataset, two well-known MIL benchmarks, and a real-world medical problem. We expect that this work provides useful ideas beyond MIL that can foster further research in the field.

A deep multiple-instance text binary classification for topic relevant content extraction on social media

Social media platforms have rich text data, which can be used in data mining and analysis. However, given the fact, the evolution speed of natural languages is rapid in social media, and data on social media is very noisy. This is a great challenge to the accuracy of data analysis. To overcome this problem, we propose a topic-relevant content extraction (TRCE) based on deep multiple instance classification, leveraging existing information and hierarchical relationships among texts under a thread on social media as weak supervision to extract topic-strong-relevant data and filter out noise accurately without manually labeling data. The proposed method introduces latent variables, Bernoulli distribution, and variational inference into multiple-instance learning (MIL) to generate pseudo labels. Then we employ a dual-stream neural network with a 3-stage training process to achieve training MIL end-to-end. Experimental results show TRCE has a significant improvement compared with other MIL methods. Meanwhile, it only has a little decrease compared with supervised text classification on accuracy and F1 score. Given the fact TRCE does not need manually labeled data at all, while supervised classification relies heavily on labeled data, TRCE is a competitive method to extracting topic-relevant data and filtering out noise on social media.

Self-supervised learning-based cervical cytology for the triage of HPV-positive women in resource-limited settings and low-data regime

Screening Papanicolaou test samples has proven to be highly effective in reducing cervical cancer-related mortality. However, the lack of trained cytopathologists hinders its widespread implementation in low-resource settings. Deep learning-assisted telecytology diagnosis emerges as an appealing alternative, but it requires the collection of large annotated training datasets, which is costly and time-consuming. In this paper, we demonstrate that the abundance of unlabeled images that can be extracted from Pap smear test whole slide images presents a fertile ground for self-supervised learning methods, yielding performance improvements compared to off-the-shelf pre-trained models for various downstream tasks. In particular, we propose Cervical Cell Copy-Pasting (C3P) as an effective augmentation method, which enables knowledge transfer from public and labeled single-cell datasets to unlabeled tiles. Not only does C3P outperforms naive transfer from single-cell images, but we also demonstrate its advantageous integration into multiple instance learning methods. Importantly, all our experiments are conducted on our introduced in-house dataset comprising liquid-based cytology Pap smear images obtained using low-cost technologies. This aligns with our long-term objective of deep learning-assisted telecytology for diagnosis in low-resource settings.

MIL-ViT: A multiple instance vision transformer for fundus image classification

Despite the great success of deep learning approaches, retinal disease classification is still challenging as the early-stage pathological regions of retinal diseases may be extremely tiny and subtle, which are difficult for networks to detect. The feature representations learnt by deep learning models focusing more on the local view may lead to indiscriminative semantic-level representation. On the contrary, if they focus more on the global semantic-level, they may ignore the discerning subtle local pathological regions. To address this issue, in this paper, we propose a hybrid framework, combining the strong global semantic representation learning capability of the vision Transformer (ViT) and the excellent capacity of local representation extraction from the conventional multiple instance learning (MIL). Particularly, a multiple instance vision Transformer (MIL-ViT) is implemented, where the vanilla ViT branch and the MIL branch generate semantic probability distributions separately, and a bag consistency loss is proposed to minimize the difference between them. Moreover, a calibrated attention mechanism is developed to embed the instance representation into the bag representation in our MIL-ViT. To further improve the feature representation capability for fundus images, we pre-train the vanilla ViT on a large-scale fundus image database. The experimental results validate that the generalization capability of the model using our pre-trained weights for fundus disease diagnosis is better than the one using ImageNet pre-trained weights. Extensive experiments on four publicly available benchmarks demonstrate that our proposed MIL-ViT outperforms latest fundus image classification methods, including various deep learning models and deep MIL methods. All our source code and pre-trained models are publicly available at https://github.com/greentreeys/MIL-VT.

Introducing instance label correlation in multiple instance learning. Application to cancer detection on histopathological images

In the last years, the weakly supervised paradigm of multiple instance learning (MIL) has become very popular in many different areas. A paradigmatic example is computational pathology, where the lack of patch-level labels for whole-slide images prevents the application of supervised models. Probabilistic MIL methods based on Gaussian Processes (GPs) have obtained promising results due to their excellent uncertainty estimation capabilities. However, these are general-purpose MIL methods that do not take into account one important fact: in (histopathological) images, the labels of neighboring patches are expected to be correlated. In this work, we extend a state-of-the-art GP-based MIL method, which is called VGPMIL-PR, to exploit such correlation. To do so, we develop a novel coupling term inspired by the statistical physics Ising model. We use variational inference to estimate all the model parameters. Interestingly, the VGPMIL-PR formulation is recovered when the weight that regulates the strength of the Ising term vanishes. The performance of the proposed method is assessed in two real-world problems of prostate cancer detection. We show that our model achieves better results than other state-of-the-art probabilistic MIL methods. We also provide different visualizations and analysis to gain insights into the influence of the novel Ising term. These insights are expected to facilitate the application of the proposed model to other research areas.

ProMIL: A weakly supervised multiple instance learning for whole slide image classification based on class proxy

The histopathological analysis of a suspected region is critical for cancer diagnosis, treatment, and management. Histopathological diagnosis consists in analyzing the characteristics of the lesions using tissue sections stained with hematoxylin and eosin. Classification of digital tumor pathology images, called whole slide images (WSIs), is a great challenge since WSIs usually have huge resolutions while lacking localized annotations. Multiple instance learning (MIL) is a commonly used method applied to pathological image analysis. However, most MIL methods often focus only on the global representation of WSIs, ignoring whether the category labels play other roles in the model training besides being a supervision signal. In addition, feature confusion is also a problem that should be avoided for the analysis of WSIs with weakly supervised methods. To address these problems, we propose a novel algorithm of classifying WSI for cancer diagnosis. The proposed model, ProMIL, uses only slide-level labels rather than localized annotations for analysis. There are three innovations in this work. Firstly, we present the concept of class proxy which is the representation of the intrinsic feature of each category, and plays a key role in guiding the training of the model. Secondly, we design a novel WSI representation learning module that utilizes a multi-scale feature extraction strategy to represent each patch in a WSI and then aggregates these representations using an attention mechanism to encode the WSI. Thirdly, we design a metric-learning-based weakly supervised multiclass-classifier by measuring the similarity between each WSI embedding and class proxies. The proposed ProMIL can effectively alleviate the side effect of feature confusion, and carry intuitive interpretability and scalability. To evaluate the performance of ProMIL, we conduct a series of experiments on several datasets of WSIs with different types of cancer from open data sources. It can be observed from the experimental results that ProMIL outperforms most of the compared methods and achieves better performance on a various type of cancer image data for classification, thus suggesting the proposed method is suitable for classifying different categories of cancer rather than a specific kind of cancer. Therefore, it is expected to act as a general framework to be extended to more cancer diagnoses.

Registration‐enhanced multiple instance learning for cervical cancer whole slide image classification

AbstractHistopathological diagnosis is the golden standard for cancer diagnosis and grading. The past decade has witnessed great successes of vision‐based deep learning in computer‐aided cancer diagnosis. However, fully‐supervised methods require collecting large amounts of annotated data, which is a major challenge. In this research, we propose RMIL, a novel registration‐enhanced multiple instance learning pipeline, for cervical cancer biopsy grading of whole slide images as well as patch‐level regions. It requires only slide‐level annotations and a small amount of patch‐level annotations for training. To enhance it, an efficient registration method is implemented to generate more labeled patches with discriminative local features as data augmentations. In addition, attention mechanisms and self‐supervised learning are introduced to enhance feature extraction. Experiments showed that its classification accuracy is increased by 11.11%, compared to the vanilla multiple instance learning methods. The method has great potential for transferring to other scenarios.

Iterative multiple instance learning for weakly annotated whole slide image classification

Objective. Whole slide images (WSIs) play a crucial role in histopathological analysis. The extremely high resolution of WSIs makes it laborious to obtain fine-grade annotations. Hence, classifying WSIs with only slide-level labels is often cast as a multiple instance learning (MIL) problem where a WSI is regarded as a bag and tiled into patches that are regarded as instances. The purpose of this study is to develop a novel MIL method for classifying WSIs with only slide-level labels in histopathology analysis. Approach. We propose a novel iterative MIL (IMIL) method for WSI classification where instance representations and bag representations are learned collaboratively. In particular, IMIL iteratively finetune the feature extractor with selected instances and corresponding pseudo labels generated by attention-based MIL pooling. Additionally, three procedures for robust training of IMIL are adopted: (1) the feature extractor is initialized by utilizing self-supervised learning methods on all instances, (2) samples for finetuning the feature extractor are selected according to the attention scores, and (3) a confidence-aware loss is applied for finetuning the feature extractor. Main results. Our proposed IMIL-SimCLR archives the optimal classification performance on Camelyon16 and KingMed-Lung. Compared with the baseline method CLAM, IMIL-SimCLR significantly outperforms it by 3.71% higher average area under curve (AUC) on Camelyon16 and 4.25% higher average AUC on KingMed-Lung. Additionally, our proposed IMIL-ImageNet achieve the optimal classification performance on TCGA-Lung with the average AUC of 96.55% and the accuracy of 96.76%, which significantly outperforms the baseline method CLAM by 1.65% higher average AUC and 2.09% higher average accuracy respectively. Significance. Experimental results on a public lymph node metastasis dataset, a public lung cancer diagnosis dataset and an in-house lung cancer diagnosis datasets show the effectiveness of our proposed IMIL method across different WSI classification tasks compared with other state-of-the-art MIL methods.

Prototypical multiple instance learning for predicting lymph node metastasis of breast cancer from whole-slide pathological images

Computerized identification of lymph node metastasis of breast cancer (BCLNM) from whole-slide pathological images (WSIs) can largely benefit therapy decision and prognosis analysis. Besides the general challenges of computational pathology, like extra-high resolution, very expensive fine-grained annotation, etc., two particular difficulties with this task lie in (1) modeling the significant inter-tumoral heterogeneity in BCLNM pathological images, and (2) identifying micro-metastases, i.e., metastasized tumors with tiny foci. Towards this end, this paper presents a novel weakly supervised method, termed as Prototypical Multiple Instance Learning (PMIL), to learn to predict BCLNM from WSIs with slide-level class labels only. PMIL introduces the well-established vocabulary-based multiple instance learning (MIL) paradigm into computational pathology, which is characterized by utilizing the so-called prototypes to model pathological data and construct WSI features. PMIL mainly consists of two innovatively designed modules, i.e., the prototype discovery module which acquires prototypes from training data by unsupervised clustering, and the prototype-based slide embedding module which builds WSI features by matching constitutive patches against the prototypes. Relative to existing MIL methods for WSI classification, PMIL has two substantial merits: (1) being more explicit and interpretable in modeling the inter-tumoral heterogeneity in BCLNM pathological images, and (2) being more effective in identifying micro-metastases. Evaluation is conducted on two datasets, i.e., the public Camelyon16 dataset and the Zbraln dataset created by ourselves. PMIL achieves an AUC of 88.2% on Camelyon16 and 98.4% on Zbraln (at 40x magnification factor), which consistently outperforms other compared methods. Comprehensive analysis will also be carried out to further reveal the effectiveness and merits of the proposed method.

Deep semi-supervised multiple instance learning with self-correction for DME classification from OCT images.

Supervised deep learning has achieved prominent success in various diabetic macular edema (DME) recognition tasks from optical coherence tomography (OCT) volumetric images. A common problematic issue that frequently occurs in this field is the shortage of labeled data due to the expensive fine-grained annotations, which increases substantial difficulty in accurate analysis by supervised learning. The morphological changes in the retina caused by DME might be distributed sparsely in B-scan images of the OCT volume, and OCT data is often coarsely labeled at the volume level. Hence, the DME identification task can be formulated as a multiple instance classification problem that could be addressed by multiple instance learning (MIL) techniques. Nevertheless, none of previous studies utilize unlabeled data simultaneously to promote the classification accuracy, which is particularly significant for a high quality of analysis at the minimum annotation cost. To this end, we present a novel deep semi-supervised multiple instance learning framework to explore the feasibility of leveraging a small amount of coarsely labeled data and a large amount of unlabeled data to tackle this problem. Specifically, we come up with several modules to further improve the performance according to the availability and granularity of their labels. To warm up the training, we propagate the bag labels to the corresponding instances as the supervision of training, and propose a self-correction strategy to handle the label noise in the positive bags. This strategy is based on confidence-based pseudo-labeling with consistency regularization. The model uses its prediction to generate the pseudo-label for each weakly augmented input only if it is highly confident about the prediction, which is subsequently used to supervise the same input in a strongly augmented version. This learning scheme is also applicable to unlabeled data. To enhance the discrimination capability of the model, we introduce the Student-Teacher architecture and impose consistency constraints between two models. For demonstration, the proposed approach was evaluated on two large-scale DME OCT image datasets. Extensive results indicate that the proposed method improves DME classification with the incorporation of unlabeled data and outperforms competing MIL methods significantly, which confirm the feasibility of deep semi-supervised multiple instance learning at a low annotation cost.

Continuation Multiple Instance Learning for Weakly and Fully Supervised Object Detection.

Weakly supervised object detection (WSOD) is a challenging task that requires simultaneously learning object detectors and estimating object locations under the supervision of image category labels. Many WSOD methods that adopt multiple instance learning (MIL) have nonconvex objective functions and, therefore, are prone to get stuck in local minima (falsely localize object parts) while missing full object extent during training. In this article, we introduce classical continuation optimization into MIL, thereby creating continuation MIL (C-MIL) with the aim to alleviate the nonconvexity problem in a systematic way. To fulfill this purpose, we partition instances into class-related and spatially related subsets and approximate MIL's objective function with a series of smoothed objective functions defined within the subsets. We further propose a parametric strategy to implement continuation smooth functions, which enables C-MIL to be applied to instance selection tasks in a uniform manner. Optimizing smoothed loss functions prevents the training procedure from falling prematurely into local minima and facilities learning full object extent. Extensive experiments demonstrate the superiority of CMIL over conventional MIL methods. As a general instance selection method, C-MIL is also applied to supervised object detection to optimize anchors/features, improving the detection performance with a significant margin.

Multi-instance positive and unlabeled learning with bi-level embedding

Multiple Instance Learning (MIL) is a widely studied learning paradigm which arises from real applications. Existing MIL methods have achieved prominent performances under the premise of plenty annotation data. Nevertheless, sufficient labeled data is often unattainable due to the high labeling cost. For example, the task in web image identification is to find similar samples among a large size of unlabeled dataset through a small number of provided target pictures. This leads to a particular scenario of Multiple Instance Learning with insufficient Positive and superabundant Unlabeled data (PU-MIL), which is a hot research topic in MIL recently. In this paper, we propose a novel method called Multiple Instance Learning with Bi-level Embedding (MILBLE) to tackle PU-MIL problem. Unlike other PU-MIL method using only simple single-level mapping, the bi-level embedding strategy are designed to customize specific mapping for positive and unlabeled data. It ensures the characteristics of key instance are not erased. Moreover, the weighting measure adopted in positive data can extracts the uncontaminated information of true positive instances without interference from negative ones. Finally, we minimize the classification error loss of mapped examples based on class-prior probability to train the optimal classifier. Experimental results show that our method has better performance than other state-of-the-art methods.

Lung cancer diagnosis using deep attention-based multiple instance learning and radiomics.

BackgroundEarly diagnosis of lung cancer is a key intervention for the treatment of lung cancer in which computer‐aided diagnosis (CAD) can play a crucial role. Most published CAD methods perform lung cancer diagnosis by classifying each lung nodule in isolation. However, this does not reflect clinical practice, where clinicians diagnose a patient based on a set of images of nodules, instead of looking at one nodule at a time. Besides, the low interpretability of the output provided by these methods presents an important barrier for their adoption.MethodIn this article, we treat lung cancer diagnosis as a multiple instance learning (MIL) problem, which better reflects the diagnosis process in the clinical setting and provides higher interpretability of the output. We selected radiomics as the source of input features and deep attention‐based MIL as the classification algorithm. The attention mechanism provides higher interpretability by estimating the importance of each instance in the set for the final diagnosis. To improve the model's performance in a small imbalanced dataset, we propose a new bag simulation method for MIL.Results and conclusionThe results show that our method can achieve a mean accuracy of 0.807 with a standard error of the mean (SEM) of 0.069, a recall of 0.870 (SEM 0.061), a positive predictive value of 0.928 (SEM 0.078), a negative predictive value of 0.591 (SEM 0.155), and an area under the curve (AUC) of 0.842 (SEM 0.074), outperforming other MIL methods. Additional experiments show that the proposed oversampling strategy significantly improves the model's performance. In addition, experiments show that our method provides a good indication of the importance of each nodule in determining the diagnosis, which combined with the well‐defined radiomic features, to make the results more interpretable and acceptable for doctors and patients.

A linear programming approach to multiple instance learning

Multiple instance learning (MIL) aims to classify objects with complex structures and covers a wide range of real-world data mining applications. In MIL, objects are represented by a bag of instances instead of a single instance, and class labels are provided only for the bags. Some of the earlier MIL methods focus on solving MIL problem under the standard MIL assumption, which requires at least one positive instance in positive bags and all remaining instances are negative. This study proposes a linear programming framework to learn instance level contributions to bag label without emposing the standart assumption. Each instance of a bag is mapped to a pseudo-class membership estimate and these estimates are aggregated to obtain the bag-level class membership in an optimization framework. A simple linear mapping enables handling various MIL assumptions with adjusting instance contributions. Our experiments with instance-dissimilarity based data representations verify the effectiveness of the proposed MIL framework. Proposed mathematical models can be solved efficiently in polynomial time.

Detection of Prostate Cancer in Whole-Slide Images Through End-to-End Training With Image-Level Labels.

Prostate cancer is the most prevalent cancer among men in Western countries, with 1.1 million new diagnoses every year. The gold standard for the diagnosis of prostate cancer is a pathologists' evaluation of prostate tissue. To potentially assist pathologists deep / learning / based cancer detection systems have been developed. Many of the state-of-the-art models are patch / based convolutional neural networks, as the use of entire scanned slides is hampered by memory limitations on accelerator cards. Patch-based systems typically require detailed, pixel-level annotations for effective training. However, such annotations are seldom readily available, in contrast to the clinical reports of pathologists, which contain slide-level labels. As such, developing algorithms which do not require manual pixel-wise annotations, but can learn using only the clinical report would be a significant advancement for the field. In this paper, we propose to use a streaming implementation of convolutional layers, to train a modern CNN (ResNet / 34) with 21 million parameters end-to-end on 4712 prostate biopsies. The method enables the use of entire biopsy images at high-resolution directly by reducing the GPU memory requirements by 2.4 TB. We show that modern CNNs, trained using our streaming approach, can extract meaningful features from high-resolution images without additional heuristics, reaching similar performance as state-of-the-art patch-based and multiple-instance learning methods. By circumventing the need for manual annotations, this approach can function as a blueprint for other tasks in histopathological diagnosis. The source code to reproduce the streaming models is available at https://github.com/DIAGNijmegen/ pathology-streaming-pipeline.

A comparative study of multiple instance learning methods for cancer detection using T-cell receptor sequences.

As a branch of machine learning, multiple instance learning (MIL) learns from a collection of labeled bags, each containing a set of instances. The learning process is weakly supervised due to ambiguous instance labels. Since its emergence, MIL has been applied to solve various problems including content-based image retrieval, object tracking/detection, and computer-aided diagnosis. In biomedical research, the use of MIL has been focused on medical image analysis and molecule activity prediction. We review and apply 16 methods to investigate the applicability of MIL to a novel biomedical application, cancer detection using T-cell receptor (TCR) sequences. This important application can be a viable approach for large-scale cancer screening, as TCRs can be easily profiled from a subject's peripheral blood. We consider two feasible data-generating mechanisms, and for the purpose of performance evaluation, we simulate data under each mechanism, where we vary potentially important factors to mimic realistic situations. We also apply the methods to sequencing data of ten cancer types from The Cancer Genome Atlas, as an early proof of concept for distinguishing tumor patients from healthy individuals via TCR sequencing of peripheral blood. We find that given an appropriate MIL method is used, satisfactory performance with Area Under the Receiver Operating Characteristic Curve above 80% can be achieved for five in the ten cancers. Based on our numerical results, we make suggestions about selection of a proper method and avoidance of any method with poor performance. We further point out directions of future research as well as identify a pressing need of new MIL methodologies for improved performance (for some cancer types) and more explainable outcomes.