ProMIL: A weakly supervised multiple instance learning for whole slide image classification based on class proxy

Abstract

The histopathological analysis of a suspected region is critical for cancer diagnosis, treatment, and management. Histopathological diagnosis consists in analyzing the characteristics of the lesions using tissue sections stained with hematoxylin and eosin. Classification of digital tumor pathology images, called whole slide images (WSIs), is a great challenge since WSIs usually have huge resolutions while lacking localized annotations. Multiple instance learning (MIL) is a commonly used method applied to pathological image analysis. However, most MIL methods often focus only on the global representation of WSIs, ignoring whether the category labels play other roles in the model training besides being a supervision signal. In addition, feature confusion is also a problem that should be avoided for the analysis of WSIs with weakly supervised methods. To address these problems, we propose a novel algorithm of classifying WSI for cancer diagnosis. The proposed model, ProMIL, uses only slide-level labels rather than localized annotations for analysis. There are three innovations in this work. Firstly, we present the concept of class proxy which is the representation of the intrinsic feature of each category, and plays a key role in guiding the training of the model. Secondly, we design a novel WSI representation learning module that utilizes a multi-scale feature extraction strategy to represent each patch in a WSI and then aggregates these representations using an attention mechanism to encode the WSI. Thirdly, we design a metric-learning-based weakly supervised multiclass-classifier by measuring the similarity between each WSI embedding and class proxies. The proposed ProMIL can effectively alleviate the side effect of feature confusion, and carry intuitive interpretability and scalability. To evaluate the performance of ProMIL, we conduct a series of experiments on several datasets of WSIs with different types of cancer from open data sources. It can be observed from the experimental results that ProMIL outperforms most of the compared methods and achieves better performance on a various type of cancer image data for classification, thus suggesting the proposed method is suitable for classifying different categories of cancer rather than a specific kind of cancer. Therefore, it is expected to act as a general framework to be extended to more cancer diagnoses.