MF-OMKT: Model fusion based on online mutual knowledge transfer for breast cancer histopathological image classification

Abstract

Pathological diagnosis is considered as the benchmark for the detection of breast cancer. With the increasing number of patients, computer-aided histopathological image classification can assist pathologists in improving breast cancer diagnosis accuracy and working efficiency. However, a single model is insufficient for effective diagnosis, and this also does not accord with the principle of centralized decision-making. Starting from the real pathological diagnosis scenario, we propose a novel model fusion framework based on online mutual knowledge transfer (MF-OMKT) for breast cancer histopathological image classification. The OMKT part based on deep mutual learning (DML) imitates the mutual communication and learning between multiple experienced pathologists, which can break the isolation of single models and provides sufficient complementarity among heterogeneous networks for MF. The MF part based on adaptive feature fusion uses the complementarity to train a powerful fusion classifier. MF imitates the centralized decision-making process of these pathologists. We used the MF-OMKT model to classify breast cancer histopathological images (BreakHis dataset) into benign and malignant as well as eight subtypes. The accuracy of our model reaches the range of [99.27 %, 99.84 %] for binary classification. And that for multi-class classification reaches the range of [96.14 %, 97.53 %]. Additionally, MF-OMKT is applied to the classification of skin cancer images (ISIC 2018 dataset) and achieves an accuracy of 94.90 %. MF-OMKT is an effective and versatile framework for medical image classification.