Multi-scale Multi-instance Multi-feature Joint Learning Broad Network (M3JLBN) for gastric intestinal metaplasia subtype classification

Abstract

According to worldwide statistics, gastric intestinal metaplasia (GIM) is one of the most important characteristics for detecting the lesions in early gastric cancer. Currently, there are many detection methods employing multi-instance learning (MIL) on clinical medical images. For gastrointestinal endoscope (GE) images, there are very few existing works, which suffer from several issues: (i) only binary (i.e., healthy and GIM) cases are handled; (ii) the inter-instance correlation (i.e., the correlation between image patches or instances) cannot be captured; (iii) the multi-class label correlations among GIM subtype labels are neglected; (iv) multi-scale information is not considered. To address these issues, a novel Multi-scale Multi-instance Multi-feature Joint Learning Broad Network (M3JBLN) is proposed for practically required GIM subtype classification, which includes two new modules: (i) A multi-class MIL prediction module is designed based on probabilistic representation which can provide multi-class label correlations (pseudo label) through labeled multi-scaling training samples; (ii) A novel multiple features joint learning broad network (MFJLBN) based on broad learning system (BLS) is designed by integrating a new representation layer and multi-scaling module, which can jointly obtain inter-instance correlation in feature space under multiple scales. Under these two modules, the proposed M3JBLN jointly considers the multi-features of each instance at multiple scales to achieve more accurate subtype classification. The proposed M3JBLN is evaluated on a limited available GIM dataset acquired from patients who visited the Endoscopy Center of Kiang Wu Hospital (Macau, China) between January 2017 and April 25, 2021. Our method can respectively improve the performance with at least 7.5%, 7.6%, 7.6% and 6.6% under the four-evaluation metrics (accuracy, precision, recall and F1-score) compared to other classic methods.