MRCG: A MRI Retrieval Framework With Convolutional and Graph Neural Networks for Secure and Private IoMT.

Abstract

In the context of Industry 4.0, the medical industry is horizontally integrating the medical resources of the entire industry through the Internet of Things (IoT) and digital interconnection technologies. Speeding up the establishment of the public retrieval database of diagnosis-related historical data is a common call for the entire industry. Among them, the Magnetic Resonance Imaging (MRI) retrieval system, which is one of the key tools for secure and private the Internet of Medical Things (IoMT), is significant for patients to check their conditions and doctors to make clinical diagnoses securely and privately. Hence, this paper proposes a framework named MRCG that integrates Convolutional Neural Network (CNN) and Graph Neural Network (GNN) by incorporating the relationship between multiple gallery images in the graph structure. First, we adopt a Vgg16-based triplet network jointly trained for similarity learning and classification task. Next, a graph is constructed from the extracted features of triplet CNN where each node feature encodes a query-gallery image pair. The edge weight between nodes represents the similarity between two gallery images. Finally, a GNN with skip connections is adopted to learn on the constructed graph and predict the similarity score of each query-gallery image pair. Besides, Focal loss is also adopted while training GNN to tackle the class imbalance of the nodes. Experimental results on some benchmark datasets, including the CE-MRI dataset and a public MRI dataset from the Kaggle platform, show that the proposed MRCG can achieve 88.64% mAP and 86.59% mAP, respectively. Compared with some other state-of-the-art models, the MRCG can also outperform all the baseline models.