Deep Learning and Late Fusion Technique in Medical X-ray Image

Abstract

Classification of images is essential in medical database because of the different image modalities such as X-ray images, Computed Tomography (CT) images, Magnetic Resonance imaging (MRI) etc. In addition to the varieties of image modalities present in different databases, they are of different body parts and need to be properly classified to enhance effective retrieval for purposes such as medical diagnosis, teaching and research. Most of the hand-crafted techniques have various limitations which reduce their potentials to accurately classify medical X-ray images. For more than a decade, various researchers have employed the use of different handcrafted techniques for medical image classification. However, the major problem associated with the techniques is their inability to extract discriminative features that are relevant enough to accurately classify medical images such as radiographic images. This study focuses on employing deep learning and fusion technique to classify medical X-ray images. The proposed technique uses a single pre-trained neural network and a late fusion technique for the classification of ImageCLEF 2007 and 2015 dataset. The employment of a single pre-trained neural network, both as a feature extractor and as a fine-tuned network, makes the technique unique especially when considering the nature of the dataset used. The combination of the posterior probabilities generated from SVM and Softmax classifiers using a single deep pre-trained neural network produces an overall classification accuracy of 95.54% in classifying the dataset into 116 categories on ImageCLEF 2007. This is the highest when compared to the use of AlexNet + SVM and fine-tuned AlexNet alone which produced 84.35% and 86.47% classification accuracies respectively. On ImageCLEF 2015, it produces an overall classification accuracy percentage of 87.72%.