A VQA System for Medical Image Classification Using Transfer Learning

Abstract

The Visual Question Answering (VQA) is based on Computer Vision and Natural Language Processing (NLP). The goal of VQA system is to predict the textual answer to a question based on the image. The VQA system takes images and questions as input and combines the information of the input to generate readable answers as output. The medical VQA system has the following advantages. The radiologists can use the VQA system for their inference about the medical image. It can also help the patients to get basic information about the clinical image prior to doctor consultation. In this paper, we discuss about the VQA on medical images using the ImageCLEF 2019 VQA dataset. We build a medical VQA system using transfer learning on radiology images using MobileNet for input images on the plane class and predict the answer. The proposed VQA model is evaluated on the test dataset and the accuracy obtained is 80.8% on the plane class.