Feature Extraction from Radiology Images for Visual Question Answering System Using CNN and BiLSTM Model

Abstract

AbstractTo extract meaningful information from medical images is one of the challenging processes in the medical domain. As in the medical domain system growing rapidly, the extraction of input data should have efficiency and satisfaction output to the end user. To generate a textual representation is not only the required challenge in semantic segmentation, but it also needs to consider semantic interaction between question and answers simultaneously. The latest visual classification techniques are giving better performance in runtime and efficiency than previous techniques, and this can be used in some of the applications which include security features, medical domain, traffic identification and other fields. Convolutional neural network (CNN) model for visual feature extracting and Bidirectional Long Short-Term Memory (BiLSTM) model is used for feature extraction for textual data is the better choice for visual question answering from radiology image. This proposed system can solve different approaches of image classification using CNN and textual classification using BiLSTM. This system helps to extract the features of radiology image also it satisfies the users with appropriate answers for their questions and the answers should be in both objective and descriptive manner. A visualization system that projects the answers as a baseline that displays the corresponding area with different colors, which makes it easier to note the answers in a visual method for the relevant questions, has a higher accuracy. The advantage of this system is that the answers will be traceable for more clarity and further treatments. The mechanism of BiLSTM is to assign the appropriate weights according to the semantic similarity of the question answer pair for the radiological image. BiLSTM focuses on significant information and raises the probability of selecting the correct answer. The random forest classification method will be used for further to get appropriate answers with related images.KeywordsCNNBiLSTMClassification modelsRadiology imagesVisual question answer