Abstract
Visual Question Answering (VQA) is a multi disciplinary challenging problem involving various fields such as Natural Language Processing, Computer Vision, Deep Learning and Knowledge Representation, which has garnered much interest among researchers, especially with the recent advancements in machine perception. The problem at hand not only involves reasoning over visual elements present in the image or natural language understanding of the input query, but also may involve outside world knowledge in order to infer the answer. In this paper, we convert the VQA problem to a factoid question answering task over a set of natural language facts extracted from images in the Visual Genome Dataset [1]. Recent literatures in textual question answering have established the effectiveness of End to End Memory Networks (MemN2N) over the standard LSTMs. Inspired by the approaches incorporated by researchers in this direction, as a first step to create an explainable VQA model, this paper proposes the incorporation of MemN2N with soft attention for inferring the answer from a set of regional facts extracted from the image. We also experiment with the addition of a Bayesian Neural layer for posterior reasoning of the answer from a fixed vocabulary, as proposed in [2] which shows a significant improvement in the accuracy score compared to the other models tested.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.