Visual attention-based comparative study on disaster detection from social media images

Abstract

The availability of images of events almost in real-time on social media has a prospect in many application developments. A humanitarian technology for disaster type and level assessment can be developed using the images and video available on social media. In this paper, we investigate the potential use of various available deep learning techniques to develop such an application. For our research, based on the use of publicly available image data, we have started collecting disaster images from various sources from South Asia. We created the South Asia Disaster (SAD) image dataset containing 493 images from various online news portals. Using the Keras as our framework to run our models: Visual Geometry Group (VGG-16 and VGG-19), Inception-V3 and Inception-ResNet-V2 (ResNet: Residual Network). However, to boost up the training speed, we dropped the fully connected layer and added a small, fully connected model. To identify the five different disasters: fire disaster, flood disaster, human disaster, infrastructure disaster, natural disaster; our proposed method with VGG-16 model’s recognition accuracy was 84.51%, which is the highest accuracy on the SAD dataset. After performing the testing, we calculate the VGG-16 classifier’s attention to visualize which part of the disaster images VGG-16 pays attention.