Multiple instance learning based deep CNN for image memorability prediction

Abstract

Image memorability is a recent topic in the domain of computer vision, which enables one to measure the degree at which images are memorable to human cognitive system. Initial research on image memorability shown that memorability is an inherent characteristic of an image, and humans are consistent in remembering images. Further, it is also demonstrated that memorability of an image can be determined using machine learning and computer vision techniques. In this paper, a novel deep learning based image memorability prediction model is proposed. The proposed model automatically learns and utilises multiple visual factors such as object semantics, visual emotions, and saliency to predict image memorability scores. In particular, the proposed model employs multiple instance learning framework to utilise emotion cues evoking from single global context and multiple local contexts of an image. An extensive set of experiments are being carried out on large-scale image memorability dataset LaMem. The experimental results show that the proposed model performs better than current state-of-the-art models by reaching a rank correlation of 0.67, which is close to human consistency (ρ = 0.68).