Modeling the Visual and Linguistic Importance of Objects

Abstract

Previous work measuring the visual importance of objects has shown that only spatial information, such as object position and size, is predictive of importance, whilst low-level visual information, such as saliency, is not (Spain and Perona 2010, IJCV91, 59–76). Objects are not important solely on the basis of their appearance. Rather, they are important because of their contextual information (eg, a pen in an office versus in a bathroom), which is needed in tasks requiring cognitive control (eg, visual search; Henderson 2007, PsySci16 219–222). Given that most visual objects have a linguistic counterpart, their importance depends also on linguistic information, especially in tasks where language is actively involved—eg, naming. In an eye-tracking naming study, where participants are asked to name 5 objects in a scene, we investigated how visual saliency, contextual features, and linguistic information of the mentioned objects predicted their importance. We measured object importance based on the urn mode...