Cognitive Processes in Eye Guidance: Algorithms for Attention in Image Processing

Abstract

When inspecting an image for the first time, how does the viewer decide where to look next? The saliency map hypothesis proposes that viewers initially analyse the image for variations in low-level visual features including intensity, colour, and edge orientation, and that their eyes are guided towards the most salient region. The saliency of objects in scenes may provide an explanation of why some experiments find that incongruent objects attract attention whilst other studies do not find this effect. Experiments that have monitored eye movements during scene inspection have found some support for the saliency map hypothesis, particularly when pictures are inspected in anticipation of a memory test. Under some circumstances the hypothesis fails to account for inspection patterns. When scenes are inspected to check the presence or absence of a named object, or when two images are compared to determine whether they are identical, or when the viewer has specialised domain knowledge of the scene depicted, then saliency has little influence. This paper evaluates the saliency map hypothesis of scene perception using evidence of eye movements made when images are first inspected, and concludes that visual saliency can be used by viewers, but that its use is both task-dependent and knowledge-dependent.