<title>Visual attention control in image sequences using multiple knowledge sources</title>

Abstract

Both biological visual systems and image understanding systems are forced by resource limitations to reduce input data to their essential part, to keep the amount of data manageable in succeeding processing levels and to maintain a realistic chance to achieve real-time performance even for complex tasks. We present a new design and implementation of a visual attention control system (GOAL) for a significant reduction of data while maintaining salient information. The visual attention system deals not with synthetic images or simple static images but is developed for complex dynamic real image sequences emphasizing arbitrary traffic scenes recorded from a car built-in camera. GOAL is part of the image understanding system MOVIE for real-time interpretation of traffic scenes and supports the model-based scene analysis (MOSAIC) by directing high-level vision processes to salient regions. Based on a model of human attention (the guided search model from Wolfe and Cave) requirements for the module `visual attention' of an image understanding system are derived. GOAL combines different knowledge sources (both motion and shape-oriented) to achieve a robust, spatially restricting, and expectation-driven attention control system. The knowledge sources consist of four very basic image operations, namely (1) enhanced difference image method, (2) direct depth method, (3) local symmetry detection, and (4) 2D - 3D line movement. Each knowledge source contributes to an accumulated evidence for the existence of attention fields. The knowledge sources are temporally stabilized by using a Kalman filter. The nonlinear combination of multiple knowledge sources makes the selection of attention fields much more robust even with merely increasing computational power. This is shown with results from various real image sequences.