Content-based event retrieval using semantic scene interpretation for automated traffic surveillance

Abstract

This paper proposes an object segmentation and tracking algorithm for visual surveillance applications. In order to detect moving objects from a dynamic background scene which may have temporal clutters such as swaying plants, we devised an adaptive background update method and a motion classification rule. A two-dimensional token-based tracking system using a Kalman filter is designed to track individual objects under occlusion conditions. We propose a new occlusion reasoning approach where we consider two different types of occlusion: explicit occlusion and implicit occlusion. By tracking individual objects with segmented data, we can generate motion trajectories and set a motion model using polynomial curve fitting. The trajectory model is used as an indexing key for accessing the individual object in the semantic level. We also propose an efficient way of indexing and searching based on object-specific features at different semantic levels. The proposed searching scheme supports various queries including query by example, query by sketch, and query on weighting parameters for event-based retrieval. When retrieving an interested video clip, the system returns the best matching event in the similarity order. In addition, we implement a temporal event graph for direct accessing and browsing of a specific event in the video sequence.