A Video Event Detection and Mining Framework

Abstract

We present a video event mining framework that consists of comprehensive set of tools for event detection, annotation, content browsing and a video analysis database. Central to our framework is the video analysis database and the VideoViews database browser that supports both top-down and bottom-up analysis of the video data. to support event mining. We present two methods for video event detection, namely an expert system (CLIPS) rules based approach and a 2-level Hidden Markov Model built upon split and merge behaviors. We devised interfaces for these event detection methods to be operated on the video data in the database for training and detection. We embed scene, object and event data into the video stream as metadata. VideoViews provides interfaces to event detection and video annotation tools. Our video analysis database description scheme represents the structure of the video data from video clips to scenes, objects and their tracks as well as the semantics from simple behaviors to more complex events that may take place over multiple video scenes and/or clips. This framework and the combination of tools enable the users to visualize the raw video data and the processed video information from a number of perspectives facilitating efficient video event mining.