SAM: an interoperable metadata model for multimodal surveillance applications

Abstract

Metadata interoperability is crucial for various kinds of surveillance applications and systems, e.g. metadata mining in multi-sensor environments, metadata exchange in networked camera systems or information fusion in multi-sensor and multi-detector environments. Different metadata formats have been proposed to foster metadata interoperability, but they show significant limitations. ViPER, CVML and MPEG Visual Surveillance MAF support only the visual modality, CVML's frame based approach leads to inefficient representation, and MPEG-7's comprehensiveness handicaps its efficient usage for a specific application. To overcome these limitations we propose the Surveillance Application Metadata (SAM) model, capable of describing online and offline analysis results as a set of time lines containing events. A set of sensors, detectors, recorded media items and object instances is described centrally and linked from the event descriptions. The time lines can be related to a subset of sensors and detectors for any modality and different levels of abstraction. Hierarchical classification schemes are used for many purposes, such as types of properties and their values, event types, object classes, coordinate systems etc. in order to allow for application specific adaptations without modifying the data model while ensuring the controlled use of terms. The model supports efficient representation of dense spatio-temporal information such as object trajectories. SAM is not bound to a specific serialization but can be mapped to different existing formats within the limitations evoked by the target format. SAM specifications and examples have been made available