Intelligent Visual Surveillance: Towards Cognitive Vision Systems

Abstract

Automated visual surveillance systems are required to emulate the cognitive abilities of surveillance personnel, who are able to detect, recognise and assess the severity of suspicious, unusual and threatening behaviours. We describe the architecture of our surveillance system, emphasising some of its high-level cognitive capabilities. In particular, we present a methodology for automatically learning semantic labels of scene features and automatic detection of atypical events. We also describe a framework that supports learning of a wider range of semantics, using a motion attention mechanism and exploiting long-term consistencies in video data. Visual surveillance systems are widely used in public places. Traditional surveillance systems consist of cameras, storage devices, video monitors and security personnel. Se- curity staff monitor the activity in the scene, watching for suspicious or threatening activities. In addition to online monitoring, post-examination of recorded video data may be required to identify suspicious persons, vehicles or events. Both tasks are tedious, as security staff need to identify spe- cific and unusual events from a large number of very com- mon and repetitive events. Unfortunately, human operators usually struggle to deal with the required huge cognitive overload, even for a small surveillance system of few cam- eras. Current commercial surveillance systems make use of digital technology to capture, store and process video data. For example, Video Motion Detectors (VMDs) are able to automatically detect scene motion and send a notification signal to an operator. However, their operation is still primi- tive and not sufficiently discriminatory (e.g. in busy envi- ronments, motion is continuously detected). In general, visual surveillance systems are required to minimise the role of human operators. More specifically, some of the requirements are automatic detection of suspi- cious events that eases online monitoring, and context-based databases of the observed events that facilitate conceptual querying and searching. Cognitive vision systems that evolve and adapt to their environments could potentially fulfil such requirements. These systems are expected to outperform the human operators, as they will be able to operate reliably and continuously, without the fatigue constraint.