An approach to automatic real-time novelty detection, object identification, and tracking in video streams based on recursive density estimation and evolving Takagi-Sugeno fuzzy systems

Abstract

Recently, surveillance, security, patrol, search, and rescue applications increasingly require algorithms and methods that can work automatically in real time. This paper reports a new real-time approach based on three novel techniques for automatic detection, object identification, and tracking in video streams, respectively. The novelty detection and object identification are based on the newly proposed recursive density estimation (RDE) method. RDE is using a Cauchy-type of kernel, which is calculated recursively as opposed to the widely used (in particular in the kernel density estimation (KDE) approach) Gaussian one. The key difference is that the proposed approach works on a per frame basis and does not require a window (usually of size of several dozen) of frames to be stored in the memory and processed. It should be noted that the new RDE approach is free from user- or problem-specific thresholds by differ from the other state-of-the-art approaches. Finally, an evolving Takagi–Sugeno (eTS)-type fuzzy system is proposed for tracking. The proposed approach has been compared with KDE and Kalman filter (KF) and has proven to be significantly (in an order of magnitude) faster and computationally more efficient than RDE and more precise than KF. © 2010 Wiley Periodicals, Inc. © 2011 Wiley Periodicals, Inc.