Multiobject behavior recognition by selective attention

Abstract

AbstractThis study proposes a method for recognition of the behavior and number of multiple objects without separation of the objects from images. Most conventional techniques of behavior recognition have used bottom‐up processing, in which features were first extracted from images, and then the extracted features were subjected to time‐series analysis. However, separation of objects from images at the feature extraction stage resulted in unstable processing. This study aims at stable recognition of multiobject behavior. For this purpose, a mechanism of selective attention is proposed. With this mechanism, particular image regions (focusing regions) are allotted to all states of the NFA (nondeterministic finite automaton) that performs sequence analysis, and feature extraction (event detection) is performed inside such regions. This approach makes it possible to detect events irrespective of noise (that is, changes that may occur in the image beyond the focusing regions), while nondeterministic state transition means that all possible event sequences are analyzed; hence, the behavior of multiple objects can be recognized without separation of the objects from the images. Object‐specific color tokens are assigned to NFA active state sets, and then are transferred along with the state transitions, which is referred to as the object discrimination mechanism. Introduction of this mechanism allows simultaneous multiobject behavior recognition and detection of the number of objects. In addition, the proposed system has been extended to treat multiview images, and its effectiveness has been proven experimentally. © 2001 Scripta Technica, Electron Comm Jpn Pt 3, 84(9): 56–66, 2001