Appearance Based Multiple Agent Tracking Under Complex Occlusions

Abstract

Agents entering the field of view can undergo two different forms of occlusions, either caused by crowding or due to obstructions by background objects at finite distances from the camera. This work aims at identifying the nature of occlusions encountered in multi-agent tracking by using a set of qualitative primitives derived on the basis of the Persistence Hypothesis - objects continue to exist even when hidden from view. We construct predicates describing a comprehensive set of possible occlusion primitives including entry/exit, partial or complete occlusions by background objects, crowding and algorithm failures resulting from track loss. Instantiation of these primitives followed by selective agent feature updates enables us to develop an effective scheme for tracking multiple agents in relatively unconstrained environments. The agents are primarily detected as foreground blobs and are characterized by their centroid trajectory and a non-parametric appearance model learned over the associated pixel co-ordinate and color space. The agents are tracked through a three stage process of motion based prediction, agent-blob association with occlusion primitive identification and appearance model aided agent localization for the occluded ones. The occluded agents are localized within associated foreground regions by a process of iterative foreground pixel assignment to agents followed by their centroid update. Satisfactory tracking performance is observed by employing the proposed algorithm on a traffic video sequence containing complex multi-agent interactions.