Integrated Detection, Tracking and Recognition for IR Video-based Vehicle Classification

Abstract

We present an approach for vehicle classification in IR video sequences by integrating detection, tracking and recognition. The method has two steps. First, the moving target is automatically detected using a detection algorithm. Next, we perform simultaneous tracking and recognition using an appearance-model based particle filter. We present a probabilistic algorithm for tracking and recognition that incorporates robust template matching and incremental subspace update. There are two template matching methods used in the tracker: one is robust to small perturbation and the other to background clutter. Each method yields a probability of matching. The templates are represented using mixed probabilities and updated when the appearance models cannot adequately represent the variations in object appearance. We also model the tracking history using a nonlinear subspace described by probabilistic kernel principal components analysis, which provides a third probability. The most-recent tracking result is incrementally integrated into the nonlinear subspace by augmenting the kernel Gram matrix with one row and one column. The product of the three probabilities is defined as the observation likelihood used in a particle filter to derive the tracking and recognition result. The tracking result is evaluated at each frame. Low confidence in tracking performance initiates a new cycle of detection, tracking and classification. We demonstrate the robustness of the proposed method using outdoor IR video sequences.