Coordinated Tracking Of Multiple Point Targets With Multiple Cameras

Abstract

Coordinated tracking of multiple point targets with multiple camerasJeffrey LabuzCenter for Machine IntelligenceUniversity of South Carolina, Department of Electrical and Computer EngineeringColumbia, South Carolina 29208ABSTRACTPresented in this paper are the system and measurement models, and the recursive filter equations, for an extended Kalmanfilter (EKF) to be used in tracking video point targets. The filter is designed to maintain estimates of a target's position,velocity, and acceleration in three dimensions (3D) based on two -dimensional (2D) measurements of the target's bearings asobserved by several cameras. The geometric mapping from object points in 3D world coordinates to image points in 2Dimage coordinates is modeled by the central projection for pinhole cameras. The recursive equations of the EKF incorporatethe Jacobian of this nonlinear camera transformation.1. INTRODUCTIONThe general problem addressed here is the tracking of numerous point targets as they move along three -dimensional (3D)trajectories in world coordinates. Several calibrated, pinhole -modeled cameras are assumed to be viewing the targets fromdifferent perspectives. A point target is defined as a 3D object, or object portion, whose camera image is large enough to bedetected above background noise yet not so large that distinguishing features of the object (portion) can be resolved. Twoexamples of typical point targets that might require tracking are distant maneuvering aircraft and the corners of a machinedpart on a conveyor belt. The tracking filter described here assumes that the point targets are moving independently. However,if the tracked points are found to remain fixed distances apart in 3D space, indicating that they are features of a rigid object ora 3D scene, then it should be possible to estimate the structure of the object or scene and the relative motion between thecameras and the object or scene.It is common practice in tracking applications to utilize a Kalman filter to maintain estimates of the current state of eachtarget (its position, velocity, and acceleration in world coordinates), and to predict future states, regardless of the degree ofmeasurement corruption. The design of any Kalman filter begins with a state -space model, possibly nonlinear and time -varying, of both the target dynamics (system model) and the observations (measurement model), complete with modelingerror terms and their covariances. The measurements are recursively filtered to obtain the desired state estimates. The Kalmanfilter is the optimum observer for dynamic systems with linear system and measurement models, and modeling error termswhich are Gaussian, white noise processes.This paper presents system and measurement models, and the corresponding Kalman filter equations, for applicationsinvolving 3D video point target tracking. The targets are assumed to be maneuvering with constant acceleration, to theextent that the acceleration derivative is modeled as a zero -mean Gaussian random variable. This results in a simple systemmodel that is both linear and time -invariant. In contrast, the observations are the two -dimensional (2D) positions of thecentral projections of the targets onto the image planes of the pinhole -modeled cameras. This results in a nonlinearmeasurement model, which requires the use of an extended Kalman filter (EKF). The filter equations incorporate the Jacobianof the camera's nonlinear projective mapping. This Jacobian arises from Taylor series expansion of the nonlinear mappingfunction and truncation of all second -order and higher -order terms. If the cameras are allowed to move with respect to theworld coordinate frame, or if they are allowed to zoom (i.e. alter their focal length), then the measurement model will be time -varying as well as nonlinear. Note that the research reported here can also be applied to any of several higher -order variationsof the standard EKF, including the iterated EKF, the second -order EKF, and the adaptive EKF.The tracking filter described here relies on the detection of valid target images and the matching of those target imagesbetween cameras. A complete tracking system based on this tracking filter would be required to recognize target appearances,or births, and disappearances, or deaths, in the camera images in order to initialize and terminate individual target tracks. Thispaper does not directly address the problems of detection, matching, and birth and death recognition. However, it should benoted that the use of a tracking filter would simplify these problems somewhat. This is because predicted target positions,