Robust background subtraction for automated detection and tracking of targets in wide area motion imagery

Abstract

Performing persistent surveillance of large populations of targets is increasingly important in both the defence and security domains. In response to this, Wide Area Motion Imagery (WAMI) sensors with Wide FoVs are growing in popularity. Such WAMI sensors simultaneously provide high spatial and temporal resolutions, giving extreme pixel counts over large geographical areas. The ensuing data rates are such that either very bandwidth data links are required (e.g. for human interpretation) or close-to-sensor automation is required to down-select salient information. For the latter case, we use an iterative quad-tree optical-flow algorithm to efficiently estimate the parameters of a perspective deformation of the background. We then use a robust estimator to simultaneously detect foreground pixels and infer the parameters of each background pixel in the current image. The resulting detections are referenced to the coordinates of the first frame and passed to a multi-target tracker. The multi-target tracker uses a Kalman filter per target and a Global Nearest Neighbour approach to multi-target data association, thereby including statistical models for missed detections and false alarms. We use spatial data structures to ensure that the tracker can scale to analysing thousands of targets. We demonstrate that real-time processing (on modest hardware) is feasible on an unclassified WAMI infra-red dataset consisting of 4096 by 4096 pixels at 1Hz simulating data taken from a Wide FoV sensor on a UAV. With low latency and despite intermittent obscuration and false alarms, we demonstrate persistent tracking of all but one (low-contrast) vehicular target, with no false tracks.