Online real-time pedestrian tracking from medium altitude aerial footage with camera motion cancellation

Abstract

Using medium altitude footage in various image based pedestrian detection and tracking tasks have several advantages as opposed to low altitude footage. For instance, much larger covered area, or weaker perspective distortions and occlusions. Furthermore, it does not pose privacy issues. However, it also has a disadvantage that is the irregular camera motion. Such motions can be observed with some ground level footage as well. Some state-of-the-art multi-object tracking (MOT) algorithms, which excel at pedestrian tracking, handle these implicitly, however this is insufficient in the case of medium altitude footage. The problem is further enhanced in online real-time tracking where frames may need to be skipped. Although there are explicit frame registration methods developed for wide area motion imagery (WAMI), they operate on assumptions that are not true in medium altitude footage such as the background is mostly stationary. We show that by augmenting the pedestrian tracking algorithms with a simple visual object tracking (VOT) based frame registration method their accuracy on medium altitude footage can be substantially improved without hindering the execution speed. Moreover, we show that the algorithms’ execution speed can be significantly increased by better utilization of the resources. To evaluate these, we introduce helicopter video sequences with MOT annotation.