Particle Filter Tracking of Camouflaged Targets by Adaptive Fusion of Thermal and Visible Spectra Camera Data

Abstract

This paper presents a method for tracking a moving target by fusing bi-modal visual information from a deep infra-red thermal imaging camera and a conventional visible spectrum color camera. The tracking method builds on well-known methods for color-based tracking using particle filtering, but it extends these to handle fusion of color and thermal information when evaluating each particle. The key innovation is a method for continuously relearning local background models for each particle in each imaging modality, comparing these against a model of the foreground object being tracked, and thereby adaptively weighting the data fusion process in favor of whichever imaging modality is currently the most discriminating at each successive frame. The method is evaluated by testing on a variety of extremely challenging video sequences, in which people and other targets are tracked past occlusion, clutter, and distracters causing severe and sustained camouflage conditions in one or both imaging modalities.