Jointly detecting and multiple people tracking by semantic and scene information

Abstract

In this paper, we propose a new method for online multiple people tracking, which combines the detection process and the single object tracking process, and establishes the interactions between them. The detector detects objects in the still images which ignores the sequential information. Meantime, the single object tracker does not use the category semantic information during tracking. To take both the sequential and semantic information into account, we exchange information among the detector and the trackers. More specifically, the trackers deliver sequential information to the detector by providing the detector with the extra proposals. The detector supplements each tracker with the robust semantic information by using bounding box regression to modify the tracking result. Besides, the interactions also happen among the trackers through the occlusion speculation, the perspective model interpretation and the trajectory merging process. The experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art MOT methods.