Data association algorithm for large-scale multi-object tracking with complex interactions

Abstract

We present an online multi-object tracking algorithm to track multiple objects across a large number of image frames. Our work is motivated by the need to study evolution of nanoscale objects by transmission electron microscopy. The proposed approach is based on the existing multi-way data association tracking algorithm that is capable of tracking interacting objects with complex behaviors (i.e., merge, split, overlap, and appearance or disappearance). The multi-way data association is an offline algorithm to associate objects across all image frames at one step with a global optimization, which does not scale very well for large number of image frames. The proposed online tracking algorithm processes image frames as they arrive by detecting all objects in the newly arrived image frame and making the associations of the objects to those detected from the previous frame by the multi-way data association. This frame-by-frame association scheme can cause fragmented traces of the objects that are occasionally misdetected for some image frames. We overcome this issue by allowing previously unassociated objects to be associated when the objects reappear within a fixed number of future image frames, namely the frame-delayed association. We combine the multi-way data association with the frame-delayed association to be able to track interacting objects with accurate handling of object disappearance events. The proposed method is validated through applications to simulated multi-object tracking problem and a real multi-object tracking problem. The outcome of the proposed method is compared with four state-of-the-art algorithms.