ReMOT: A model-agnostic refinement for multiple object tracking

Abstract

Although refinement is commonly used in visual tasks to improve pre-obtained results, it has not been studied for Multiple Object Tracking (MOT) tasks. This could be attributed to two reasons: i) it has not been explored what kinds of errors should — and could — be reduced in MOT refinement; ii) the refinement target, namely, the tracklets, are intertwined and interactive in a 3D spatio-temporal space, and therefore changing one tracklet may affect the others. To tackle these issues, i) we define two types of errors in imperfect tracklets, as Mix-up Error and Cut-off Error, to clarify the refinement goal; ii) we propose a Refining MOT Framework (ReMOT), which first splits imperfect tracklets and then merges the split tracklets with appearance features improved by self-supervised learning. Experiments demonstrate that ReMOT can make significant improvements to state-of-the-art MOT results as powerful post-processing. As a new application, we demonstrate that ReMOT has the potential of being used to assist semi-automatic MOT data annotation and partially release humans from the tedious work.