MSFFT: Multi-Scale Feature Fusion Transformer for cross platform vehicle re-identification

Abstract

A vital component of Intelligent Transportation Systems (ITS) is vehicle re-identification, which allows vehicles to be identified across surveillance devices. Re-identification of vehicles is usually done using information collected from standalone surveillance devices such as fixed surveillance cameras (CCTVs) or aerial devices (UAVs). Re-identifying vehicles across standalone surveillance systems is challenging when there is a severe illumination change, a change of viewpoint, or an occlusion. Cross platform surveillance (CCTV+UAV) based vehicle re-identification is yet to be explored and can mitigate some of the challenges faced during re-identifying vehicles with standalone surveillance systems. This paper proposes a novel cross platform vehicle identification dataset called MCU-VReID using 42 CCTVs and a UAV. A novel re-identification method called Multi-Scale Feature Fusion Transformer (MSFFT) is proposed to re-identify vehicles observed across the cross platform surveillance systems. The network consists of inception layers with transformer networks that enable it to learn the vehicle’s features at a variety of scales. The vehicles observed by two contrasting surveillance systems appear to be transformed representations of one another. Hence a two-stage training approach is facilitated for re-identifying vehicles observed across cross platform surveillance systems. The two-stage training approach aims to learn vehicle semantic transformations in the first stage using self-supervised approaches. The knowledge gained at the first stage relating to vehicle semantic transformations is transferred at the second stage of training to perform re-identification. Extensive experiments using the method demonstrate that MSFFT significantly improves over state-of-the-art methods to perform cross platform vehicle re-identification.