Sort and Deep-SORT Based Multi-Object Tracking for Mobile Robotics: Evaluation with New Data Association Metrics

Ricardo Pereira,Urbano J Nunes,Guilherme Carvalho,Luís Garrote

doi:10.3390/app12031319

Abstract

Multi-Object Tracking (MOT) techniques have been under continuous research and increasingly applied in a diverse range of tasks. One area in particular concerns its application in navigation tasks of assistive mobile robots, with the aim to increase the mobility and autonomy of people suffering from mobility decay, or severe motor impairments, due to muscular, neurological, or osteoarticular decay. Therefore, in this work, having in view navigation tasks for assistive mobile robots, an evaluation study of two MOTs by detection algorithms, SORT and Deep-SORT, is presented. To improve the data association of both methods, which are solved as a linear assignment problem with a generated cost matrix, a set of new object tracking data association cost matrices based on intersection over union, Euclidean distances, and bounding box metrics is proposed. For the evaluation of the MOT by detection in a real-time pipeline, the YOLOv3 is used to detect and classify the objects available on images. In addition, to perform the proposed evaluation aiming at assistive platforms, the ISR Tracking dataset, which represents the object conditions under which real robotic platforms may navigate, is presented. Experimental evaluations were also carried out on the MOT17 dataset. Promising results were achieved by the proposed object tracking data association cost matrices, showing an improvement in the majority of the MOT evaluation metrics compared to the default data association cost matrix. In addition, promising frame rate values were attained by the pipeline composed of the detector and the tracking module.

Highlights

Vision-based Multi-Object Tracking (MOT) methods analyze image sequences to establish object correspondences over the images [1,2]
Throughout the years, MOT tasks were mainly performed by the tracking by detection paradigm [9], where objects were detected by an object detector and fed to the object tracking method, which dealt with the object association between previous frames and the present one
Eight new tracking data association metrics based on intersection over union, Euclidean distances, and bounding boxes ratio were proposed

Summary

Introduction

Vision-based Multi-Object Tracking (MOT) methods analyze image sequences to establish object correspondences over the images [1,2]. Throughout the years, MOT tasks were mainly performed by the tracking by detection paradigm [9], where objects were detected by an object detector and fed to the object tracking method, which dealt with the object association between previous frames and the present one. With the emergence of Deep learning-based Neural Networks (DNNs) [13,14], new state-of-the-art methods have been proposed in object vision-based tasks such as object classification [15], recognition [16], and tracking [11,17,18]. To improve the object association step of tracking algorithms, Convolutional Neural Networks (CNNs) have been applied to extract object appearance features, which are used to compute similarity values between two objects’ feature maps, extracted over two consecutive images. CNNs have been used to locate objects to track consecutive images [19,20]

Objectives

Methods

Results

Conclusion