Bunch-of-Keys Module for Optimizing a Single Image Detector Based on the Property of Sequential Images

Abstract

In image processing, deep learning networks have been continuously developed and are used in many fields. However, most networks do not reflect image continuity. In this paper, we propose a novel bunch-of-keys module connected to the backend of a deep learning network to improve the detector performance on sequential images. This module optimizes existing deep learning networks to detect sequential images without retraining. This procedure reduces time and computing costs, and the average precision improves with a minimal drop in the frames per second. By adopting a sliding window method that uses three consecutive images, the keys are generated by comparing the positions of the detected boxes for each of the images using generalized intersection over union. The two key types perform correcting operations. The rectifying key has the effect of adding or merging undetected bounding boxes in mid-frame. The tracking key has the effect of compensating for bounding boxes lost for no reason in the third frame. The candidate box extracted using each key determines whether to add or merge to the target image in the correction task. This task calculates the complete intersection over union (CIoU) score between candidate boxes and all boxes of the target image and is divided into add or merge cases according to a set of CIoU criterion. As a result of adding or merging the bounding box to the missing object, detection performance was improved up to 3% in terms of the average precision.