Research of multi-object detection and tracking using machine learning based on knowledge for video surveillance system

Abstract

Recently, as the risk of crime and accidents increases, interest in security and surveillance of individuals and the public is increasing rapidly, and video surveillance system technology is continuously developing. Reliable object detection in the system is the basis of all elements using image information and it is used in various applications using the information, so accurate object detection and tracking are needed. Therefore, we propose a system for analyzing images with a knowledge-based deep learning technology for multi-object recognition and tracking enhancement. Algorithms for recognizing objects using existing convolution neural network (CNN) classifiers have a problem that it is difficult to process in real time because the processing time is increased when there are a lot of objects to be classified in the image. Therefore, we propose an algorithm that combines optical flow while maintaining the recognition performance through a knowledge-based CNN. An optical flow-based tracker can forecast the position of objects in the next frame based on the position of objects in the current frame. A CNN-based detector can detect the position of objects through a knowledge-based mining method between the two images. CNN-based detectors also carry out mining method on current frame information. This detector can select more capacity features based on the background to more accurately forecast the location of the tracked targets and targets. The fusion of the tracker and detector compensates for accumulated errors that can occur in the tracker and for drift from the detector. The experimental results show that the proposed algorithm combining CNN and optical flow can detect and trace multiple objects in a video stream, and can carry out robust detection and tracing even in a complex environment.