Human flow recognition using deep networks and vision methods

Abstract

The paper focuses on developing the system for people flow recognition based on many video camera images. The project arose based on the real need — its application in the city malls to find the most visited places and support customers’ campaigns. The potential application in other domains is possible (public buildings, airports). In contrast to the existing solutions, our approach involves on-edge image analysis to decrease the risk of data loss and the system cost. In the project, we design the whole processing pipeline composed of the component modules responsible for detecting, tracking, and reidentifying (shuffling) people. Our experimental platform enabled us to compare multiple method variants for each module. Based on extensive experimental research, the final solution uses the pretrained SSD method in the detection module. The centroid algorithm applied to displacement vectors combined with the Siamese network is the basis for the object tracking module. The best model to solve the reidentification task is the Resnet50. For the Market 1501 dataset, it achieved Rank-1 efficiency of 84.6%. The system gives a visualization of the main paths of people’s movements in the form of a heat map and assigns the direction where people most often look. In the experimental study, we assessed the system’s effectiveness and time efficiency, and the current results give a perspective for its commercialization in the nearest future.