Combining keypoint-based and segment-based features for counting people in crowded scenes

Abstract

The counting of the number of people within a scene is a practical machine vision task, and it has been considered as an important application for security purposes. Most of the people counting algorithms generally extract the foreground segments and map the number of people to some features such as foreground area, texture, or edge count. Keypoint-based approaches, on the other hand, have also been proposed, which involves the use of statistical features of keypoints, such as the number of moving keypoints to estimate the crowd size. In contrast to the foreground segment-based methods, keypoint-based approaches are not sensitive to background changes, illuminations, occlusions, and shadows. However, they have limited performance due to the lack of sufficient features. In this paper, in order to estimate the crowd count, the combination of keypoint-based and segment-based (foreground) features is proposed. However, the whole approach is based on the keypoints and not all the image pixels. The proposed method, firstly, extracts the salient keypoints in the scene. Then, foreground segments are obtained by a simple morphological operation on the moving keypoints and hence the system does not suffer from difficulties associated with foreground/background segmentation. Various features are extracted from each foreground segment together with the corresponding keypoints which are highly correlated with the size, density, and occlusion level of the crowd. Finally, a combination of the segment-based and keypoint-based features is used to estimate the number of people in crowds. The experiment demonstrates that the proposed method achieves lower counting error rates compared to the existing approaches.