Real-world pedestrian detection method enhanced by semantic segmentation

Abstract

Although pedestrian detection has been largely improved with the emergence of convolutional neural networks (CNN), the performance in autonomous driving still faces various challenges, which mainly include large-scale variation, illumination variation, and occlusion of different levels. A robust pedestrian detector enhanced by semantic segmentation is proposed. Inspired by the benefits of multitask learning, our main idea lies in integrating the task of semantic segmentation into the detection framework with auxiliary supervision, inheriting the merits of the two-stream network. Specifically, anchor boxes with various scales are paved on the feature maps of a base CNN; detection is performed based on bounding box classification and regression. On the other stream, semantic segmentation is also performed based on the same feature maps. Extensive experiments on the recently published large-scale pedestrian detection benchmark, i.e., CityPersons, show that the additional supervision from semantic segmentation can significantly improve the detection accuracy without extra computational burdens during inference, which demonstrates the superiority of the proposed method.