How to detect occluded crosswalks in overview images? Comparing three methods in a heavily occluded area

Abstract

Crosswalk presence data is crucial for pedestrian safety and urban planning. However, obtaining such data at a large scale is often challenging due to the high cost associated with traditional collection methods. While automated methods based on computer vision have been explored to detect crosswalks from aerial images, a major obstacle to their application is the handling of candidate crosswalks occluded by objects or shadows in the aerial imagery. To address this challenge, this study explores different deep learning-based solutions, including the aerial-view and street-view methods, which are commonly used, and a combination of the two − dual-perspective method. Deep learning models based on Convolutional Neural Networks with VGG16 architecture were trained using 16,815 images to automatically detect crosswalks from both aerial and street view images. To compare the performance of these methods in handling occlusions, 1,378 images from a heavily occluded area were processed separately by the three methods. The results showed that the aerial-view method suffered the most when dealing with images from a heavily occluded area, resulting in the lowest accuracy, precision, recall, and F1 score among the three methods. On the other hand, the street-view method outperformed the aerial-view method significantly. The dual-perspective method demonstrated the highest accuracy and precision values, indicating its superiority in accurately predicting the location of a crosswalk. However, the street-view method exhibited the highest recall value, highlighting its superior ability to recover an occluded crosswalk among all methods.