Visibility of points: Mining occlusion cues for monocular 3D object detection

Abstract

Monocular 3D object detection aims at achieving prediction from two-dimensional image plane to three-dimensional physical world. It is an inevitable problem that occlusion phenomena limit the performance in practice. To solve the challenging problem that directly represents the spatial information of occlusion relation, we propose the visibility states of points to describe the spatial distance relationships of occlusion pairs and the implied orientation information. The visibility state introduction can better represent the level and direction of occlusion information and enhance the network’s understanding of occlusion information. Furthermore, we redesign an end-to-end detector to encode features of visibility states to integrate occlusion ordering cues of the whole image to assist object localization in world space. Experiments on the KITTI3D dataset indicate that our method succeeds in establishing visibility states as occlusion cues and promoting the performance of the original detector. Our method is effective, and the performance is comparable with state-of-the-art approaches, especially outstanding in Moderate and Hard cases. Specifically, our method improves the accuracy of 3D moderate case detection to 42.75% and hard case to 37.03% in the KITTI3D dataset.