Image enhancement towards object detection via foreground salience guidance

Abstract

Although the schemes based on deep learning applied to car autonomous driving show good performance in object detection un- der condition of normal weather, there are many difficulties in complex weather scenarios. When facing heavy fog interference, the general solution is to design two networks for image enhancement and object detection respectively, which means one dehazes the foggy image and the other detects the dehazed image. But this may bring two problems: 1. After the image is dehazed, it may cause some distortion of the image, especially the areas that are unlikely to be reconstructed due to severe haze; 2. For object detection, the interesting part of an image is the nearby area where the object is located. The rest background area that is far away from the object may not help. In response to the above two problems, in this work we proposed to use foggy images except whose area of object bounding box is clear as ground-truth in the training of image enhancement. In other words, only the key area around the object is clear in order to guide the network to perform image dehazing operations focusing on the target areas and finally improves the following object detection performance. By using U-net for image dehazing on the Foggy Cityscapes dataset and Faster R- CNN network with the same structure for object detection, the average accuracy mAP is increased by % compared to normal training, which demonstrates the effectiveness of our approach on foggy image object detection.