OUTSIDE: Multi-Scale Semantic Segmentation of Universal Outdoor Scenes

Abstract

Semantic segmentation aims at providing a fine-grained image prediction by assigning each pixel to a specific semantic category. Convolutional neural networks offer significant benefits for solving this problem. However, the success of such networks is closely related to the availability of corresponding data sets. To facilitate semantic segmentation in a broader range of scenarios, such as augmented reality in outdoor environments or universal image-to-image translation, adequate training data sets are necessary. We present OUTSIDE15k, a large-scale data set for semantic segmentation of universal outdoor scenes. The data is labeled with 24 different semantic classes. The images contain multiple outdoor scenarios and cover a variety of different resolutions. Additionally, we present OUTSIDE-Net, an improved neural network architecture integrating multi-level pooling, feature fusion, and a spatial mask for semantic segmentation of universal outdoor scenes. It extracts spatial and semantic features from the input images to perform the segmentation. With the presented data set, we show the capability of our network which outperforms state-of-the-art approaches by achieving up to 91.5% pixel accuracy.