Semantic Scene Labeling via Deep Nested Level Set

Abstract

Semantic scene labeling plays a very important role in intelligent transportation tasks, such as autonomous driving and advanced driver assistance. Recently, thanks to the advances of deep learning, significant improvements have been achieved for this pixel-wise labeling task. Although effective, current methods lack of explicitly modeling the boundary of objects, resulting in inaccurate labeling results. Meanwhile, traditional level set based methods perform better to capture the evolution of boundaries. However, they are sensitive to the model initialization. To address these issues, in this work we propose a novel deep learning framework, named deep nested level set (DNLS) for boundary-aware semantic scene labeling. Different from previous works, our proposed framework explicitly takes deep learned features and object boundary information into account. More specifically, our proposed framework first predicts semantic probability maps and boundary locations of objects using a bifurcated fully convolutional network (BFCN). Then, these probability maps are seamlessly integrated into a nested level set function for accurate scene labeling. As a result, our approach can automatically initialize the nested level set function, and the whole framework can be trained in an end-to-end manner, providing a new solution for accurate semantic scene parsing. Extensive experiments on public CamVid and Cityscapes datasets demonstrate that our proposed framework produces high-quality predictions with clear object boundaries and spatial consistency.