Coarse-to-fine road scene segmentation via hierarchical graphical models

Abstract

The road scene segmentation is an important problem which is helpful for a higher level of the scene understanding. This article presents a novel approach for image semantic segmentation of road scenes via a hierarchical graph-based inference. A deep encoder–decoder network is first applied for a fast pixel-wise classification. Then, hierarchical graph-based inference is performed to get an accurate segmentation result. In the inference process, all the object classes are grouped into fewer categories which contains at least one class. The category labels are assigned to image superpixels using Markov random field model. For each category, a pixel-level labeling based on fully connected conditional random fields is performed to divide image into different classes. After the inference for all categories, the results are integrated together to get the final segmentation. In additional to low-level affinity functions, the feature maps from network are integrated in pairwise potentials of the graphical models. This hierarchical inference scheme can alleviate the confusion of classes belonging to different categories. It performs well for small objects without adding more computational burden. Both qualitative and quantitative assessments are adopted to evaluate the proposed method. The results on benchmark data sets prove the effectiveness of the proposed hierarchical scheme, and the performance is competitive with the state-of-the-art methods.