Non-parametric scene parsing: Label transfer methods and datasets

Abstract

Scene parsing is the problem of densely labeling every pixel in an image with a meaningful class label. Driven by powerful methods, remarkable progress has been achieved in scene parsing over a short period of time. With growing data, non-parametric scene parsing or label transfer approach has emerged as an exciting and rapidly growing research area within Computer Vision. This paper constitutes a first survey examining label transfer methods through the lens of non-parametric, data-driven philosophy. We provide insights on non-parametric system design and its working stages, i.e. algorithmic components such as scene retrieval, scene correspondence, contextual smoothing, etc. We propose a synthetic categorization of all the major existing methods, discuss the necessary background, the design choices, followed by an overview of the shortcomings and challenges for a better understanding of label transfer. In addition, we introduce the existing standard benchmark datasets, the evaluation metrics, and the comparisons of model-based and data-driven methods. Finally, we provide our recommendations and discuss the current challenges and promising research directions in the field.