Evaluation of algorithm selection approach for semantic segmentation based on high-level information feedback

Abstract

In this paper we discuss certain theoretical properties of the algorithm selection approach to the problem of semantic segmentation in computer vision. We show that an algorithm's score depends on final task. Thus to properly evaluate an algorithm and to determine its suitability, precise score value obtained on well formulated tasks can be used only. When an algorithm suitability is well known, the algorithm can be efficiently used for a task by applying it in the most favorable environmental conditions determined during the evaluation. However, high quality algorithm selection is possible only if each algorithm suitability is well known because only then the algorithm selection result can improve the best possible result given by a single algorithm. The task dependent evaluation is demonstrated on segmentation and object recognition. Additionally, we also discuss the importance of high level symbolic knowledge in the selection process. The importance of this symbolic hypothesis is demonstrated on a set of learning experiments with both a Bayesian Network and SVM. We show that task dependent evaluation is required to allow efficient algorithm selection. Also by studying symbolic preference of algorithms for semantic segmentation we show that algorithm selection accuracy can be improved by 10 to 15%.