Semi-unsupervised Learning: An In-depth Parameter Analysis

Abstract

Creating datasets for supervised learning is a very challenging and expensive task, in which each input example has to be annotated with its expected output (e.g. object class). By combining unsupervised and semi-supervised learning, semi-unsupervised learning proposes a new paradigm for partially labeled datasets with additional unknown classes. In this paper we focus on a better understanding of this new learning paradigm and analyze the impact of the amount of labeled data, the number of augmented classes and the selection of hidden classes on the quality of prediction. Especially the number of augmented classes highly influences classification accuracy, which needs tuning for each dataset, since too few and too many augmented classes are detrimental to classifier performance. We also show that we can improve results on a large variety of datasets when using convolutional networks as feature extractors while applying output driven entropy regularization instead of a simple weight based L2 norm.