Efficient Computation and Model Selection in Semi-Supervised Learning

Abstract

Traditional learning algorithm uses only labeled data for training. However, labeled examples are often difficult or time consuming to obtain since they require substantial labeling efforts from humans. On the other hand, unlabeled data are often relatively easy to collect. Semi-supervised learning addresses this problem by using large quantities of unlabeled data with the labeled data to build better learning algorithms. In this paper, we propose a general approach augmenting traditional supervised learning into semi-supervised learning paradigm. A regularization framework which balances a tradeoff between loss and penalty is established. We investigate different implementations of loss function and suggest the methods which have the least computation expenses. The value of a hyperparameter, which determines the balance between loss and penalty, is crucial in model selection. Hence, we derive an algorithm that can fit the entire path of solutions for every value of hyperparameter. Its computational complexity is quadratic in the number of labeled examples only rather than the total number of labeled and unlabeled examples.