Self-taught Learning: Image Classification Using Stacked Autoencoders

Abstract

Availability of a large amount of unlabeled data and practical challenges associated with annotating datasets for different domains has led to the development of models that can obtain knowledge from one domain (called source domain) and use this knowledge in a similar domain (called the target domain). This forms the core of transfer learning. Self-taught learning is one of the popular paradigms of transfer learning frequently used for classification tasks. In a typical self-taught learning setting, we have a source domain having a large amount of unlabeled data instances and a target domain having limited labeled data instances. With this setting, self-taught learning proceeds as follows: Given ample unlabeled data instances in the source domain, we try to obtain their optimal representation. Basically, we are learning the transformation that maps unlabeled data instances to their optimal representation.The transformation learnt in the source domain is then used to transform target domain instances; the transformed target domain instances along with their corresponding labels are then used in supervised classification tasks. In our work, we have applied self-taught learning in image classification task. For this, we have used stacked autoencoders (for grayscale images) and convolutional autoencoders (for colored images) to obtain an optimal representation of the images present in the source domain. The transformation function learnt in the source domain is then used to transform target domain images. The transformed target domain instances along with their labels are then used for building the supervised classifier (in our case, SVM). Rigorous experiments on MNIST, CIFAR10 and CIFAR100 dataset show that our self-taught learning approach is doing well against the baseline model (where no transfer learning has been used) even when there are limited number of labeled data instances in the target domain.