Dealing with Limited Access to Data: Comparison of Deep Learning Approaches

Abstract

Nowadays deep learning is a tool, which is used in many different pattern recognition problems. For achieving strong results, which are reported in literature, often large amounts of data are necessary. In many use cases collecting or labeling data is very difficult or not possible at all. The aim of this paper is to compare three different algorithmic approaches in order to deal with limited access to labeled and unlabeled data. The drawbacks and benefits of each method are shown and compared to each other. A successful algorithmic approach, which is especially successful for one- and few-shot learning problems, is the usage of external data during the classification task. In this paper siamese neural networks will be investigated in order to evaluate how the usage of external data improves the accuracy. Another widely used approach is consistency regularization. Using consistency regularization state of the art results in semi-supervised learning (SSL) benchmarks are achieved. Virtual adversarial training (VAT) has shown strong results and is chosen as a representative algorithm for consistency regularization. The last approach is the usage of generative adversarial networks (GANs). In literature GANs are often used in order to create additional data and therefor to increase the generalization capability of the classification network. Furthermore the usage of unlabeled data for further performance improvement is considered. The use of unlabeled data is investigated for GANs and VAT.