Active Learning for Image Classification: A Deep Reinforcement Learning Approach

Abstract

Active learning aims to select “worthy” data for annotation such that the model could achieve better performance using as less labeled data as possible. Previous research works mainly use heuristic selection methods to solve this problem. Since the process of data selection and model training is separated, these methods have limitations in effectiveness. This paper proposes a new active learning framework, which uses deep reinforcement learning as the data selection strategy. Instead of choosing “worthy” data through heuristic algorithms, we use the deep reinforcement learning algorithm explicitly learning a data selection policy. The deep convolutional neural network is used to extract images' features which serve as “state” in the reinforcement learning algorithm. And we use deep Q-learning algorithm to train a Q-network. According to the output of the Q-network to decide taking which “action”, i.e. annotate the data or not. The framework proposed in this paper can be trained by an end-to-end manner. Comprehensive experimental evaluations on CIFAR-10, CIFAR-100 and SVHN datasets with VGG-16 model and four different depth ResNet models demonstrate that the proposed method outperforms those state-of-art active learning methods for the task of image classification.