Hybrid active learning for non-stationary streaming data with asynchronous labeling

Abstract

Active learning enables supervised classifiers to learn using fewer labeled samples, by actively selecting samples for human labeling. Most Active Learning approaches can be categorized as pool-based or stream-based. Pool-based strategies select instances to be labeled from the available pool of unlabeled data, by evaluating each instance, whereas stream-based strategies examine every instance in the incoming stream of unlabeled data and decide sequentially whether they want that instance to be labeled or not. Stream-based strategies enable the ability to adapt the classifier model more quickly as the incoming data changes, while pool-based strategies often exhibit better learning rates. In this paper, we propose a framework and method for Hybrid Active Learning that integrates pool-based and stream-based strategies to harvest the benefits of both, in a streaming data classification scenario where concept drift may be prevalent, and labeling is asynchronous. In addition, we propose (i) prioritized aggregation of selection to combine selected instances for labeling from the pool-based and stream-based strategies, and (ii) batch period adaptation to dynamically change the triggering pattern of the pool-based strategy based upon the detection of concept drift.