Cost-Effective Active Semi-Supervised Learning on Multivariate Time Series Data With Crowds

Abstract

Traditional active learning algorithms have several limitations: 1) they cannot obtain satisfactory results on high dimensional datasets, especially for multivariate time series (MTS) data; 2) traditional crowd-based labeling approaches do not consider the swarm intelligence of crowds, which cannot guarantee the confidence of labeling results; and 3) up to now, few works have addressed the issue of tradeoff between the labeling accuracy and the labeling cost with crowdsourcing labelers. There is also a lack of research on crowd-based active learning on MTS data. To efficiently address the above issues, we propose a new framework of active semi-supervised learning. First, two criteria are advanced to measure the importance of an unlabeled sample from different perspectives. A dynamic time wrapping (DTW)-based similarity matrix is used to represent the MTS data. Next, to confidently label the most valuable samples with minimum cost, we advance a cost-effective crowd selection model and an adaptive labeler selection approach (ALS) to select the most suitable labelers, which could minimize the total cost of labeling and achieve better classification performance. Experiments on ten datasets show the effectiveness of our proposed methods.