Combining Active Semi-supervised Learning and Rare Category Detection

Abstract

Loveland, Rohan Kaplan, NoahThe task of exploring data sets that are too large to allow individual examination of all instances necessitates new tools to identify and group interesting classes. This work investigates a hybrid system that performs both rare category detection and active semi-supervised clustering using the previously developed Farpoint algorithm. The prior algorithm focused solely on rare category detection, while the current research incorporates a capability for simultaneously obtaining high clustering accuracy. We investigate two methods of constrained clustering. One method is a constrained variant of K-Means that favors compact clusters, while the other is a novel variant of hierarchical agglomerative clustering that determines a cut-off threshold based on constraint violation. Using artificially skewed MNIST data we empirically compare the relative performance of the two new methods with the original Farpoint algorithm. Results show substantial improvement in clustering accuracy.