Multi-dimensional classification via kNN feature augmentation

Abstract

In multi-dimensional classification (MDC), each training example is represented by a single instance (feature vector) while associated with multiple class variables, each of which specifies its class membership w.r.t. one specific class space. Most existing MDC approaches try to model dependencies among class variables in output space when inducing predictive functions, while the potential usefulness of manipulating feature space hasn’t been investigated. As a first attempt towards feature manipulation in input space for MDC, a simple yet effective approach named Kram is proposed which enriches the original feature space with augmented features based on kNN techniques. Specifically, simple counting statistics on the class membership of neighboring MDC examples as well as distance information between MDC examples and their k nearest neighbors are used to generate augmented feature vector. In this way, discriminative information from class space is expected to be brought into the feature space which would be helpful to the following MDC predictive model induction. To validate the effectiveness of the proposed feature augmentation techniques, comprehensive comparative studies are conducted over fifteen benchmark data sets. Compared to the original feature space, it is clearly shown that the kNN-augmented features generated by the proposed Kram approach can significantly improve generalization abilities of existing MDC approaches.