A feature weighted penalty based dissimilarity measure for k-nearest neighbor classification with missing features

Abstract

The k-Nearest Neighbor (kNN) classifier is an elegant learning algorithm widely used because of its simple and non-parametric nature. However, like most learning algorithms, kNN cannot be directly applied to data plagued by missing features. We make use of the philosophy of a Penalized Dissimilarity Measure (PDM) and incorporate a PDM called the Feature Weighted Penalty based Dissimilarity (FWPD) into kNN, forming the kNN-FWPD classifier which can be directly applied to datasets with missing features, without any preprocessing (like marginalization or imputation). Extensive experimentation on simulations of four different missing feature mechanisms (using various datasets) suggests that the proposed method can handle the missing feature problem much more effectively compared to some of the popular imputation mechanisms (used in conjunction with kNN).