Improving k Nearest Neighbors and Naïve Bayes Classifiers Through Space Transformations and Model Selection

Abstract

Improving classifiers’ performance is the goal of techniques like prototype selection, normalization, and feature mapping; these techniques aim to reduce the complexity and improve the accuracy of models. In this manuscript, we present a boosting artifact for the well-known single-label $k$ -Nearest Neighbors ( $k$ -NN) classifier and the Naive Bayes (NB) classifier. The improvement comes as a pipeline that includes several data transformations orchestrated by a model selection scheme. The construction of these classifiers relies on the composition of simpler parts found in several open-source libraries and can be effortlessly put together to replicate our proposal. We also explore ensembling and the effect of preprocessing and normalizing the data. We compare our approach experimentally with 17 popular classifiers using raw and rank-based scores on 34 different benchmarks; statistical tests support our results. For instance, our results regarding average performance ranks under balanced error rate show that the models created with our proposal achieve first, third, and fourth-best ranks, compared with 10th position of raw $k$ -NN and 14th of raw NB.