A Comparison of Classification Algorithms Based on The Number of Features

Abstract

As the most widely known data mining algorithm, classification algorithms have attracted wide attention. K-Nearest Neighbor (KNN) algorithm and decision tree algorithm are the two widely known algorithms in classification algorithms. Sometimes, people not sure how to choose the suitable to solve the classification problems. In this paper, we establish KNN algorithm model and decision tree ID3 algorithm model to analyze the accuracy of the two algorithms in the same data set with different number of features. Through the learning curve and cross validation, we find ID3 algorithm is better than KNN algorithm, and when the number of feature increased the accuracy of KNN is increasing while ID3 is decreased.