A Framework for a Decision Tree Learning Algorithm with K-NN

Abstract

AbstractIn this paper, we proposed a modified decision tree learning algorithm. We tried to improve the conventional decision tree learning algorithm. There are some approaches to do it. These methods have a modified learning phase and a decision tree made by them includes some new attributes and/or class label gotten by modified process. As a result, It is possible that exists modified decision tree learning algorithm degrade of the comprehensibility of a decision tree. So we focus on the prediction phase and modified it. Our proposed approach makes a binary decision tree based on ID3, which is one of well-known conventional decision tree learning algorithms and predicts the class label of new data items based on K-NN instead of the algorithm used in ID3 and most of the conventional decision tree learning algorithm. Most of the conventional decision tree learning algorithms predicts a class label based on the ratio of class labels in a leaf node. They select the class label which has the highest proportion of the leaf node. However, when it is not easy to classify dataset according to class labels, leaf nodes includes a lot of data items and class labels. It causes to decrease the accuracy rate. It is difficult to prepare good training dataset. So we predict a class label from k nearest neighbor data items selected by K-NN in a leaf node. We implemented three programs. First program is based on our proposed approach. Second program is based on the conventional decision tree learning algorithms and third program is based on K-NN. In order to evaluate our approach, we compared these programs using a part of open datasets from UCL learning repository. Experimental result shows our approach is better than others.KeywordsDecision tree learning algorithmK-NNID3Min-max normalizationz-score standardizationThe machine learning repository