A Multi-armed Bandit Approach to Cost-Sensitive Decision Tree Learning

Abstract

Several authors have studied the problem of inducing decision trees that aim to minimize costs of misclassification and take account of costs of tests. The approaches adopted vary from modifying the information theoretic attribute selection measure used in greedy algorithms such as C4.5 to using methods such as bagging and boosting. This paper presents a new framework, based on game theory, which recognizes that there is a trade-off between the cost of using a test and the misclassification costs. Cost-sensitive learning is viewed as a Multi-Armed Bandit problem, leading to a novel cost-sensitive decision tree algorithm. The new algorithm is evaluated on five data sets and compared to six well known algorithms J48, EG2, MetaCost, AdaCostM1, ICET and ACT. The preliminary results are promising showing that the new multi-armed based algorithm can produce more cost-effective trees without compromising accuracy.