Induction of Decision Trees Using Relieff

Abstract

In the context of machine learning from examples this paper deals with the problem of estimating the quality of attributes with and without dependencies between then. Greedy search prevents current inductive machine learning algorithms to detect significant dependencies between the attributes. Recently, Kira and R.endell developed the RELIEF algorithm for estimating the quality of attributes that is able to detect dependencies between attributes. We show strong relation between R.ELIEF’s estimates and impurity functions, that are usually used for heuristic guidance of inductive learning algorithms. We propose to use RELIEFF, an extended version of RELIEF, instead of myopic impurity functions. We have reimplemented Assistant, a system for top down induction of decision trees, using RELIEFF as an estimator of attributes at each selection step. The algorithm is tested on several artificial and several real world problems. Results show the advantage of the presented approach to inductive learning and open a wide range of possibilities for using RELIEFF.