Self Organizing Decision Tree Based on Reinforcement Learning and its Application on State Space Partition

Abstract

Most of tree induction algorithms are typically based on a top-down greedy strategy that sometimes makes local optimal decision at each node. Meanwhile, this strategy may induce a larger tree than needed such that requires more redundant computation. To tackle the greedy problem, a reinforcement learning method is applied to grow the decision tree. The splitting criterion is based on long-term evaluations of payoff instead of immediate evaluations. In this work, a tree induction problem is regarded as a reinforcement learning problem and solved by the technique in that problem domain. The proposed method consists of two cycles: split estimation and tree growing. In split estimation cycle, an inducer estimates long-term evaluations of splits at visited nodes. In the second cycle, the inducer grows the tree by the learned long-term evaluations. A comparison with CART on several datasets is reported. The proposed method is then applied to tree-based reinforcement learning. The state spare partition in a critic actor model, adaptive heuristic critic (AHC), is replaced by a regression tree, which is constructed by the proposed method. The experimental results are also demonstrated to show the feasibility and high performance of the proposed system.