A Partitioning Algorithm with Application in Pattern Classification and the Optimization of Decision Trees

Abstract

The efficient partitioning of a finite-dimensional space by a decision tree, each node of which corresponds to a comparison involving a single variable, is a problem occurring in pattern classification, piecewise-constant approximation, and in the efficient programming of decision trees. A two-stage algorithm is proposed. The first stage obtains a sufficient partition suboptimally, either by methods suggested in the paper or developed elsewhere; the second stage optimizes the results of the first stage through a dynamic programming approach. In pattern classification, the resulting decision rule yields the minimum average number of calculations to reach a decision. In approximation, arbitrary accuracy for a finite number of unique samples is possible. In programming decision trees, the expected number of computations to reach a decision is minimized.