On the depth of decision trees over infinite 1-homogeneous binary information systems

Mikhail Moshkov

doi:10.1016/j.array.2021.100060

Abstract

Abstract In this paper, we study decision trees, which solve problems defined over a specific subclass of infinite information systems, namely: 1-homogeneous binary information systems. It is proved that the minimum depth of a decision tree (defined as a function on the number of attributes in a problem’s description) grows – in the worst case – logarithmically or linearly for each information system in this class. We consider a number of examples of infinite 1-homogeneous binary information systems, including one closely related to the decision trees constructed by the CART algorithm.

Highlights

Decision trees have been widely applied to solve problems in the fields of knowledge representation, classification, combinatorial optimization, computational geometry, and so forth, e.g. [7, 17, 21]
Decision trees over infinite sets of attributes, in particular, linear [10, 15, 17], quasilinear [17], algebraic decision trees [12, 23], and related to them algebraic computation trees [5, 11] have been intensively studied as algorithms in combinatorial optimization and computational geometry
There are two approaches to the study of decision trees over infinite sets of attributes: the local approach, where decision trees use only attributes from the problem description, and the global approach, where decision trees use arbitrary attributes from the considered infinite set of attributes [3, 17, 18]

Summary

Introduction

Decision trees have been widely applied to solve problems in the fields of knowledge representation, classification, combinatorial optimization, computational geometry, and so forth, e.g. [7, 17, 21]. For an arbitrary infinite information system, in the worst case, the minimum depth of a decision tree (as a function on the number of attributes in a problem’s description) either is bounded from below by a logarithm and from above by a logarithm to the power 1+ ε , where ε is an arbitrary positive constant or grows linearly. For each infinite 1-homogeneous binary information system, in the worst case, the minimum depth of a decision tree (as a function on the number of attributes in a problem’s description) grows either logarithmically or linearly. We define a partial order ≺ on the set of attributes F of an infinite 1-homogeneous binary information systems U = ( A, F ) : for any f1, f2 ∈ F , f1 ≺ f2 if and only if the equation f2 ( x) = 0 is a consequence of the equation f1 ( x) = 0.

Main notions

Examples of infinite 1-homogeneous binary information systems

Behavior of Shannon function

Conclusions