On the Depth of Decision Trees with Hypotheses.

Mikhail Moshkov

doi:10.3390/e24010116

Abstract

In this paper, based on the results of rough set theory, test theory, and exact learning, we investigate decision trees over infinite sets of binary attributes represented as infinite binary information systems. We define the notion of a problem over an information system and study three functions of the Shannon type, which characterize the dependence in the worst case of the minimum depth of a decision tree solving a problem on the number of attributes in the problem description. The considered three functions correspond to (i) decision trees using attributes, (ii) decision trees using hypotheses (an analog of equivalence queries from exact learning), and (iii) decision trees using both attributes and hypotheses. The first function has two possible types of behavior: logarithmic and linear (this result follows from more general results published by the author earlier). The second and the third functions have three possible types of behavior: constant, logarithmic, and linear (these results were published by the author earlier without proofs that are given in the present paper). Based on the obtained results, we divided the set of all infinite binary information systems into four complexity classes. In each class, the type of behavior for each of the considered three functions does not change.

Highlights

Introduction of Decision Trees with HypothesesDecision trees are studied in different areas of computer science, in particular in exact learning [1], rough set theory [2,3,4], and test theory [5]
We extend the model considered in test theory and rough set theory by adding the notion of a hypothesis that is an analog of equivalence query
Based on the results of exact learning, rough set theory, and test theory [1,11,12,13,14,15,16], we study for an arbitrary infinite binary information system three functions of the Shannon type that characterize the growth in the worth case of the minimum depth of a decision tree solving a problem with the growth of the number of attributes in the problem description

Summary

Basic Notions

Let A be a set of elements and F be a set of functions from A to {0, 1}. Functions from. We say that a decision tree Γ over z solves the problem z relative to U if, for each element a ∈ A and for each complete path ξ in Γ such that a ∈ A(ξ ), the terminal node of the path ξ is labeled with the tuple z( a). The decision tree Γ over z solves the problem z relative to U if, for each complete path ξ in Γ, the set ∆U (z)π (ξ ) contains at most one tuple, and if this set contains exactly one tuple, the considered tuple is assigned to the terminal node of the path ξ. We denote by hU (z) the minimum depth of a decision tree over z, which solves z relative to U and uses both attributes from F (z) and hypotheses over z.

Main Results

Proof of Theorem 2

Proof of Theorem 3

Conclusions