On the Complexity of Teaching

Abstract

While most theoretical work in machine learning has focused on the complexity of learning, recently there has been some work on the complexity of teaching. In this paper we study the complexity of teaching by considering a variant of the on-line learning model in which a helpful teacher selects the instances. We measure the complexity of teaching a concept from a given concept class by a combinatorial measure we call the teaching dimension. Informally, the teaching dimension of a concept class is the minimum number of instances a teacher must reveal to uniquely identify any target concept chosen from the class. We show that this dimension measure is fundamentally different than the Vapnik-Chervonenkis dimension and the dimension measure of Natarajan. We also show that the problem of computing an optimal teaching sequence for a given target concept is equivalent to finding an optimal set covering, and describe a straightforward relation between the teaching dimension and learning with membership queries. Finally, we compute the teaching dimension for several well-studied concept classes and give some general results to aid in computing the teaching dimension for other concept classes.