Elementary formal systems, intrinsic complexity, and procrastination

Abstract

Abstract Recently, rich subclasses of elementary formal systems (EFS) have been shown to be identifiable in the limit from only positive data. Examples of these classes are Angluin's pattern languages, unions of pattern languages by Wright and Shinohara, and classes of languages definable by length-bounded elementary formal systems studied by Shinohara. The present paper employs two distinct bodies of abstract studies in the inductive inference literature to analyze the learnability of these concrete classes. The first approach uses constructive ordinals to bound the number of mind changes.ωdenotes the first limit ordinal. An ordinal mind change bound ofωmeans that identification can be carried out by a learner that after examining some element(s) of the language announces an upper bound on the number of mind changes it will make before converging; a bound ofω·2 means that the learner reserves the right to revise this upper bound once; a bound ofω·3 means the learner reserves the right to revise this upper bound twice, and so on. A bound ofω2means that identification can be carried out by a learner that announces an upper bound on the number of times it may revise its conjectured upper bound on the number of mind changes. It is shown in the present paper that the ordinal mind change complexity for identification of languages formed by unions of up to n pattern languages isωn. It is also shown that this bound is essential. Similar results are also shown to hold for classes definable by length-bounded elementary formal systems with up to n clauses. The second approach employs reductions to study the intrinsic complexity of learnable classes. It is shown that the class of languages formed by taking unions of up ton+1 pattern languages is a strictly more difficult learning problem than the class of languages formed by the union of up tonpattern languages. It is also shown that a similar hierarchy holds for the bound on the number of clauses in the case of languages definable by length-bounded EFS. In addition to building bridges between three distinct areas of inductive inference, viz., learnability of EFS subclasses, ordinal mind change complexity, and intrinsic complexity, this paper also presents results that relate topological properties of learnable classes to that of intrinsic complexity and ordinal mind change complexity. For example, it is shown that a class that is complete according to the reductions for intrinsic complexity has infinite elasticity. Since EFS languages and their learnability results have counterparts in traditional logic programming, the present paper demonstrates the possibility of using abstract results of inductive inference to gain insights into inductive logic programming.