Complexity of grammars by group theoretic methods

Abstract

Let G be a context free (phrase) structure grammar generating the context free language L. The set P = P G of all “generation histories” of words in L can be coded as words in some augmented alphabet. It is proved here that P = R∩G where R is a regular (finite automaton definable) set and G is a “free group kernel” or Dyck set, a result first proved by Chomsky and Schützenberger [3]. We can construct the Lower central series of the free group kernel G 1∼ G 2∼ … ∼ G n∼ …, so ∩ G n= G . Let P n= R∩G n , so ∩ P n =P. P n is the n-th order approximation of P.P n need not be a context free language but it can be computed by n cascade or sequential banks of counters (integers). We give two equivalent characterizations of P n , one “grammatical” and one “statistical”, which follow from the theorems of Magnus, Witt, M. Hall, etc. for free groups. The main new theoretical tool used here for the study of grammars is the Magnus transform on the free group, a→1+ a, a −1→1−a+a 2− a 3+a 4… , which acts like a non-commutative Fourier transform.