Computational consequences of agreement and ambiguity in natural language

Abstract

We argue that the modern computer science technique of computational complexity analysis can provide powerful insights into the algorithm-neutral analysis of information-processing tasks. In particular, we show that a simple, theory-neutral linguistic model of syntactic agreement and lexical ambiguity demonstrates that natural language parsing may be computationally intractable, extending the classic work of Chomsky and Miller (1963). Significantly, we show that it may be syntactic features rather than complex rules that can cause this difficulty. Informally, human languages and the computationally intractable satisfiability problem (SAT) share two costly computational mechanisms: both enforce agreement among terminal symbols across unbounded distances and both allow terminal symbol ambiguity. In natural languages, lexical elements may be required to agree (or disagree) on such features as person, number, and gender (e.g., subject/verb agreement in English); in SAT, agreement ensures the consistency of variable truth assignments. Lexical ambiguity can appear freely in natural language utterances ( can may be a noun, verb, or auxiliary), while a variable in a SAT formula may be either true of false. When coupled with a deterministic performance model, this complexity result explains a subtle psycholinguistic distinction between discovering and verifying the grammaticality of an utterance. Finally, the applicability of computational complexity analysis to other cognitive faculties such as vision is discussed.