Algorithmic Aspects of Natural Language Processing

Abstract

Examples of natural languages are Chinese, English and Italian. They are called natural as they evolved in a more or less natural way, without too many deliberate considerations. This sets them apart from formal languages, amongst which are programming languages, which are designed to allow easy processing by computer algorithms. Typically, programs in programming languages such as C or Java can be processed (compiled) in close to linear time in their length. One particular feature that most programming languages have in common, and that allows for their fast processing, is absence of ambiguity. That is, only one structure, called a parse or parse tree, can be assigned to any program, and this parse can have only one meaning. Furthermore, the design of many programming languages is such that the single parse can be found deterministically, which means that every parsing step contributes a fragment of the resulting parse. As parses have a size linear in the length of the input, this explains why parsing is possible in linear time. Subsequent processing of the parse, for example in order to compile to machine code, is also commonly possible in close to linear time. Natural languages are quite different in this respect. Like programs in a programming language, sentences in a natural language can be assigned parses, but often the sentences are ambiguous and allow more than one parse. Even for a single parse, there may be ambiguity in the meanings of words or expressions. The existence of ambiguity in natural language is witnessed by frequent misunderstandings in daily life, but it is also an essential feature of poetry and puns.