A hybrid framework for natural language processing of large data bases

Abstract

At the Information Retrieval Research Laboratory we are working on combining techniques in common use in information retrieval with concepts from artificial intelligence to provide enhanced processing of large data bases. Information retrieval research has provided the inverted file, which indexes the occurrence of each word in a text file. Our system will include a complete inverted index as an underlying representation of the natural language text and will impose additional information structures on that index. Specifically, the index will include: information about the semantic function of words at each of their occurrences, linguistic knowledge connecting the vocabulary nodes, "fact nodes" attached to vocabulary nodes describing entities (e.g. people), and procedural knowledge needed to find facts and build fact nodes - attached to word nodes and functioning as word demons. The information structure will have some features of a semantic net, some of a network of frames. The procedural and linguistic knowledge attached to a vocabulary node will fill the function of an ATN segment that is attached to the sentence ATN when the associated word is recognized and will also be responsible for recognizing limited kinds of facts and attaching them as attributes to the appropriate entity.