Recognizing Textual Entailment and Computational Semantics

Abstract

Recognizing textual entailment (RTE)—deciding whether one piece of text contains new information with respect to another piece of text—remains a big challenge in natural language processing. One attempt to deal with this problem is combining deep semantic analysis and logical inference, as is done in the Nutcracker RTE system. In doing so, various obstacles will be met on the way: robust semantic analysis, designing interfaces to state-of-the-art theorem provers, and acquiring relevant background knowledge. The coverage of the parser and semantic analysis component is high, yet performance on RTE examples yields high precision but low recall. An empirical study of Nutcracker’s output reveals that the true positives are caused by sophisticated linguistic analysis such as coordination, active-passive alternation, pronoun resolution and relative clauses; the small set of false positives are caused by insufficient syntactic and semantic analyses. But most importantly, the false negatives are produced mainly by lack of background knowledge that is only implicit in the RTE examples.