A recursive TF-ISF Based Sentence Retrieval Method with Local Context

Abstract

Sentence retrieval consists of retrieving relevant sentences from a document base in response to a query. Question answering, novelty detection, summarization, opinion mining and information provenance make use of sentence retrieval. Most of the sentence retrieval methods are trivial adaptations of document retrieval methods. However some newer sentence retrieval methods based on the language modeling framework successfully use some kind of context of sentences. Unlike that there is no successful improvement of the TF-ISF method that takes into account the context of sentences. In this paper we propose a recursive TF-ISF based method that takes into account the local context of a sentence. The context is considered the previous and next sentence of current sentence. We compared the new method to the TF-ISF baseline and to an earlier unsuccessful method that also incorporates a similar context into TF-ISF. We got statistically significant improvements of the results in comparison to both of the methods. Additional benefit of our method is the clear explicit model of the context that will allow us to automatically generate a document representation with context suitable for sentence retrieval which is important for our future work.