An Unsupervised Technical Readability Ranking Model by Building a Conceptual Terrain in LSI

Abstract

Searching for domain-specific related information has gained a high popularity in recent years. Naturally, everyone is not at par with each other when it comes to knowledge about the concepts of a domain. A doctor may be well versed in her field of specialization and probably would search for advanced medical documents on the Internet. But she may look for a much simpler material related to Computer Programming. However, current information retrieval (IR) systems just return a mixed set of results based on similarity and popularity of the web pages. Existing methods which have tried to address the issue of matching readers with texts in domain-specific IR either use an ontology or some seed concepts thereby limiting their application in certain domains only. Moreover, readability methods cannot address the issue in domain-specific IR ranking because they fail to give precise prediction when applied on web pages. We address this problem in domain-specific search using a conceptual model where the sequence of the terms in a document is modeled as a connected conceptual terrain. Our model has achieved significant improvement in ranking documents by technical readability.