An Ordered Relatedness Metric for Relevance Identification

Abstract

In this paper we introduce a WordNet relations-based metric to determine semantic relatedness. Semantic relatedness is used to identify the degree of relevance between a review's text and a submission's text in order to determine whether the review pertains to the right submission. We use only Word Net since using additional corpuses or knowledge resources to determine similarity would be time consuming, especially when the metric is used to perform token-based pair wise comparison across documents. We compare our semantic relatedness metric with path and content-based measures that use only Word Net. We show that our metric is better than the other relatedness metrics at identifying relevance of academic reviews from Expertiza, a collaborative web-based learning application. We also show that our semantic relatedness metric produces higher correlations than most of the other metrics on the WordSim353 and Rubenstein & Good enough datasets.