A Sense Based Similarity Measure for Cross-Lingual Documents

Abstract

As cross-lingual information retrieval attracts increasing attention, tools that measure cross-lingual document similarity become desirable. Since the way that people convey thoughts at the abstract concept level makes little, if any, difference in the languages they use, it is possible to measure semantic similarity between different lingual documents based on the concepts conveyed by the documents. In this paper, we use senses for document representation to alleviate the barrier of different languages and adopt fuzzy set functions to cope with the inherent fuzziness among senses and propose two document similarity measures- one based on Tversky's notion on similarity and the other on the much used information retrieval criterion. Their performances are compared experimentally. We only focus on documents in English and Chinese. But the proposed approach can be easily extended to process documents in other languages.