Cross-language Information Retrieval Experiments Research Articles

Cross-language information retrieval

AbstractThe article reviews the motivation behind the development of cross-language information retrieval (CLIR) – a relatively new area of information retrieval in multilingual textual databases, defines its objectives and position among the research disciplines dealing with various aspects of processing electronic texts. A short historical overview is followed by a description of the most important methodologies (document translation and query translation) and language resources used in connection with them. Regarding the resources, attention is focused on the two- and multilingual ontologies (thesauri, transfer lexicons and similarity thesauri), corpora, their construction and use with the CLIR experiments. The article primarily aims at illustrating the various methodological approaches, while the functioning of particular systems is less prominent. There are no references to the condition of CLIR in Slovenia or to the existence of language resources suitable for processing Slovenian texts in CLIR systems, since this topic calls for a separate review.

Exploiting query logs for cross-lingual query suggestions

Query suggestion aims to suggest relevant queries for a given query, which helps users better specify their information needs. Previous work on query suggestion has been limited to the same language. In this article, we extend it to cross-lingual query suggestion (CLQS): for a query in one language, we suggest similar or relevant queries in other languages. This is very important to the scenarios of cross-language information retrieval (CLIR) and other related cross-lingual applications. Instead of relying on existing query translation technologies for CLQS, we present an effective means to map the input query of one language to queries of the other language in the query log. Important monolingual and cross-lingual information such as word translation relations and word co-occurrence statistics, and so on, are used to estimate the cross-lingual query similarity with a discriminative model. Benchmarks show that the resulting CLQS system significantly outperforms a baseline system that uses dictionary-based query translation. Besides, we evaluate CLQS with French-English and Chinese-English CLIR tasks on TREC-6 and NTCIR-4 collections, respectively. The CLIR experiments using typical retrieval models demonstrate that the CLQS-based approach has significantly higher effectiveness than several traditional query translation methods. We find that when combined with pseudo-relevance feedback, the effectiveness of CLIR using CLQS is enhanced for different pairs of languages.

Using Statistical Term Similarity for Sense Disambiguationin Cross-Language Information Retrieval

With the increasing availability of machine-readable bilingual dictionaries, dictionary-based automatic query translation has become a viable approach to Cross-Language Information Retrieval (CLIR). In this approach, resolving term ambiguity is a crucial step. We propose a sense disambiguation technique based on a term-similarity measure for selecting the right translation sense of a query term. In addition, we apply a query expansion technique which is also based on the term similarity measure to improve the effectiveness of the translation queries. The results of our Indonesian to English and English to Indonesian CLIR experiments demonstrate the effectiveness of the sense disambiguation technique. As for the query expansion technique, it is shown to be effective as long as the term ambiguity in the queries has been resolved. In the effort to solve the term ambiguity problem, we discovered that differences in the pattern of word-formation between the two languages render query translations from one language to the other difficult.