Cross-language Information Retrieval Performance Research Articles

An axiomatic approach to corpus-based cross-language information retrieval

A major challenge in cross-language information retrieval (CLIR) is the adoption of translation knowledge in retrieval models, as it affects term weighting which is known to highly impact the retrieval performance. Despite its importance, how different approaches for integration of translation knowledge into retrieval models relatively perform has not been analytically examined. In this paper, we present an analytical investigation of using translation knowledge in CLIR. In particular, by adopting the axiomatic analysis framework, we formulate impacts of using translation knowledge on document ranking as constraints that any cross-language retrieval model should satisfy. We then consider state-of-the-art CLIR methods and check whether they satisfy these constraints. Our study shows that none of the existing methods satisfies all constraints. Based on the defined constraints, we propose the hierarchical query modeling method for CLIR which satisfies more constraints and achieves a higher CLIR performance, compared to the existing methods.

Studying the effect and treatment of misspelled queries in Cross-Language Information Retrieval

In contrast with their monolingual counterparts, little attention has been paid to the effects that misspelled queries have on the performance of Cross-Language Information Retrieval (CLIR) systems. The present work makes a first attempt to fill this gap by extending our previous work on monolingual retrieval in order to study the impact that the progressive addition of misspellings to input queries has, this time, on the output of CLIR systems. Two approaches for dealing with this problem are analyzed in this paper. Firstly, the use of automatic spelling correction techniques for which, in turn, we consider two algorithms: the first one for the correction of isolated words and the second one for a correction based on the linguistic context of the misspelled word. The second approach to be studied is the use of character n-grams both as index terms and translation units, seeking to take advantage of their inherent robustness and language-independence. All these approaches have been tested on a from-Spanish-to-English CLIR system, that is, Spanish queries on English documents. Real, user-generated spelling errors have been used under a methodology that allows us to study the effectiveness of the different approaches to be tested and their behavior when confronted with different error rates. The results obtained show the great sensitiveness of classic word-based approaches to misspelled queries, although spelling correction techniques can mitigate such negative effects. On the other hand, the use of character n-grams provides great robustness against misspellings.

Extracting translations from comparable corpora for Cross-Language Information Retrieval using the language modeling framework

A main challenge in Cross-Language Information Retrieval (CLIR) is to estimate a proper translation model from available translation resources, since translation quality directly affects the retrieval performance. Among different translation resources, we focus on obtaining translation models from comparable corpora, because they provide appropriate translations for both languages and domains with limited linguistic resources. In this paper, we employ a two-step approach to build an effective translation model from comparable corpora, without requiring any additional linguistic resources, for the CLIR task. In the first step, translations are extracted by deriving correlations between source–target word pairs. These correlations are used to estimate word translation probabilities in the second step. We propose a language modeling approach for the first step, where modeling based on probability distribution provides two key advantages. First, our approach can be tuned easier in comparison with heuristically adjusted previous work. Second, it provides a principled basis for integrating additional lexical and translational relations to improve the accuracy of translations from comparable corpora. As an indication, we integrate monolingual relations of word co-occurrences into the process of translation extraction, which helps to extract more reliable translations for low-frequency words in a comparable corpus. Experimental results on an English–Persian comparable corpus show that our method outperforms the previous approaches in terms of both translation quality and the performance of CLIR. Indeed, the proposed method is naturally applicable to any comparable corpus, regardless of its languages. In addition, we demonstrate the significant impact of word translation probabilities, estimated in the second step of our approach, on the performance of CLIR.

Improving Performance of English-Hindi Cross Language Information Retrieval using Transliteration of Query Terms

The main issue in Cross Language Information Retrieval (CLIR) is the poor performance of retrieval in terms of average precision when compared to monolingual retrieval performance. The main reasons behind poor performance of CLIR are mismatching of query terms, lexical ambiguity and un-translated query terms. The existing problems of CLIR are needed to be addressed in order to increase the performance of the CLIR system. In this paper, we are putting our effort to solve the given problem by proposed an algorithm for improving the performance of English-Hindi CLIR system. We used all possible combination of Hindi translated query using transliteration of English query terms and choosing the best query among them for retrieval of documents. The experiment is performed on FIRE 2010 (Forum of Information Retrieval Evaluation) datasets. The experimental result show that the proposed approach gives better performance of English-Hindi CLIR system and also helps in overcoming existing problems and outperforms the existing English-Hindi CLIR system in terms of average precision.

Prediction of performance of cross-language information retrieval using automatic evaluation of translation

This study develops regression models for predicting the performance of cross-language information retrieval (CLIR). The model assumes that CLIR performance can be explained by two factors: (1) the ease of search inherent in each query and (2) the translation quality in the process of CLIR systems. As operational variables, monolingual information retrieval (IR) performance is used for measuring the ease of search, and the well-known evaluation metric BLEU is used to measure the translation quality. This study also proposes an alternative metric, weighted average for matched unigrams (WAMU), which is tailored to gauging translation quality for special IR purposes. The data for regression analysis are obtained from a retrieval experiment of English-to-Italian bilingual searches using the CLEF 2003 test collection. The CLIR and monolingual IR performances are measured by average precision score. The result shows that the proposed regression model can explain about 60% of the variation in CLIR performance, and WAMU has more predictive power than BLEU. A back translation method for applying the regression model to operational CLIR systems in real situations is discussed.

Statistical query translation models for cross-language information retrieval

Query translation is an important task in cross-language information retrieval (CLIR), which aims to determine the best translation words and weights for a query. This article presents three statistical query translation models that focus on the resolution of query translation ambiguities. All the models assume that the selection of the translation of a query term depends on the translations of other terms in the query. They differ in the way linguistic structures are detected and exploited. The co-occurrence model treats a query as a bag of words and uses all the other terms in the query as the context for translation disambiguation. The other two models exploit linguistic dependencies among terms. The noun phrase (NP) translation model detects NPs in a query, and translates each NP as a unit by assuming that the translation of a term only depends on other terms within the same NP. Similarly, the dependency translation model detects and translates dependency triples, such as verb-object, as units. The evaluations show that linguistic structures always lead to more precise translations. The experiments of CLIR on TREC Chinese collections show that all three models have a positive impact on query translation and lead to significant improvements of CLIR performance over the simple dictionary-based translation method. The best results are obtained by combining the three models.

Experiments with dictionary-based CLIR using graded relevance assessments: Improving effectiveness by pseudo-relevance feedback

Research on cross-language information retrieval (CLIR) has typically been restricted to settings using binary relevance assessments. In this paper, we present evaluation results for dictionary-based CLIR using graded relevance assessments in a best match retrieval environment. A text database containing newspaper articles and a related set of 35 search topics were used in the tests. First, monolingual baseline queries were automatically formed from the topics. Secondly, source language topics (in English, German, and Swedish) were automatically translated into the target language (Finnish), using structured target queries. The effectiveness of the translated queries was compared to that of the monolingual queries. Thirdly, pseudo-relevance feedback was used to expand the original target queries. CLIR performance was evaluated using three relevance thresholds: stringent, regular, and liberal. When regular or liberal threshold was used, a reasonable performance was achieved. Using stringent threshold, equally high performance could not be achieved. On all the relevance thresholds the performance of the translated queries was successfully raised by pseudo-relevance feedback based query expansion. However, the performance of the stringent threshold in relation to the other thresholds could not be raised by this method.

English-Arabic Cross-Language Information Retrieval Based on Parallel Documents

In cross-language information retrieval (CLIR), queries in a source language are used to retrieve relevant documents in a target language. This paper describes the automatic construction of a translation matrix for English-Arabic CLIR applications. English-Arabic parallel sentences are translated using an adaptation of the expected mutual information measure (EMIM) technique and a matrix of translation terms is created. We evaluate our translation matrix as part of a retrieval system by comparing the effectiveness of cross-language retrieval and monolingual retrieval. The results show that the performance of English-Arabic CLIR using the translation matrix for query translation is either comparable to or better than that of monolingual Arabic retrieval depending on the number of translation terms used and the type of indexing method chosen.

Translation events in cross-language information retrieval

Cross-Language Information Retrieval (CLIR) systems enable users to formulate queries in their native language to retrieve documents in foreign languages. Because queries and documents in CLIR do not necessarily share the same language, translation is needed before matching can take place. This translation step tends to cause a reduction in the retrieval performance of CLIR as compared to monolingual information retrieval. To understand the events occurring in crosslanguage retrieval query translation and the relation of these events to retrieval performance, the study explored the following research questions:

Japanese-English Cross-Language Information Retrieval Using Machine Translation and Pseudo-Relevance Feedback

This paper evaluates the effectiveness of cross-language information retrieval (CLIR) based on machine-translation (MT) where the search requests are in Japanese and the documents are in English [24]. Our experiments use a subcollection of the TREC test collections and two bilingual researchers for separately translating the TREC requests into Japanese. Our main findings are as follows: (1) With the aid of pseudo-relevance feedback, MT-based Japanese-English CLIR can be as effective as best-case monolingual retrieval. In particular, although poor MT quality often leads to poor initial CLIR performance, pseudo-relevance feedback can alleviate the harm in many cases; (2) The manual request translation process that is inherent in conventional CLIR performance evaluation can be a more dominant factor than query length and overall MT quality.