Cross-language Information Retrieval Method Research Articles

An axiomatic approach to corpus-based cross-language information retrieval

A major challenge in cross-language information retrieval (CLIR) is the adoption of translation knowledge in retrieval models, as it affects term weighting which is known to highly impact the retrieval performance. Despite its importance, how different approaches for integration of translation knowledge into retrieval models relatively perform has not been analytically examined. In this paper, we present an analytical investigation of using translation knowledge in CLIR. In particular, by adopting the axiomatic analysis framework, we formulate impacts of using translation knowledge on document ranking as constraints that any cross-language retrieval model should satisfy. We then consider state-of-the-art CLIR methods and check whether they satisfy these constraints. Our study shows that none of the existing methods satisfies all constraints. Based on the defined constraints, we propose the hierarchical query modeling method for CLIR which satisfies more constraints and achieves a higher CLIR performance, compared to the existing methods.

Cross-Language Semantic Web Service Discovery to Improve the Selection Mechanism by using Data Mining Techniques

Web service became one of the important methods for communication through the internet and its usage increased in the levels of users and developers. Semantic web service represents the second generation of web services that contains more description and information about its contents. Searching and dealing with web service is done through process called web service discovery which returns a Semantic Web Service Description Language file (SWSDL) for each web service. This research aims to expand the semantic web service usage through adding the multilanguage capability to the web service’s discovery process and through recommending other web services to the user based on his history in using web services. These aims were achieved by modifying the web service discovery model through adding two important techniques the Cross Language Information Retrieval (CLIR) technique and the data mining association rules technique. This research proposed two sub models, the first sub model proposed the application of CLIR techniques and information retrieval method to support Bilingual Web service discovery process the second language that proposed here is Arabic. Text mining techniques were applied on SWSDL content and user’s query to be ready for CLIR methods, this sub model was tested on a curated catalogue of Life Science Web Services http://www.biocatalogue.org/ and achieving 99.38 % accuracy and 87.23 precision of the effectiveness of the monolingual system. The second sub model proposed a process of web service recommendation by applying the data mining techniques to suggest another web service beside the one he got from the discovery process based on the user’s history. This sub model was tested on the mention curated web services site and the results were 65 % of users chose services from the services that recommended by the proposed sub model.

QUERY TRANSLATION USING CONCEPTS SIMILARITY BASED ON QURAN ONTOLOGY FOR CROSS-LANGUAGE INFORMATION RETRIEVAL

In Cross-Language Information Retrieval (CLIR) process, the translation effects have a direct impact o n the accuracy of follow-up retrieval results. In diction ary-based approach, we are dealing with the words t hat have more than one meaning which can decrease the retrieval performance if the query translation retur n an incorrect translations. These issues need to be ove rcome using efficient technique. In this study we p roposed a Cross-Language Information Retrieval (CLIR) method based on domain ontology using Quran concepts for disambiguating translation of the query and to improve the dictionary-based query translation. For experimentation, we use Quran ontology written in E nglish and Malay languages as a bilingual parallelcorpora and Quran concepts as a resource for cross- language query translation along with dictionary-ba sed translation. For evaluation, we measure the perform ance of three IR systems. IR 1 is natural language query IR, IR 2 is natural language query CLIR based on dictionary (as a Baseline) and IR 3 is the retrieval of this research proposed method using Mean Average Precision (MAP) and average precision at 11 points of recall. The experimental result shows that our prop osed method brings significant improvement in retri eval accuracy for English document collections, but defi cient for Malay document collections. The proposed CLIR method can obtain query expansion effect and improve retrieval performance in certain language.

Cross-lingual latent semantic analysis

Cross-lingual information retrieval is a difficult task typically involving query translation into multiple languages followed by monolingual retrieval in each language. Latent Semantic Analysis allows cross-lingual retrieval without translating queries by working from an already existing corpus of translations. Thus, collecting such a corpus obviates the need to construct complicated translation tools, making this technique particularly applicable to querying less commercially appealing languages. First, we extend work on retrieval from an English-French corpora split into training and test sets to examine the effects of training on a corpus from a completely different. Success is measured by the proportion of direct translations correctly considered most similar by Latent Semantic Analysis. Secondly, an English only similarity task from the literature is also extended to train on a different corpus to the one being tested on. Here the degradation in performance is measured through examining the variation in the correlations between the inter-document similarity judgements calculated by Latent Semantic Analysis and an experimentally derived baseline of human judgements of inter-document similarity. Higher order indexing schemes discarding uncommon terms, sparse matrix representations and the removal of factors with very low eigenvalues are used to enhance efficiency. Performance degradation from exogenous training is shown in both cases. The best results occur using stopping, log-entropy weighting and over 500 factors. References K. Boerner. Extracting and visualizing semantic structures in retrieval results for browsing. In Peter J. Nuernberg, David L. Hicks and Richard Furuta, editors, Proceedings of the fifth ACM conference on Digital libraries, pages 234--235. ACM 2000. doi:http://doi.acm.org/10.1145/336597.336672 Deerwester, S. C., Dumais, S. T., Landauer, T. K., Fernas, G. W. and Harshman, R. A., Indexing by Latent Semantic Analysis, Journal of the American Society of Information Science, 41, 1990, 391--407. doi:10.1002/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9 S. T. Dumais, T. K. Landauer and M. L. Littman. Automatic cross-linguistic information retrieval using Latent Semantic Indexing. In SIGIR'96 - Workshop on Cross-Linguistic Information Retrieval, pages 16--23. ACM, 1996. T. K. Landauer and M. L. Littman. Fully automatic cross-language document retrieval using latent semantic indexing. In Gregory Grefenstette, editor, Proceedings of the Sixth Annual Conference of the UW Centre for the New Oxford English Dictionary and Text Research, pages 31--38. UW Centre for the New OED and Text Research, Waterloo Ontario, 1990. Landauer, T. K., Littman, M. L. and Stornetta, W. S., A statistical method for cross-language information retrieval. Unpublished manuscript, 1992. Landauer, T. K., Foltz, P. W. and Laham, D., Introduction to Latent Semantic Analysis, Discourse Processes, textbf{25}, 1998, 259--284. Lloyd, R. and Shakiban, C., Improvements in Latent Semantic Analysis, American Journal of Undergraduate Research, 3, 2004, 29--34. http://www.ajur.uni.edu/v3n2 B. Pincombe. Comparison of Human and Latent Semantic Analysis (LSA) Judgements of Pairwise Document Similarities for a News Corpus. Research Report DSTO-RR-0278. DSTO, 2004. http://dspace.dsto.defence.gov.au/dspace/bitstream/1947/3334/1/DSTO-RR-0278%0PR.pdf P. G. Young. Cross-language information retrieval using latent semantic indexing. Technical Report UT-CS-94-259. University of Tennessee, 1994. M. D. Lee, B. M. Pincombe and M. B. Welsh. An empirical evaluation of models of text document similarity. In Bruno G. Bara, Lawrence Barsalou and Monica Bucciarelli, editors, Proceedings of the 27th Annual Conference of the Cognitive Science Society, pages 1254--1259. Lawrence Erlbaum Associates, Mahwah, NJ, 2005. http://hdl.handle.net/2440/28910

Improving query translation in English–Korean cross-language information retrieval

Query translation is a viable method for cross-language information retrieval (CLIR), but it suffers from translation ambiguities caused by multiple translations of individual query terms. Previous research has employed various methods for disambiguation, including the method of selecting an individual target query term from multiple candidates by comparing their statistical associations with the candidate translations of other query terms. This paper proposes a new method where we examine all combinations of target query term translations corresponding to the source query terms, instead of looking at the candidates for each query term and selecting the best one at a time. The goodness value for a combination of target query terms is computed based on the association value between each pair of the terms in the combination. We tested our method using the NTCIR-3 English–Korean CLIR test collection. The results show some improvements regardless of the association measures we used.