Multilingual Information Retrieval System Research Articles

Advancements in Neural Machine Translation: Methodological Innovations and Empirical Insights for Cross-Linguistic Discourse Preservation

Abstract: Natural Language Processing (NLP), in the form recognizable today, really began to take hold in the 1980s, when machine learning helped propel it to soaring heights. However, due to a lack of processing power, machine learning, and to an extent, NLP, started slowing down in innovation and ideas and had almost ground to a relative halt, until the last decade, when a sudden increase in both productivity and interest in the machine learning helped increase the amount of knowledge in the space itself. This review provides several different case studies using different methodologies. The first paper was a deep analysis on how researchers were able to use Tesseract and Google Vision in tandem with automatic data mining methods to enrich the Cherokee language database in order to preserve it from extinction. The second paper takes a query translation-based approach toward translating English to Indian languages and utilizes a Multilingual Cross-Language Information Retrieval (MLCIR) system with tools such as Part of Speech Tagger (POST), Stop-Word, and Porter Stemmer. The third paper presents CoVe, which transfers knowledge from machine translation to improve performance on NLP tasks like sentiment analysis and question answering by using contextualized word vectors along with word embeddings, achieving new state-of-the-art results on some datasets. The fourth paper aims to translate English to Pakistan Sign Language (PSL) and also uses POST and goes through dependency analysis, sentence classification, and PSL using PLS trees. The fifth paper uses a Multilingual Neural Machine Translation (NMT) system for LowResource languages and incorporates two main models: a recurrent NMT and a transformer NMT. The sixth paper analyzes how a fine-tuned transformer model seems to work better than transformer models trained from scratch on high-resource languages, while vice-versa seems to occur for low-resource languages. The seventh paper adds to this by talking about how multilingual translation seems to work better than a back-translation model. Given the diverse array of approaches that could be used, we aim to identify the most efficient and correct methodology for future researchers to use in their work, based on the papers in this literature review.

A Review on Indexing Techniques and its application in Multilingual Information Retrieval System

To implement the indexing in multilingual dataset, the indexing process must know. This paper gives the brief about indexing and presents role of indexing, logical view of indexing, types of indexing, indexing process, data structure of indexing, representation of indexing and indexing framework. Indexing is an important concept to implement in retrieval process. By the indexing, retrieval process will reduce the time. The indexing part performs the complete article pool to shape index constructions that permit for resourceful search. These are typically inverted tables. Each article is so preprocessed and recorded to a vector demonstration. At the end indexing method is applied to the multilingual information retrieval system and shows the results.

Exploring Topic-language Preferences in Multilingual Swahili Information Retrieval in Tanzania

Habitual switching of languages is a common behaviour among polyglots when searching for information on the Web. Studies in information retrieval (IR) and multilingual information retrieval (MLIR) suggest that part of the reason for such regular switching of languages is the topic of search. Unlike survey-based studies, this study uses query and click-through logs. It exploits the querying and results selection behaviour of Swahili MLIR system users to explore how topic of search (query) is associated with language preferences—topic-language preferences. This article is based on a carefully controlled study using Swahili-speaking Web users in Tanzania who interacted with a guided multilingual search engine. From the statistical analysis of queries and click-through logs, it was revealed that language preferences may be associated with the topics of search. The results also suggest that language preferences are not static; they vary along the course of Web search from query to results selection. In most of the topics, users either had significantly no language preference or preferred to query in Kiswahili and changed their preference to either English or no preference for language when selecting/clicking on the results. The findings of this study might provide researchers with more insights in developing better MLIR systems that support certain types of users and in certain scenarios.

Toward multi-lingual information retrieval system based on internet linguistic diversity measurement

Abstract We introduce a method for measuring the quantity of online content of a set of languages at domain level. This measurement is used for building a Multi-Lingual Information Retrieval (MLIR) system that identifies which languages are strongly represented on the internet about a specific query topic. The system architecture includes two modules; the off-line module builds a linguistic diversity index for languages at topic level and the on-line module, where the suitable language for search is identified based the index for retrieving the relevant documents to the user query in that language. The conducted experiments explore the usefulness of building such an index and its usage effect on both of monolingual and traditional MLIR system. From the obtained results, it has been proven that the more internet resources, the better the accuracy of the retrieved results, and therefore the better the system performance.

An empirical analysis of user behaviour on multilingual information retrieval

PurposeThis paper aims to understand user information behaviours when they perform multilingual information retrieval. It also offers reference for the development of multilingual information retrieval systems and relevant service platforms.Design/methodology/approachThe authors designed an experiment on multilingual information retrieval with WorldWideScience, utilized Camtasia studio7 (a screen capturing and recording tool) to record overall operational processes of subjects and collected participants’ thought processes with think-aloud protocols. Meanwhile, a questionnaire survey and interviews were used to examine the subjects’ background information, their feelings for the experiment and their ideas about the experimental platform, respectively. Thirty-two valid data points were obtained by 41 subjects.FindingsThe users preferred their own language for retrieval. Most users from social science chose general search or advanced search freely according to the tasks. The majority of the participants selected key words directly from the tasks as search terms. Doctoral candidates were more likely to construct a search query with logic symbols. Translation tools were utilized for assisting retrieval and solving doubts of translation. When facing obstacles, users stayed on the original web page to explore continually, followed by back to homepage.Originality/valueThis paper provides a study of user behaviour through investigating how users behave on the whole process of retrieving multilingual information. The findings offer advice for optimizing the function of multilingual information retrieval systems and service platforms.

Information Retrieval System and Machine Translation: A Review

In this paper we introduce some of the most important areas of information retrieval i.e. Cross-lingual Information Retrieval (CLIR), Multi-lingual Information Retrieval (MLIR), Machine translation approaches and techniques. In today's growing nation, local database storage and retrieval is essential for the developing countries. CLIR deals with asking queries in one language and demand for retrieving documents in another language. MLIR deals with asking questions in one or more languages and retrieving documents in one or more different languages. Machine translation plays an important role to achieve the CLIR and MLIR system.

Effective medical resources searching using an ontology‐driven medical information retrieval system

PurposeAs international users increase rapidly, multilingual systems have become a very important service for global users. The purpose of this paper is to design and implement an ontology‐driven medical information retrieval (OMIR) system by building a medical ontology based on the Centers for Disease Control and Prevention's (CDC) medical records.Design/methodology/approachA traditional cataloging scheme is used as a navigation menu in the CDC system. This traditional cataloging scheme is transformed to a unique medical ontology for global users in the OMIR system. An experimental study was conducted on both an ontology‐driven medical information system (OMIR) and the CDC system.FindingsThe medical ontology can be used to filter out unsuitable resources based on semantic relationships. In addition, the recommended resources can be categorized and provide the patron with different languages to access resources. The OMIR system provides better relevancy and shorter search times compared with alternative systems.Research limitations/implicationsThe OMIR system is currently implemented for medical resources from the CDC. The developed method may also be applied to other domain areas.Originality/valueThis paper represents a practical method of building a multilingual medical information retrieval system and explains the functional use of ontological knowledge. This study provides insights into medical information seeking performance on the medical database systems.

Performance Assessment of AMRR and ADCG Metrics in MLIR and IR Systems

Multiple language documents retrieval is being done by using Multilingual Information Retrieval (MLIR) system. MLIR system deals with the use of queries in one language and retrieves the documents in various languages. The Query translation plays a central role in MLIR system research. We have used English as the source language and Hindi, French and German as the target languages. The experimental results are evaluated to analyze and compare the performance of proposed MLIR system metrics, Average Mean Reciprocal Rank (AMRR) and Average Discounted Cumulative Gain (ADCG), in Information Retrieval (IR) and MLIR system. Experimental results show that the performance of AMRR, ADCG in MLIR system has been improved 81.67%, 43.93% over IR system respectively.

Precision at K in Multilingual Information Retrieval

Retrieval (IR) is used to store and represent the knowledge and the retrieval of information relevant for a special user query. Multilingual Information Retrieval (MLIR) system helps the users to pose the query in one language and retrieve the documents in more than one language. One of the basic performance measures of IR systems is Precision. While this measure work well in monolingual web retrieval, not suitable for CLIR (Cross-lingual Information Retrieval) or MLIR where two or more languages are involved. This paper proposed a metric which measures Precision at K which is the proportion of relevant documents in the first K positions when more than one document languages involved in the retrieval system i.e. MLIR. Experimental results demonstrate that the proposed metric is effective in systems where more than one document languages involved in the retrieval.

A Generic Approach to Processing Parallel Corpora of the Europarl for Distributional Discourse Patterns

This work presents a generic approach to building a featured test set in the light of further query processing by multilingual information retrieval systems. Language pair phenomena are found crucial points both in the process of query translation and document retrieval. Therefore, our approach aimed at improvement of machine translation quality is presented on the base of the French aligned to English version of the Europarl corpora. We investigate frequency of the language discourse occurrences commonly used in speech and writing. Linguistic structures are extracted from the corpora together with their representatives in order to create a featured test set of English and French grammatical patterns. In multilingual retrieval process a selection of patterns with the highest frequency simultaneously in these two languages constitutes an indication to a user of such a query formulation that may result in the most relevant system responses. This study of linguistic properties shows how to make the query easier for automatic translation and consequently to improve the system responsiveness.

Minimizing cultural differences using ontology‐based multilingual and multicultural information retrieval system

Minimizing cultural differences using ontology‐based multilingual and multicultural information retrieval system