Multilingual Information Access Research Articles

MHaDiG: A Multilingual Humor-aided Multiparty Dialogue Generation in multimodal conversational setting

Building a multimodal humor-aided dialogue generation system in a multilingual setting is a very challenging task and it could even be more challenging in the Hindi language because of its morphological richness and free word order. Existing works only solved the problem of unimodal humorous dialogue generation, not humor-aided, in a monolingual setting. However, due to the tremendous growth in multimodal and multilingual content, there is a great demand to build multimodal humor-aided systems that support multilingual information access. To this end, in this paper, we propose a deep learning-based multimodal sentiment and emotion-aware humor-aided multiparty dialogue generation in multilingual (Hindi and English) setting. We evaluate our proposed approach (MHaDiG) on the recently released Sentiment, Humor, and Emotion-aware Multilingual Multimodal Multiparty Dataset (SHEMuD). Experimental results show the efficacy of MHaDiG and show the effect of multilingualism over monolingualism in the multimodal conversational setting.

An exploration of ethnic minorities' needs for multilingual information access of public digital cultural services

PurposeEthnic minorities (EMs), who make up a sizable proportion of multilingual users, are more likely to browse and search in their native language. It is helpful to identify multilingual users' information needs to provide public digital cultural services (PDCS) for making their life better.Design/methodology/approachThe in-context interview is an efficient way to explore EMs' information needs and evoke their daily experience with PDCS. The material from 31 one-on-one interviews with EMs in China was recorded and analyzed using thematic analysis.FindingsThe findings reveal that language proficiency is a critical factor influencing multilingual information access (MLIA) and multilingual users' information needs. Moreover, language ability, digital literacy and cultural literacy are important components of multilingual information literacy (MLIL), which is helpful for EMs to access PDCS. In light of Kochen's theory, the information needs of PDCS can be classified into the aroused need of resources, the recognized need of functions and services and expressed need. For the expressed need, it is necessary to develop a one-stop convergence platform of PDCS to process various requests of resources, functions and services in the future.Originality/valueThe findings will be valuable for governments, public institutions and social organizations in identifying, addressing and resolving these issues about PDCS.

Multistage BiCross encoder for multilingual access to COVID-19 health information.

The Coronavirus (COVID-19) pandemic has led to a rapidly growing 'infodemic' of health information online. This has motivated the need for accurate semantic search and retrieval of reliable COVID-19 information across millions of documents, in multiple languages. To address this challenge, this paper proposes a novel high precision and high recall neural Multistage BiCross encoder approach. It is a sequential three-stage ranking pipeline which uses the Okapi BM25 retrieval algorithm and transformer-based bi-encoder and cross-encoder to effectively rank the documents with respect to the given query. We present experimental results from our participation in the Multilingual Information Access (MLIA) shared task on COVID-19 multilingual semantic search. The independently evaluated MLIA results validate our approach and demonstrate that it outperforms other state-of-the-art approaches according to nearly all evaluation metrics in cases of both monolingual and bilingual runs.

Multilingual Information Access (MLIA) Tools on Google and WorldCat: Bi/Multilingual University Students’ Experience and Perceptions

This article reports on the results of an exploratory user-centered study that examined how technological advancements in natural language processing (NLP) such as the availability of multilingual information access (MLIA) tools impact the information searching behavior of bi/multilingual academic users. Thirty-one bi/multilingual students participated in a controlled lab-based user experiment in which they carried out two assigned tasks each on Google and WorldCat for a total of four tasks, and then completed a post experiment questionnaire. The captures from the experiment showed 86.7% of the participants using multilingual information access tools. Further analyses of the captures also showed that participants were more likely to use MLIA tools when the instructions for the task were stated in their native language. An independent samples t-test revealed that participants spent less time on their searches when they used MLIA tools. The study revealed considerable diversity in the information searching behavior of the participants, even within the same pair of languages, and even for the same user. Diversity was noted for instance, on which tasks MLIA tools were used and in how these tools were used. User-centered designed, personalized multilingual information retrieval (PMLIR) models could hold promise for best representing the information searching behavior of bi/multilingual users.

Metadata records machine translation combining multi‐engine outputs with limited parallel data

One way to facilitate Multilingual Information Access (MLIA) for digital libraries is to generate multilingual metadata records by applying Machine Translation (MT) techniques. Current online MT services are available and affordable, but are not always effective for creating multilingual metadata records. In this study, we implemented 3 different MT strategies and evaluated their performance when translating English metadata records to Chinese and Spanish. These strategies included combining MT results from 3 online MT systems (Google, Bing, and Yahoo!) with and without additional linguistic resources, such as manually‐generated parallel corpora, and metadata records in the two target languages obtained from international partners. The open‐source statistical MT platform Moses was applied to design and implement the three translation strategies. Human evaluation of the MT results using adequacy and fluency demonstrated that two of the strategies produced higher quality translations than individual online MT systems for both languages. Especially, adding small, manually‐generated parallel corpora of metadata records significantly improved translation performance. Our study suggested an effective and efficient MT approach for providing multilingual services for digital collections.

3.5K runs, 5K topics, 3M assessments and 70M measures: What trends in 10 years of Adhoc-ish CLEF?

Multilingual information access and retrieval is a key concern in today global society and, despite the considerable achievements over the past years, it still presents many challenges. In this context, experimental evaluation represents a key driver of innovation and multilinguality is tackled in several evaluation initiatives worldwide, such as CLEF in Europe, NTCIR in Japan and Asia, and FIRE in India. All these activities have run several evaluation cycles and there is a general consensus about their strong and positive impact on the development of multilingual information access systems. However, a systematic and quantitative assessment of the impact of evaluation initiatives on multilingual information access and retrieval over the long period is still missing.Therefore, in this paper we conduct the first systematic and large-scale longitudinal study on several CLEF Adhoc-ish tasks – namely the Adhoc, Robust, TEL, and GeoCLEF labs – in order to gain insights on the performance trends of monolingual, bilingual and multilingual information access systems, spanning several European and non-European languages, over a range of 10 years.We learned that monolingual retrieval exhibits a stable positive trend for many of the languages analyzed, even though the performance increase is not always steady from year to year due to the varying interests of the participants, who may not always be focused on just increasing performances. Bilingual retrieval demonstrates higher improvements in recent years – probably due to the better language resources now available – and it also outperforms monolingual retrieval in several cases. Multilingual retrieval shows improvements over the years and performances are comparable to those of bilingual and monolingual retrieval, and sometimes even better. Moreover, we have found evidence that the rule-of-thumb of a 3-year duration for an evaluation task is typically enough since top performances are usually reached by the third year and sometimes even by the second year, which then leaves room for research groups to investigate relevant research issues other than top performances.Overall, this study provides quantitative evidence that CLEF has achieved the objective which led to its establishment, i.e. making multilingual information access a reality for European languages. However, the outcomes of this paper not only indicate that CLEF has steered the community in the right direction, but they also highlight the many open challenges for multilinguality. For instance, multilingual technologies greatly depend on language resources and targeted evaluation cycles help not only in developing and improving them, but also in devising methodologies which are more and more language-independent. Another key aspect concerns multimodality, intended not only as the capability of providing access to information in multiple media, but also as the ability of integrating access and retrieval over different media and languages in a way that best fits with user needs and tasks.

Multilingual Information Retrieval & Use: Perceptions and Practices Amongst Bi/Multilingual Academic Users

In recent years, technological advancements in Natural Language Processing (NLP) such as machine translation have made it possible for users to access information in multiple languages, even those in which they may not be proficient. The current study investigated the information searching behavior of bi/multilingual academic users, and examined their practices and perceptions regarding searching for information on the Internet and on electronic databases. Bi/multilingual students were recruited from a Canadian university and a community college both located in London, Ontario. A total of 250 (N=250) students completed a web survey through a link that was embedded in an invitation e-mail. Results showed that though advancements in NLP technology have alleviated some of the linguistic related challenges that some bi/multilingual academic users face while searching for information online, language barriers do still exist for some especially at the query formulation stage. The study found that an increase in Multilingual Information Access (MLIA) tools on electronic databases coupled with appropriate information literacy instruction could be helpful in further alleviating language barriers.

CLEF 15th Birthday

2014 marks the 15th birthday for CLEF, an evaluation campaign activity which has applied the Cranfield evaluation paradigm to the testing of multilingual and multimodal information access systems in Europe. This paper provides a summary of the motivations which led to the establishment of CLEF, and a description of how it has evolved over the years, the major achievements, and what we see as the next challenges.

Report on the 3rd MUMIA Training School on Information Retrieval and Interactive Information Access, July 21-25, 2014, FORTH, Heraklion, Crete, Greece

MUMIA 1 is a research network (funded by the European Cooperation in Science and Technology COST) on Multilingual and Multifaceted Interactive Information Access. MUMIA started in 2010 and ends on December 2014. About 300 senior scientists, experienced researchers as well as early stage researchers and PhDs mostly from the IR/NLP/MT communities participate in the Action 2 . Overall, the objective of the Action is to implement and efficiently run a series of networking activities that will bring together key players from academia and industry and to enable cross fertilization of disciplines related to information access. Another objective is the exploration of the state of the art of the main disciplines involved in the model of MUMIA and to advance their mutual understanding, a key factor for the development of next generation search systems.

An investigation of effective and efficient multilingual information access to digital collections

This paper presents the background, research design, and current progress of a new project on exploring the application of various machine translation strategies working toward multilingual information access for digital collections.

Multilingual Digital Libraries: A review of issues in system-centered and user-centered studies, information retrieval and user behavior

PurposeThis study aims to identify, collect and critical review the research literature on Multilingual Digital Libraries in English language from 1997 to 2012. Design/methodology/approachThe present literature review has followed the rules of systematic review. In particular, the identified relevant papers were categorized based on their expressed aim on two core themes, that of system-centered and user-centered studies. The assigned papers were further analyzed and six sub-themes emerged for the system-centered studies and four for the user-centered studies. Additional categorization was also provided according to type of publication. FindingsThe literature concerning Multilingual Digital Libraries is vast and mainly focuses on two aspects the “System” and the “Users”. The majority of papers tried to meet the challenges raised for enabling multilingual information retrieval in Digital Libraries. Unfortunately, these efforts undertaken by a small number of researchers or research groups apparently working in isolation and therefore resulting in the development of numerous different tools and techniques. Relatively few studies have focused on the user and aimed to explore users' behavior and expectations when interacting with Multilingual Digital Libraries. As a result, further research is needed to reach to some tangible and usable findings. Originality/valueThis literature review captures the diversity of the research conducted regarding multilingual information access and retrieval in Digital Libraries. It organizes the vast literature in comprehensive themes and sub-themes enabling easy access to specific information. LimitationsThis study reviews only papers in English due to language restrictions from 1997 to 2012.

Multilingual Digital Libraries: A review of issues in system-centered and user-centered studies, information retrieval and user behavior

This study aims to identify, collect and critical review the research literature on Multilingual Digital Libraries in English language from 1997 to 2012.Design/methodology/approach: The present literature review has followed the rules of systematic review. In particular, the identified relevant papers were categorized based on their expressed aim on two core themes, that of system-centered and user-centered studies. The assigned papers were further analyzed and six sub-themes emerged for the system-centered studies and four for the user-centered studies. Additional categorization was also provided according to type of publication.Findings: The literature concerning Multilingual Digital Libraries is vast and mainly focuses on two aspects the “System” and the “Users”. The majority of papers tried to meet the challenges raised for enabling multilingual information retrieval in Digital Libraries. Unfortunately, these efforts undertaken by a small number of researchers or research groups apparently working in isolation and therefore resulting in the development of numerous different tools and techniques. Relatively few studies have focused on the user and aimed to explore users' behavior and expectations when interacting with Multilingual Digital Libraries. As a result, further research is needed to reach to some tangible and usable findings.Originality/value: This literature review captures the diversity of the research conducted regarding multilingual information access and retrieval in Digital Libraries. It organizes the vast literature in comprehensive themes and sub-themes enabling easy access to specific information.Limitations: This study reviews only papers in English due to language restrictions from 1997 to 2012.

CLEF 2013

The 2014 Conference and Labs of the Evaluation Forum (CLEF) event was held at the University of Sheffield, UK, on September 15–18, 2014. The conference was entitled “Information Access Evaluation meets Multilinguality, Multimodality and Interaction” and addressed issues around multilingual and multimodal information access, information interaction, as well as the evaluation of search systems. CLEF celebrated its 15th anniversary this year having been conceived in 2000 as the Cross-Language Evaluation Forum aimed at stimulating research and innovation in multimodal and multilingual information access and retrieval. Over the years, it has actively nurtured and engaged a vibrant, multidisciplinary research community in the study, design, and implementation of evaluation methods for multiple tasks using diverse data sets and in many languages. In the first 10 years, CLEF conducted a series of experimental labs that were reported annually at workshops held in conjunction with the European Conference on Digital Libraries (ECDL). In 2010, now a mature and well-respected evaluation forum, it expanded to include a complementary peer-reviewed conference for reporting the evaluation of information access and retrieval systems regardless of data type, format, language, etc. Since then CLEF has continued that format with keynotes, contributed papers, lab sessions, and poster sessions, including reports from other benchmarking initiatives from around the world. The CLEF 2014 conference had more than 150 participants from different academic institutions and industrial organisations. Though the majority (114) of participants come from Europe, there was also considerable interest in CLEF worldwide, with 20 participants from the Americas, 16 from Asia, 1 from Australia, and 2 from the Middle East.

A study of relevance feedback techniques in interactive multilingual information access

PurposeWith the vast amount of multilingual information available online, it becomes increasingly critical for libraries to use various multilingual information access techniques in order to effectively support patrons' online information requests. However, this is still a relatively under‐explored area. This paper aims to study the effectiveness and the adoptability of query expansion and translation enhancement in the context of interactive multilingual information access.Design/methodology/approachRelying on an interactive multilingual information access system called ICE‐TEA, the authors conducted a controlled experiment (English‐to‐Chinese translation) involving human subjects to assess the retrieval effectiveness, analyzed the collected search logs to examine users' behavior, and employed pre‐ and post‐questionnaires to obtain users' opinions about the system.FindingsThe results confirm that significant improvement in retrieval effectiveness can be achieved by combining query expansion with translation enhancement (as compared to a case when there is no relevance feedback). However, users' ability to understand, interact with and even perceive the complex process of searches involving the combination of query expansion and translation enhancement may greatly impact the effectiveness of the techniques. The results also confirm that human‐generated queries were short queries, which calls for careful consideration of how longer queries perform in real search because many search engines rely on longer and more complex queries.Originality/valueThis study examines two important relevance feedback techniques in the context of human‐involved multilingual information access. This study is a valuable addition to the information seeking behaviour literature.

Collaboration and crowdsourcing

PurposeThis study aims to understand key features of existing multilingual digital libraries and to suggest strategies for building and/or sustaining multilingual information access for digital libraries.Design/methodology/approachA case study approach was applied to examine four American multilingual digital libraries: Project Gutenberg, Meeting of Frontiers, The International Children's Digital Library, and the Latin American Open Archives Portal. This examination used a framework derived from digital library evaluation practice. The missions, goals, funding, partners, users, collections, services, and technologies of these digital libraries were analyzed to present their key multilingual features. The collaboration and crowdsourcing characteristics were highlighted and discussed.FindingsThese four multilingual libraries benefit substantially, both in the creation of the library and in its access, from the collaboration of groups domestic and international with different language expertise. For building the multilingual collection and services, some libraries involved both staff and users. For multilingual access to the collection, however, none of the libraries used machine translation or cross‐language information retrieval technologies.Research limitations/implicationsThe four cases are all publicly available digital libraries in the USA. Their features may not be applicable to digital libraries in other countries or to commercial digital information services.Practical implicationsWith the advancement of machine translation technologies and the wide application of social media, multilingual digital libraries may have even better opportunities to sustain their multilingual capabilities through crowdsourcing and the application of new technologies.Originality/valueThis study summarizes the key features of four existing multilingual digital libraries. It provides insights into important factors for building successful multilingual digital libraries. The suggested strategies may help digital library developers to design appropriate multilingual information access services.

A preliminary evaluation of metadata records machine translation

PurposeThe purpose of this study is to evaluate freely available machine translation (MT) services' performance in translating metadata records.Design/methodology/approachRandomly selected metadata records were translated from English into Chinese using Google, Bing, and SYSTRAN MT systems. These translations were then evaluated using a five point scale for both fluency and adequacy. Missing count (words not translated) and incorrect count (words incorrectly translated) were also recorded.FindingsConcerning both fluency and adequacy, Google and Bing's translations of more than 70 percent of test data received scores equal to or greater than three, representative of “non‐native Chinese” and “much coverage,” respectively. SYSTRAN scored lowest in both measures. However, these differences were not statistically significant. A Pearson correlation analysis demonstrated a strong relationship (r=0.86) between fluency and adequacy. Missing count and incorrect count strongly correlated with fluency and adequacy.Originality/valueMost existing digital collections can be accessed in English alone. Few digital collections in the USA support multilingual information access (MLIA) that enables users of differing languages to search, browse, recognize and use information in the collections. Human translation is one solution, but it is neither time nor cost effective for most libraries. This study serves as a first step to understand the performance of current MT systems and to design effective and efficient MLIA services for digital collections.

Associating Automatic Natural Language Processing to Serious Games and Virtual Worlds

We present our current research activities associating automatic natural language processing to serious games and virtual worlds. Several interesting scenarios have been developed: language learning, natural language generation, multilingual information, emotion detection, real-time translations, and non-intrusive access to linguistic information such as definitions or synonyms. Part of our work has contributed to the specification of the Multi Lingual Information Framework [ISO FDIS 24616], (MLIF,2011). Standardization will grant stability, interoperability and sustainability of an important part of our research activities, in particular, in the framework of representing and managing multilingual textual information.

Learning Machine Translation. * Cyril Goutte, Nicola Cancedda, Marc Dymetman, and George Foster

Learning Machine Translation is a collection of papers on using machine learning in machine translation (MT) from the workshop on Machine Learning for Multilingual Information Access, organized at the Neural Information Processing Systems (NIPS) 19 conference in December 2006. It is not a book for readers who would like to be initiated in MT, although it starts off with an introduction to statistical machine translation (SMT) by the editors themselves. Its description of the SMT paradigm is very technical and has clearly been written from a computer scientist’s viewpoint. A much clearer introduction to SMT, including examples and instructions on how to build your own MT system, is given by Koehn (2010). The book consists of two main parts. The first deals with enabling technologies—technologies that solve problems which are not exclusive to MT, but very relevant. Each chapter in this part applies machine learning to a different problem. The second part describes research into alternative solutions, involving machine learning, for certain components of standard SMT systems, trying to break through the current ceiling of translation quality that can be obtained using state-of-the-art phrase-based SMT systems.

Fourth international workshop on uncovering plagiarism, authorship, and social software misuse

The Fourth International Workshop on Uncovering Plagiarism, Authorship, and Social Software Misuse (PAN10) was held in conjunction with the 2010 Conference on Multilingual and Multimodal Information Access Evaluation (CLEF-10) in Padua, Italy. The workshop was organized as a competition covering two tasks: plagiarism detection and Wikipedia vandalism detection. This report gives a short overview of the plagiarism detection task. Detailed analyses of both tasks have been published as CLEF Notebook Papers [3, 6], which can be downloaded at www.webis.de/publications.

CLEF 2010 conference on multilingual and multimodal information access evaluation

CLEF 2010 conference on multilingual and multimodal information access evaluation