Multilingual Text Research Articles

Analyzing Multilingual French and Russian Text using NLTK, spaCy, and Stanza

This lesson covers tokenization, part-of-speech tagging, and lemmatization, as well as automatic language detection, for non-English and multilingual text. You’ll learn how to use the Python packages NLTK, spaCy, and Stanza to analyze a multilingual Russian and French text.

At the Intersection of Translation Studies and Textual Scholarship

The Finnish Literature Society and the University of Turku organized an international conference in Turku in September 2023, aimed at scholars with an interest in translation, transmission, and textual scholarship. Translation studies and textual scholarship share many research interests, such as the processes involved in creating translations, the production of multilingual texts, and the networks related to translation. These themes were well-represented throughout the conference, with the most prevalent topics among the presenters including archival research, materiality, and genetic criticism. Articles based on the presentations are currently being collected for an edited volume.

Käännös- ja tekstuaalitieteiden risteyksessä

The Finnish Literature Society and the University of Turku organized an international conference in Turku in September 2023, aimed at scholars with an interest in translation, transmission, and textual scholarship. Translation studies and textual scholarship share many research interests, such as the processes involved in creating translations, the production of multilingual texts, and the networks related to translation. These themes were well-represented throughout the conference, with the most prevalent topics among the presenters including archival research, materiality, and genetic criticism. Articles based on the presentations are currently being collected for an edited volume.

Artificial Intelligence and Linguistic Landscape research

Abstract This article explores applications of artificial intelligence (AI) technologies in Linguistic Landscape research. Traditionally, LL research has relied on manual data collection and analysis, often involving photographs of public signage, advertisements, and other visual language displays. However, this manual approach can present challenges, including time-consuming data collection, inconsistent data quality, and potential researcher bias. Two AI technologies in particular hold promise for addressing these challenges in LL research: computer vision (CV) and large language models (LLMs). CV automates the identification and extraction of text from images, improving data accuracy and enabling large-scale image analysis. LLMs, based on natural language processing, can detect, translate, and interpret multilingual text. This article explores the affordances and challenges of using AI technologies in LL research and discusses methods to improve data collection, enhance accuracy, and support the analysis of multilingual environments. It also raises ethical issues and limitations of the technologies.

Translation as a restoration: Turkish translations of the Venice Charter

ABSTRACT The Venice Charter, translated into at least 28 different languages, has been a significant reference at both national and international levels in the historic preservation field. It was born as a multilingual text written in English, French, Italian and Russian as the final declaration of an international congress; the original form of the Charter owes its presence to the practice of translation. Cevat Erder, the translator of the Venice Charter into Turkish, proposed that ‘the Venice Charter should be preserved as an historic monument’. Accordingly, this study interrogates both the monumentality of the Charter and its method of conservation through analyses of the Turkish translations of the Venice Charter. Two practices, translation and restoration, are explored together to inquire into the challenges of transmitting authenticity. The role of translator and restorer in the afterlife of monuments is inquired by examining the creative agency of translation practice regarding conservation of the literary text.

Learning-based short text compression using BERT models

Learning-based data compression methods have gained significant attention in recent years. Although these methods achieve higher compression ratios compared to traditional techniques, their slow processing times make them less suitable for compressing large datasets, and they are generally more effective for short texts rather than longer ones. In this study, MLMCompress, a word-based text compression method that can utilize any BERT masked language model is introduced. The performance of MLMCompress is evaluated using four BERT models: two large models and two smaller models referred to as “tiny”. The large models are used without training, while the smaller models are fine-tuned. The results indicate that MLMCompress, when using the best-performing model, achieved 3838% higher compression ratios for English text and 42% higher compression ratios for multilingual text compared to NNCP, another learning-based method. Although the method does not yield better results than GPTZip, which has been developed in recent years, it achieves comparable outcomes while being up to 35 times faster in the worst-case scenario. Additionally, it demonstrated a 20% improvement in compression speed and a 180% improvement in decompression speed in the best case. Furthermore, MLMCompress outperforms traditional compression methods like Gzip and specialized short text compression methods such as Smaz and Shoco, particularly in compressing short texts, even when using smaller models.

Code-Switching in Translation: Linguistic Analysis of Multilingual Texts and their Translations

This research explores the occurrence of code-switching in translation, focusing on instances where translators switch between languages within the same textual context, both intentionally and unintentionally. This study investigated the influence of linguistic factors such as language structure, semantic complexity, and intended communicative purpose on code-switching. The analysis employs qualitative methods to identify the strategies and challenges that translators face in managing code-switching. The study reveals that code-switching significantly contributes to semantic and cultural nuances in multilingual texts, emphasizing its critical role in translation accuracy and reader comprehension. The findings offer theoretical understanding and practical guidance for translation practice, enhancing the framework of multilingual translation studies. This research highlights the importance of code-switching in preserving the integrity and richness of multilingual communication in translated works. This study was conducted through the author's personal efforts without any external funding or grants.

Deep Learning for Stylometry and Authorship Attribution: a Review of Literature

The application of deep learning techniques to stylometry and authorship attribution has emerged as a promising frontier in computational linguistics, offering new possibilities for understanding literary style and authorship in both historical and contemporary contexts. This review paper synthesizes recent advances in the use of deep learning models, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and transformer architectures, for identifying and attributing authorship based on stylistic analysis. We examine the effectiveness of these models in comparison to traditional statistical methods, highlighting their ability to capture complex linguistic patterns and nuances that are often overlooked by conventional approaches. Furthermore, we explore how deep learning models handle challenges such as multilingual texts, limited data, and variations across genres and periods. This review also addresses the interpretability of neural networks in the context of stylometry and discusses the implications of these methods for fields ranging from literary studies to digital forensics. By providing a comprehensive overview of the current state of research, this paper identifies key trends, challenges, and future directions for the application of deep learning to stylometry and authorship attribution.

Rethinking Multilingual Scene Text Spotting: A Novel Benchmark and a Character-Level Feature Based Approach

End-to-end multilingual scene text spotting aims to integrate scene text detection and recognition into a unified framework. Actually, the accuracy of text recognition largely depends on the accuracy of text detection. Due to the lackage of benchmarks with adequate and high-quality character-level annotations for multilingual scene text spotting, most of the existing methods train on the benchmarks only with word-level annotations. However, the performance of multilingual scene text spotting are not that satisfied training on the existing benchmarks, especially for those images with special layout or words out of vocabulary. In this paper, we proposed a simple YOLO-like baseline named CMSTR for character-level multilingual scene text spotting simultaneously and efficiently. Technically, for each text instance, we represent the character sequence as ordered points and model them with learnable explicit point queries. After passing a single decoder, the point queries have encoded requisite text semantics and locations, thus can be further decoded to the center line, boundary, script, and confidence of text via very simple prediction heads in parallel. Furthermore, we show the surprisingly good extensibility of our method, in terms of character class, language type, and task. On the one hand, DeepSolo not only performs well in English scenes but also masters the Chinese transcription with complex font structure and a thousand-level character classes. On the other hand, based on the extensibility of DeepSolo, we launch DeepSolo++ for multilingual text spotting, making a further step to let Transformer decoder with explicit points solo for multilingual text detection, recognition, and script identification all at once.

Recognition and normalization of multilingual symptom entities using in-domain-adapted BERT models and classification layers.

Due to the scarcity of available annotations in the biomedical domain, clinical natural language processing poses a substantial challenge, especially when applied to low-resource languages. This paper presents our contributions for the detection and normalization of clinical entities corresponding to symptoms, signs, and findings present in multilingual clinical texts. For this purpose, the three subtasks proposed in the SympTEMIST shared task of the Biocreative VIII conference have been addressed. For Subtask 1-named entity recognition in a Spanish corpus-an approach focused on BERT-based model assemblies pretrained on a proprietary oncology corpus was followed. Subtasks 2 and 3 of SympTEMIST address named entity linking (NEL) in Spanish and multilingual corpora, respectively. Our approach to these subtasks followed a classification strategy that starts from a bi-encoder trained by contrastive learning, for which several SapBERT-like models are explored. To apply this NEL approach to different languages, we have trained these models by leveraging the knowledge base of domain-specific medical concepts in Spanish supplied by the organizers, which we have translated into the other languages of interest by using machine translation tools. The results obtained in the three subtasks establish a new state of the art. Thus, for Subtask 1 we obtain precision results of 0.804, F1-score of 0.748, and recall of 0.699. For Subtask 2, we obtain performance gains of up to 5.5% in top-1 accuracy when the trained bi-encoder is followed by a WNT-softmax classification layer that is initialized with the mean of the embeddings of a subset of SNOMED-CT terms. For Subtask 3, the differences are even more pronounced, and our multilingual bi-encoder outperforms the other models analyzed in all languages except Swedish when combined with a WNT-softmax classification layer. Thus, the improvements in top-1 accuracy over the best bi-encoder model alone are 13% for Portuguese and 13.26% for Swedish. Database URL: https://doi.org/10.1093/database/baae087.

GPT is an effective tool for multilingual psychological text analysis

The social and behavioral sciences have been increasingly using automated text analysis to measure psychological constructs in text. We explore whether GPT, the large-language model (LLM) underlying the AI chatbot ChatGPT, can be used as a tool for automated psychological text analysis in several languages. Across 15 datasets (n = 47,925 manually annotated tweets and news headlines), we tested whether different versions of GPT (3.5 Turbo, 4, and 4 Turbo) can accurately detect psychological constructs (sentiment, discrete emotions, offensiveness, and moral foundations) across 12 languages. We found that GPT (r = 0.59 to 0.77) performed much better than English-language dictionary analysis (r = 0.20 to 0.30) at detecting psychological constructs as judged by manual annotators. GPT performed nearly as well as, and sometimes better than, several top-performing fine-tuned machine learning models. Moreover, GPT's performance improved across successive versions of the model, particularly for lesser-spoken languages, and became less expensive. Overall, GPT may be superior to many existing methods of automated text analysis, since it achieves relatively high accuracy across many languages, requires no training data, and is easy to use with simple prompts (e.g., "is this text negative?") and little coding experience. We provide sample code and a video tutorial for analyzing text with the GPT application programming interface. We argue that GPT and other LLMs help democratize automated text analysis by making advanced natural language processing capabilities more accessible, and may help facilitate more cross-linguistic research with understudied languages.

A Hybrid Scene Text Script Identification Network for Regional Indian Languages

In this work, we introduce WAFFNet, an attention-centric feature fusion architecture tailored for word-level multi-lingual scene text script identification. Motivated by the limitations of traditional approaches that rely exclusively on feature-based methods or deep learning strategies, our approach amalgamates statistical and deep features to bridge the gap. At the core of WAFFNet, we utilized the merits of Local Binary Pattern—a prominent descriptor capturing low-level texture features with high-dimensional, semantically-rich convolutional features. This fusion is judiciously augmented by a spatial attention mechanism, ensuring targeted emphasis on semantically critical regions of the input image. To address the class imbalance problem in multi-class classification scenarios, we employed a weighted objective function. This not only regularizes the learning process but also addresses the class imbalance problem. The architectural integrity of WAFFNet is preserved through an end-to-end training paradigm, leveraging transfer learning to expedite convergence and optimize performance metrics. Considering the under-representation of regional Indian languages in current datasets, we meticulously curated IIITG-STLI2023, a comprehensive dataset encapsulating English alongside six under-represented Indian languages: Hindi, Kannada, Malayalam, Telugu, Bengali, and Manipuri. Rigorous evaluation of the IIITG-STLI2023, as well as the established MLe2e and SIW-13 datasets, underscores WAFFNet’s supremacy over both traditional feature-engineering approaches as well as state-of-the-art deep learning frameworks. Thus, the proposed WAFFNet framework offers a robust and effective solution for language identification in scene text images.

DEALING WITH LANGUAGE VARIATION IN A SOURCE TEXT: MALAY SKETCHES AND ITS MALAY TRANSLATION

Background and Purpose: Although translation generally involves the transfer of meaning from the source language (L1) to the target language (L2), it may also involve a third language or L3, that is, any other language(s) found in either the source or target text. Language variation in a source text presents a challenge to translators, as they need to deal with both L1 and L3. This paper aims to explore the issue of language variation in a source text to see how L3 is rendered in translation. More specifically, the aim is to explore how L3 segments in French, Latin and Italian in a text in English entitled Malay Sketches are dealt with when the text is translated into Malay, and to determine the effects resulting from the solutions taken by the translators. Methodology: L3 in Malay Sketches are first identified. These L3 segments are subsequently mapped to their counterparts in the Malay translation, Gelagat & Ragam Orang Melayu. This mapping results in source language-target language pairs, which are then analysed to determine the solutions taken by the translators. These solutions are categorised based on the framework of translation solutions in L3 translation by Corrius and Zabalbeascoa (2011). Findings: It was found that L3 translation generally involves substitution in which L3 is translated into Malay. Consequently, language variation, a defining feature of the source text, becomes invisible in the Malay translation. Contributions: This study hopes to contribute to and expand current discourse on L3 in translation, especially in view of the fact that scant attention had been paid to L3 translation in Malaysia. Keywords: Malay Sketches, language variation, multilingual text, third language, translation. Cite as: Haroon, H. (2024). Dealing with language variation in a source text: Malay Sketches and its Malay translation. Journal of Nusantara Studies, 9(2), 537-556. http://dx.doi.org/10.24200/jonus.vol9iss2pp537-556

Cross-cultural adaptation of the writer-reader relationship in non-translated and translated English health information websites on HIV and TB diagnostic testing

The ideal multilingual health information website is relatable to all readers. Natives and immigrants should have a culturally adapted website in their language hosted by their place of residence that imparts the facts and incites a call to action to improve health, and ultimately reduces disease in the diverse community. The writer’s word choices and attitude as conveyed through the text influence the reader’s decision-making process. This paper will examine the differences between the non-translated and translated English versions of multilingual health information websites on HIV and TB diagnostic testing. These samples pertain to a large interdisciplinary study whose purpose is to determine whether the multilingual health communication websites are appropriately written regarding health literacy, and whether each cultural population, in terms of language adaptation, would receive the health information as intended. The study questions whether there exist differences between the translated and non-translated texts in English, Spanish and Catalan; this paper focuses on the two English sub-corpora. A comparable corpus of seventy-three multilingual health information websites underwent a quantitative and a qualitative analysis. The methodology is based on adaptations of Clerehan et al.’s (2005) Evaluative Linguistic Framework to assess the writer-reader relationship. The findings show statistically significant differences between the two English subcorpora as regards the writers’ and translators’ approach; the non-translated English sub-corpus contained more relational and engagement markers, whereas the translated English sub-corpus had more persuasion markers. These results should serve researchers and professionals in the translation and language sciences as well as the public health field for, respectively, future studies and techniques to improve the composition of multilingual health information texts in culturally diverse countries.

Особенности модификаций фразеологизмов в современных англо-, немецко-, франко- и испаноязычных версиях Библии: лингвокультурологический аспект

The purpose of the study is to establish the causes, ways and mechanisms of substitution of linguistic units that are a component of phraseological units functioning in multilingual texts of canonical versions of the Bible, in comparison with their modified versions in modern versions of the translation. The scientific novelty of the research lies in an integrated approach to the study of the process, trends and causes of modification of biblical phraseological units in the texts of the Bible translated into English, German, French and Spanish in diachrony; over the course of four centuries (XVII-XXI centuries), individual linguistic units that are components of quoted phraseological units have been subjected to substitution under the influence of both linguistic and extralingual factors, which in turn makes it possible to note the possible cultural recoding of these nations in the future. This article presents the results of a comparative analysis of biblical phraseological units and their modifications in the linguistic and cultural aspect. The linguistic factors of substitutions of components of quoted biblical phraseological units include such phenomena as the development of national languages, the phenomenon of polysemy, as well as the modernization of the religious language and the democratization of the presentation of the Holy Scriptures of Christians. Changing in the conditions of social and ordinary life of European nations are considered to be the extralinguistic factors.

Male product endorsers and the embodiment of modern Thai masculinities in skincare advertisements

As male endorsement in advertising continues to expand, and alternative forms of masculinity emerge in Asia and elsewhere, this article asks how skincare advertisements in Bangkok participate in these worldwide trends and how different forms of masculinity are represented and embodied in such advertising. The article analyses skincare advertising signs – banal sexed signs that form a significant part of Bangkok’s linguistic landscape – through the lens of social semiotics. The analysis reveals the use of both a more stereotypical form of masculinity associated with male heterosexuality and the ‘new man’, a masculinity associated with a ‘softer’ personality and Asian beauty trends. Important components contributing to the latter image include depictions of ‘feminine touch’ and multilingual texts. The findings illustrate how the new man serves as an important marketing tool, promoting not only a particular masculinity but also a modern, middle-class lifestyle.

Deep Learning Techniques for Detecting and Segmenting Text in Natural Scene Images: Review

Text detection and segmentation in natural scene images is an active research problem in computer vision and document analysis. Unlike scanned documents, scene text exhibits significant diversity in appearance, orientation, scale, font, and lighting conditions. In this review, survey the current state-of-the-art in techniques and methodologies aimed at detecting and segmenting text regions from images of natural scenes are presented. Both traditional approaches using hand-crafted features as well as modern data-driven deep learning methods will be discussed. The review will analyze common datasets, evaluation protocols and metrics used for benchmarking. Limitations of existing methods and open challenges in handling multilingual text, curved text, and efficiency will be highlighted. Promising future directions towards robust and generalizable scene text extraction systems will be identified. In summary, the review will provide a comprehensive overview of the advances, remaining challenges and future opportunities in developing automated systems for detecting and segmenting text in unconstrained natural images.

A multimodal contrastive analysis of regulations and instructions during the COVID-19 lockdown in the context of the Island of Madeira and the United Kingdom

Abstract The paper analyses aspects of multilingual and multimodal narratives employed during the COVID-19 lockdown, attempting to identify how they were used in professional and public environments. More specifically, the paper looks at editing choices, photography, design, drawing, colour, and writing to get a better understanding of multimodal communication, semiotics, image analysis, and their correlation with the written text and the way it has been perceived and interpreted, be it in online or print-based contexts. Methodologically, this is a contrastively inspired research project, looking specifically at Portuguese-speaking contexts of the Island of Madeira and English-speaking contexts of the United Kingdom, taking into account instructions about the COVID-19 pandemic. These are mainly bilingual English–Portuguese and multilingual texts, multimodal manifestations, and communicative and semiotic effectiveness.

Towards Media Monitoring: Detecting Known and Emerging Topics through Multilingual and Crosslingual Text Classification

This study aims to address challenges in media monitoring by enhancing closed-set topic classification in multilingual contexts (where both training and testing occur in several languages) and crosslingual contexts (where training is in English and testing spans all languages). To achieve this goal, we utilized a dataset from the European Media Monitoring webpage, which includes approximately 15,000 article titles across 18 topics in 58 different languages spanning a period of nine months from May 2022 to March 2023. Our research conducted comprehensive comparative analyses of nine approaches, encompassing a spectrum of embedding techniques (word, sentence, and contextual representations) and classifiers (trainable/fine-tunable, memory-based, and generative). Our findings reveal that the LaBSE+FFNN approach achieved the best performance, reaching macro-averaged F1-scores of 0.944 ± 0.015 and 0.946 ± 0.019 in both multilingual and crosslingual scenarios. LaBSE+FFNN’s similar performance in multilingual and crosslingual scenarios eliminates the need for machine translation into English. We also tackled the open-set topic classification problem by training a binary classifier capable of distinguishing between known and new topics with the average loss of ∼0.0017 ± 0.0002. Various feature types were investigated, reaffirming the robustness of LaBSE vectorization. The experiments demonstrate that, depending on the topic, new topics can be identified with accuracies above ∼0.796 and of ∼0.9 on average. Both closed-set and open-set topic classification modules, along with additional mechanisms for clustering new topics to organize and label them, are integrated into our media monitoring system, which is now used by our real client.

Semantically Enriched Cross-Lingual Sentence Embeddings for Crisis-related Social Media Texts

Tasks such as semantic search and clustering on crisis-related social media texts enhance our comprehension of crisis discourse, aiding decision-making and targeted interventions. Pre-trained language models have advanced performance in crisis informatics, but their contextual embeddings lack semantic meaningfulness. Although the CrisisTransformers family includes a sentence encoder to address the semanticity issue, it remains monolingual, processing only English texts. Furthermore, employing separate models for different languages leads to embeddings in distinct vector spaces, introducing challenges when comparing semantic similarities between multi-lingual texts. Therefore, we propose multi-lingual sentence encoders (CT-XLMR-SE and CT-mBERT-SE) that embed crisis-related social media texts for over 50 languages, such that texts with similar meanings are in close proximity within the same vector space, irrespective of language diversity. Results in sentence encoding and sentence matching tasks are promising, suggesting these models could serve as robust baselines when embedding multi-lingual crisis-related social media texts. The models are publicly available at: https://huggingface.co/crisistransformers.