Large-scale Corpus Research Articles

Assessment of Computer-Aided Translation Quality Based on Large-Scale Corpora

CAT technology, utilizing translation memory and quality control tools, boosts translation efficiency and consistency. Yet, it faces challenges with cultural nuances, context, and creativity, requiring human intervention. This study explores leveraging large-scale corpora to enhance machine translation quality assessment accuracy and efficiency. Automated evaluation models surpass manual limitations, enhancing translation system performance and decision-making. Supervised and unsupervised machine translation quality assessment methods, coupled with data augmentation strategies, reduce model bias by reinforcing the connection between source and target languages. The proposed model achieves 94.8% and 92.6% correctness on Google Translate and Wikipedia datasets, slightly surpassing manual rates. Data augmentation and cascade evaluation notably improve translation evaluation accuracy, especially with large-scale corpora. Future research targets higher-quality pseudo-labeled data, refined evaluation models, and advanced assessment methods for diverse translation needs.

Representation of migrant workers in Chinese newspapers: combining topic modeling and critical discourse analysis

Abstract Chinese migrant workers have contributed greatly to the industrialization and urbanization of China and have thus long been a popular topic in news reports. Most previous studies on the news about this group have been qualitative or quantitative analyses based on small-scale datasets, with few based on large-scale corpus data. This study combined topic modeling and critical discourse analysis (CDA) to analyze 186,180 reports of migrant workers published by 906 Chinese newspapers between 2001 and 2021. We identified seventeen topics in the news that fell into five categories: “work and labor,” “support and care,” “rights and interests,” “homecoming,” and “social emergencies.” Overall, news reports on the group have become increasingly diverse, with various diachronic changes in the proportions of topic categories and specific topics. In addition, we found three major types of representation of migrant workers: as a disadvantaged group, as outsiders, and as a group vulnerable to risks. These negative representations might be closely related to government policies, ideology, media stance, and social reality (the broader societal context in which these individuals are situated).

Automated painting color matching technology based on semantic intelligence understanding

Painting color matching technology is widely used in the production and printing process of products. Traditional painting and color matching have been unable to meet market demands. Based on this, a large-scale corpus under the existing semantic intelligent understanding system is used as the knowledge source. The computer automated painting color matching model is constructed. It is applied in case studies to address issues such as unclear query intentions, mismatched system retrieval terms, and return errors caused by uncertain factors such as synonyms and polysemy. This provides new ideas for the application of semantic intelligence understanding and automated painting color matching technology. The experimental results showed that the precision, recall, and F1 of the method used in the research were 0.8639, 0.8026, and 0.8309, respectively, significantly superior to commonly used methods. This indicates that the proposed automated painting color matching technology based on semantic intelligent understanding has high performance, which can effectively meet the painting color matching requirements.

Introduction to the Proceedings of the 2023 Symposium ‘Historical English Word-Formation’

It has not yet been ten years since Klaus Dietz (2015: 1915) prophesised that "[f]uture research work [on historical word-formation in English] will profit by two kinds of new instruments: firstly, by the Dictionary of Old English (DOE) and its Web Corpus, by the Middle English Dictionary (MED) and by the nascent third edition of the Oxford English Dictionary (OED), and secondly, by new corpora of historical English." The proceedings of the 2023 symposium respond to those predictions in manifold ways. Under the heading ‘Historical English Word-Formation’, the organisers of the symposium intended to “bring together researchers studying diachronic English word-formation and to showcase current research in this area” (Majewski 2023: 287). Although no particular temporal, thematic, or methodological focus was asked for, the five essays provide answers to some of the general questions that the symposium had initially raised, namely: How have large-scale corpus analyses and respective computational tools helped us study diachronic changes in the formation of new words? Which recent insights are there into the frequency and productivity as well as the rules and restrictions of word-formation units and patterns in the history of English? Further, which roles do regional, social, medial, and other factors as well as text types and (non‑)literary genres play for the creation of new words? The five contributions to this special issue of ZWJW illustrate that, as Dietz had anticipated, the study of word formations in past stages of English has profited extremely from the advances made in computational research and Artificial Intelligence, yet they also delineate both their advantages and limitations.

Towards building multilingual language model for medicine

The development of open-source, multilingual medical language models can benefit a wide, linguistically diverse audience from different regions. To promote this domain, we present contributions from the following: First, we construct a multilingual medical corpus, containing approximately 25.5B tokens encompassing 6 main languages, termed as MMedC, enabling auto-regressive domain adaptation for general LLMs; Second, to monitor the development of multilingual medical LLMs, we propose a multilingual medical multi-choice question-answering benchmark with rationale, termed as MMedBench; Third, we have assessed a number of open-source large language models (LLMs) on our benchmark, along with those further auto-regressive trained on MMedC. Our final model, MMed-Llama 3, with only 8B parameters, achieves superior performance compared to all other open-source models on both MMedBench and English benchmarks, even rivaling GPT-4. In conclusion, in this work, We present a large-scale corpus, a benchmark and a series of models to support the development of multilingual medical LLMs.

Phoneme Recognition in Korean Singing Voices Using Self-Supervised English Speech Representations

In general, it is difficult to obtain a huge, labeled dataset for deep learning-based phoneme recognition in singing voices. Studying singing voices also offers inherent challenges, compared to speech, because of the distinct variations in pitch, duration, and intensity. This paper proposes a detouring method to overcome this insufficient dataset, and applies it to the recognition of Korean phonemes in singing voices. The method started with pre-training the HuBERT, a self-supervised speech representation model, on a large-scale English corpus. The model was then adapted to the Korean speech domain with a relatively small-scale Korean corpus, in which the Korean phonemes were interpreted as similar English ones. Finally, the speech-adapted model was again trained with a tiny-scale Korean singing voice corpus for speech–singing adaptation. In the final adaptation, melodic supervision was chosen, which utilizes pitch information to improve the performance. For evaluation, the performance on multi-level error rates based on Word Error Rate (WER) was taken. Using the HuBERT-based transfer learning for adaptation improved the phoneme-level error rate of Korean speech by as much as 31.19%. Again, on singing voices by melodic supervision, it improved the rate by 0.55%. The significant improvement in speech recognition underscores the considerable potential of a model equipped with general human voice representations captured from the English corpus that can improve phoneme recognition on less target speech data. Moreover, the musical variation in singing voices is beneficial for phoneme recognition in singing voices. The proposed method could be applied to the phoneme recognition of other languages that have less speech and singing voice corpora.

Amina: an Arabic multi-purpose integral news articles dataset

AbstractElectronic newspapers are one of the most common sources of Modern Standard Arabic. Existing datasets of Arabic news articles typically provide a title, body, and single label. Ignoring important features, like the article author, image, tags, and publication date, can degrade the efficacy of classification models. In this paper, we propose the Arabic multi-purpose integral news articles (AMINA) dataset. AMINA is a large-scale Arabic news corpus with over 1,850,000 articles collected from 9 Arabic newspapers from different countries. It includes all the article features: title, tags, publication date and time, location, author, article image and its caption, and the number of visits. To test the efficacy of the proposed dataset, three tasks were developed and validated: article textual content (classification and generation) and article image classification. For content classification, we experimented the performance of several state-of-the-art Arabic NLP models including AraBERT and CAMeL-BERT, etc. For content generation, the reformer architecture is adopted as a character text generation model. For image classification applied on Al-Sharq and Youm7 news portals, we have compared the performance of 10 pre-trained models including ConvNeXt, MaxViT, ResNet18, etc. The overall study verifies the significance and contribution of our newly introduced Arabic articles dataset. The AMINA dataset has been released at https://huggingface.co/datasets/MohamedZayton/AMINA.

Demonstratives in heritage Greek, Russian, and Turkish in Germany and the United States

Aims and Objectives: This study investigates the use of definite noun phrases involving demonstratives in adolescent and adult monolingually raised and heritage speakers of Greek, Russian, and Turkish with the following research questions: (1) Do heritage speakers of Greek, Russian, and Turkish align with monolingually raised speakers regarding the production of demonstratives? and (2) Do mode and register affect the use of demonstratives? Methodology: We conducted a corpus study on production data of heritage speakers of Greek, Russian, and Turkish residing in the United States and Germany and the respective monolingually raised speakers in Greece, Russia, and Turkey. The majority languages of the heritage speakers were German or English, respectively. Data were elicited in two distinct registers (formal vs informal) and in two distinct modes (spoken vs written). Participants were asked to narrate what happened in a short video showing a fictional minor car accident. Analysis: Oral and written narrations were annotated and analyzed using generalized linear mixed-effects regression modeling on the use of demonstratives by heritage and monolingually raised speakers accounting for individual variation, country of elicitation, mode, and register. Findings: The results show that heritage and monolingually raised speakers of Greek, Russian, and Turkish converge in their use of demonstratives. Also, mode and/or register significantly affect the production of definite noun phrases with demonstratives across all speaker groups. Originality: This is the first cross-linguistically comparable large-scale corpus study with ecologically valid production data of definite noun phrases with demonstratives in bilinguals. Significance: The study contributes to understanding the use of demonstratives in heritage and monolingual Greek, Russian, and Turkish. It provides insights into the use of demonstratives in languages with different determiner systems and the impact of mode and/or register, which seems to be pronounced roughly similarly across these languages.

Targeted Training Data Extraction—Neighborhood Comparison-Based Membership Inference Attacks in Large Language Models

A large language model refers to a deep learning model characterized by extensive parameters and pretraining on a large-scale corpus, utilized for processing natural language text and generating high-quality text output. The increasing deployment of large language models has brought significant attention to their associated privacy and security issues. Recent experiments have demonstrated that training data can be extracted from these models due to their memory effect. Initially, research on large language model training data extraction focused primarily on non-targeted methods. However, following the introduction of targeted training data extraction by Carlini et al., prefix-based extraction methods to generate suffixes have garnered considerable interest, although current extraction precision remains low. This paper focuses on the targeted extraction of training data, employing various methods to enhance the precision and speed of the extraction process. Building on the work of Yu et al., we conduct a comprehensive analysis of the impact of different suffix generation methods on the precision of suffix generation. Additionally, we examine the quality and diversity of text generated by various suffix generation strategies. The study also applies membership inference attacks based on neighborhood comparison to the extraction of training data in large language models, conducting thorough evaluations and comparisons. The effectiveness of membership inference attacks in extracting training data from large language models is assessed, and the performance of different membership inference attacks is compared. Hyperparameter tuning is performed on multiple parameters to enhance the extraction of training data. Experimental results indicate that the proposed method significantly improves extraction precision compared to previous approaches.

A transformer-based spelling error correction framework for Bangla and resource scarce Indic languages

Spelling error correction is the task of identifying and rectifying misspelled words in texts. It is a potential and active research topic in Natural Language Processing because of numerous applications in human language understanding. The phonetically or visually similar yet semantically distinct characters make it an arduous task in any language. Earlier efforts on spelling error correction in Bangla and resource-scarce Indic languages focused on rule-based, statistical, and machine learning-based methods which we found rather inefficient. In particular, machine learning-based approaches, which exhibit superior performance to rule-based and statistical methods, are ineffective as they correct each character regardless of its appropriateness. In this paper, we propose a novel detector-purificator-corrector framework, DPCSpell based on denoising transformers by addressing previous issues. In addition to that, we present a method for large-scale corpus creation from scratch which in turn resolves the resource limitation problem of any left-to-right scripted language. The empirical outcomes demonstrate the effectiveness of our approach, which outperforms previous state-of-the-art methods by attaining an exact match (EM) score of 94.78%, a precision score of 0.9487, a recall score of 0.9478, an f1 score of 0.948, an f0.5 score of 0.9483, and a modified accuracy (MA) score of 95.16% for Bangla spelling error correction. The models and corpus are publicly available at https://tinyurl.com/DPCSpell.

Analysing A/O Possession in Māori-Language Tweets

This article contributes the first corpus-based study of possession in Māori, the indigenous language of Aotearoa New Zealand. Like most Polynesian languages, Māori has a dual possessive system involving a choice between the so-called A and O categories. While Māori grammars describe these categories in terms of the inherent semantic relationship between the possessum and possessor, there have been no large-scale corpus analyses demonstrating their use in natural contexts. Social media provide invaluable opportunities for such linguistic studies, capturing contemporary language use while alleviating the burden of gathering data through traditional means. We operationalise semantic distinctions to investigate possession in Māori-language tweets, focusing on the [possessum a/o possessor] construction (e.g., te tīmatanga o te wiki ‘the beginning of the week’). In our corpus comprising 2500 tweets produced by more than 200 individuals, we find that users leverage a wide array of noun types encompassing many different semantic relationships. We observe not only the expected predominance of the O category, but also a tendency for examples described by Māori grammars as A-marked to instead be O-marked (59%). Although the A category persists in the corpus, our findings suggest that language change could be underway. Our primary dataset can be explored interactively online.

A Novel Model Combining Transformer and Bi-LSTM for News Categorization

News categorization (NC), the aim of which is to identify distinct categories of news through analyzing the contents, has acquired substantial progress since deep learning was introduced into the natural language processing (NLP) field. As a state-of-art model, transformer’s classification performance is not satisfied compared with recurrent neural network (RNN) and convolutional neural network (CNN) if it does not get pretrained. Based on the transformer model, this article proposes a novel framework that combines bidirectional long short-term memory (Bi-LSTM) network and transformer to solve this problem. In the suggested framework, the self-attention mechanism is substituted with Bi-LSTM to capture the semantic information from sentences. Meanwhile, an attention mechanism model is applied to focus on those important words and adjust their weights to solve the problem of long-distance information loss. With pooling network, the network complexity can be reduced and the main features can be highlighted by halving the dimension of the hidden state. Finally, after acquiring the hidden representation by the above structures, we utilize a contraction network to further capture the long-range associations from a text. Experiments on three large-scale corpora were performed to evaluate the suggested framework, and the results demonstrate that our model outperforms other models such as deep pyramid CNN (DPCNN), transformer.

Using Transfer Learning to Realize Low Resource Dungan Language Speech Synthesis

This article presents a transfer-learning-based method to improve the synthesized speech quality of the low-resource Dungan language. This improvement is accomplished by fine-tuning a pre-trained Mandarin acoustic model to a Dungan language acoustic model using a limited Dungan corpus within the Tacotron2+WaveRNN framework. Our method begins with developing a transformer-based Dungan text analyzer capable of generating unit sequences with embedded prosodic information from Dungan sentences. These unit sequences, along with the speech features, provide <unit sequence with prosodic labels, Mel spectrograms> pairs as the input of Tacotron2 to train the acoustic model. Concurrently, we pre-trained a Tacotron2-based Mandarin acoustic model using a large-scale Mandarin corpus. The model is then fine-tuned with a small-scale Dungan speech corpus to derive a Dungan acoustic model that autonomously learns the alignment and mapping of the units to the spectrograms. The resulting spectrograms are converted into waveforms via the WaveRNN vocoder, facilitating the synthesis of high-quality Mandarin or Dungan speech. Both subjective and objective experiments suggest that the proposed transfer learning-based Dungan speech synthesis achieves superior scores compared to models trained only with the Dungan corpus and other methods. Consequently, our method offers a strategy to achieve speech synthesis for low-resource languages by adding prosodic information and leveraging a similar, high-resource language corpus through transfer learning.

ExaAUAC: Arabic Twitter user age prediction corpus based on language and metadata features

Twitter is a rich resource for analyzing the contents of social media and extracting the age groups of users can be beneficial for recommender systems, marketing and advertising. Age detection task is an aspect of demographic information of users. In this study a large-scale corpus of Arabic Twitter users including 181k user profiles with diverse age groups consisting of −18, 18–24, 25–34, 35–49, 50–64, +65 is presented. The corpus is created by four methods: (1) collecting publicly available birthday announcement tweets using the Twitter Search application programming interface, (2) augmenting data, (3) fetching verified accounts, and (4) manual annotation. To have a best age detection model on the presented corpus, different evaluations are tested to find the model with highest accuracy and efficiency. Number of tweets, regression vs. classification, using metadata of users and tweets, using LSTM+CNN model vs. BERT are some parts of examinations done. Presented methodology is based on language and metadata features and final model is fine-tuned with BERT on 70k users and evaluated on 8200 manually annotated users. We show that our best model, compared with LSTM+CNN model and BERT-based similar model yields an improvement of up to 9% in F1-score and increment of 5% in accuracy, respectively. The model achieved macro-averaged F1-score of 44 on six age groups, and F1-score of 58 on three age groups of −25, 25–34, +35. The link of our proposed data is provided here: www.github.com/exaco/ExaAUAC.

Augmenting Low-Resource Cross-Lingual Summarization with Progression-Grounded Training and Prompting

Cross-lingual summarization (CLS) , generating summaries in one language from source documents in another language, offers invaluable assistance in enabling global access to information for people worldwide. State-of-the-art neural summarization models typically train or fine-tune language models on large-scale corpora. However, this is difficult to achieve in realistic low-resource scenarios due to the lack of domain-specific annotated data. In this paper, we present a novel cross-lingual summarization model that utilizes progressive training with mBART and employs reinforcement learning to optimize discrete prompts, which addresses low-resource cross-lingual summarization through a two-pronged approach. During training, we introduce a progressive approach based on mBART, which allows the pre-trained model to gradually acquire the ability to compress information, develop cross-lingual capabilities, and ultimately adapt to specific summarization tasks. During downstream summarization, we employ a discrete-prompts joint pre-trained model based on reinforcement learning optimization to achieve low-resource cross-lingual summarization. Experimental results on four cross-lingual summarization datasets demonstrate state-of-the-art performance and superiority compared to six baselines in low-resource scenarios.

TPKE-QA: A gapless few-shot extractive question answering approach via task-aware post-training and knowledge enhancement

Few-shot extractive question answering (EQA) is a challenging task in natural language processing, whose current methods are mainly based on pretrained language models (PLMs). Data augmentation is often employed to improve the answer predictions of EQA models in few-shot settings. However, due to the differences between pretraining objectives and the EQA task, as well as embedding space alignment bottlenecks, the performance of few-shot EQA models must be improved. We propose TPKE-QA, a few-shot extractive Question Answering approach via Task-aware Post-training and Knowledge Enhancement, with entity-noun-oriented span selection in post-training, which can automatically generate EQA-style examples from a large-scale unlabeled corpus. By post-training based on generated examples, the gap between PLMs and the EQA task is effectively filled. To avoid embedding space alignment issues, a knowledge-enhanced sequence generation and knowledge injection approach for the EQA task enables gapless knowledge enhancement and fine-tuning on the post-trained model. In experiments, TPKE-QA achieved state-of-the-art results in most few-shot settings on the MRQA 2019 benchmark.

The Ups and Downs of Black and White: Do Sensorimotor Metaphors Reflect an Evolved Perceptual Interface?

ABSTRACT The Implicit Association Test (IAT) was used to measure population levels of conceptual alignment among two polar sensory metaphors and clusters of concepts to which they are commonly applied. A total of 873 participants were tested online, to compare within- and between-cluster alignments of concepts associated with two different polar sensory metaphors (up/down and black/white). IAT results were sensitive to semantic alignments that were also picked up by Latent Semantic Analysis (LSA) using a large-scale corpus of English. However, even with these semantic alignments taken into account, the dual categorization results demonstrated strong metaphor-cluster alignments over and above the predictions of LSA. It is proposed that, rather than ontogenetic development of metaphoric concepts based on sensorimotor experience, conventionalized sensorimotor metaphors in language may be cognitive tools that provide language learners with insights supported by conceptual properties of a phylogenetically evolved perceptual interface.

Between syntax and morphology: German noun+verb units

We show that graphemic variation—at least in some writing systems—can be analysed in terms of grammatical variation given a usage- based probabilistic view of the grammar-graphemics interface. Concretely, we examine a type of noun+verb unit in German, which can be written as one word or two. We argue that the variation in writing is rooted in the units’ ambiguous status in between morphology (one word) and syntax (two words). The major influencing factors are shown to be the semantic relation between the noun and the verb (argument or oblique relation) and the morphosyntactic context. In prototypically nominal contexts, a reinterpretation of the unit as a noun+noun compound is facilitated, which favours spelling as one word, while in prototypically verbal contexts, a syntactic realisation and consequently spelling as two words is preferred. We report the results of two large-scale corpus studies and a controlled production experiment to corroborate our analysis.

A cross-attention augmented model for event-triggered context-aware story generation

Despite recent advancements, existing story generation systems continue to encounter difficulties in effectively incorporating contextual and event features, which greatly influence the quality of generated narratives. To tackle these challenges, we introduce a novel neural generation model, EtriCA, that enhances the relevance and coherence of generated stories by employing a cross-attention mechanism to map context features onto event sequences through residual mapping. This feature capturing mechanism enables our model to exploit logical relationships between events more effectively during the story generation process. To further enhance our proposed model, we employ a post-training framework for knowledge enhancement (KeEtriCA) on a large-scale book corpus. This allows EtriCA to adapt to a wider range of data samples. This results in approximately 5% improvement in automatic metrics and over 10% improvement in human evaluation. We conduct extensive experiments, including comparisons with state-of-the-art (SOTA) baseline models, to evaluate the performance of our framework on story generation. The experimental results, encompassing both automated metrics and human assessments, demonstrate the superiority of our model over existing state-of-the-art baselines. These results underscore the effectiveness of our model in leveraging context and event features to improve the quality of generated narratives.

So much for plain language: An analysis of the accessibility of U.S. federal laws over time.

Over the last 50 years, there have been efforts on behalf of the U.S. government to simplify legal documents for society at large. However, there has been no systematic evaluation of how effective these efforts-collectively referred to as the "plain-language movement"-have been. Here we report the results of a large-scale longitudinal corpus analysis (n ≈ 225 million words), in which we compared every law passed by congress with a comparably sized sample of English texts from four different baseline genres published during approximately the same time period. We also compared the entirety of the U.S. Code (the official compilation of all federal legislation currently in force) with a large sample of recently published texts from six baseline genres of English. We found that laws remain laden with features associated with psycholinguistic complexity-including center-embedding, passive voice, low-frequency jargon, capitalization, and sentence length-relative to the baseline genres of English, and that the prevalence of most of these features has not meaningfully declined since the initial onset of the plain-language efforts. These findings suggest top-down efforts to simplify legal texts have thus far remained largely ineffectual, despite the apparent tractability of these changes, and call into question the coherence and legitimacy of legal doctrines whose validity rests on the notion of laws being easily interpretable by laypeople. (PsycInfo Database Record (c) 2024 APA, all rights reserved).