Large-scale Corpus Research Articles

Effects of verb and construction frequency in sentence comprehension

Abstract Two theoretical viewpoints provide different explanations about how people extract statistical regularities from input to assess the felicity of verb usage in a sentence. The lexical approach emphasizes the role of verb frequency in determining a verb’s distributional bias within a sentence, whereas the entrenchment hypothesis highlights the conjoined roles of the frequency information from both a verb and an argument structure construction. The present study tests these accounts by investigating Korean speakers’ interpretation of two dative patterns in Korean (Dative–Accusative and Accusative–Accusative). Through the analysis of a large-scale corpus, we calculated the frequency of each dative pattern as well as the frequency of dative verbs occurring therein. Using this information, we conducted an acceptability judgment task with Korean speakers by manipulating the dative type and the verb frequency. The results showed that the speakers’ acceptability rating behavior was affected by the interaction between the verb and construction frequency such that highly entrenched verb–construction combinations were evaluated to be more acceptable. Our finding supports the entrenchment hypothesis that emphasizes the conjoined roles of usage frequency of verbs and constructions in sentence comprehension.

Multimodality information fusion for automated machine translation

Machine translation is a popular automation approach for translating texts between different languages. Although traditionally it has a strong focus on natural language, images can potentially provide an additional source of information in machine translation. However, there are presently two challenges: (i) the lack of an effective fusion method to handle the triangular-mapping function between image, text, and semantic knowledge; and (ii) the accessibility of large-scale parallel corpus to train a model for generating accurate machine translations. To address these challenges, this work proposes an effective multimodality information fusion method for automated machine translation based on semi-supervised learning. The method fuses multimodality information, texts and images to deliver automated machine translation. Specifically, our objective fuses multimodalities with alignment in a multimodal attention network, which advances the method through the power of mapping text and image features to their semantic information with accuracy. Moreover, a semi-supervised learning method is utilized for its capability in using a small number of parallel corpus for supervised training on the basis of unsupervised training. Conducted on the Multi30k dataset, the experimental results shows the promising performance of our proposed fusion method compared with state-of-the-art approaches.

Masked prediction and interdependence network of the law using data from large-scale Japanese court judgments

Court judgments contain valuable information on how statutory laws and past court precedents are interpreted and how the interdependence structure among them evolves in the courtroom. Data-mining the evolving structure of such customs and norms that reflect myriad social values from a large-scale court judgment corpus is an essential task from both the academic and industrial perspectives. In this paper, using data from approximately 110,000 court judgments from Japan spanning the period 1998–2018 from the district to the supreme court level, we propose two tasks that grasp such a structure from court judgments and highlight the strengths and weaknesses of major machine learning models. One is a prediction task based on masked language modeling that connects textual information to legal codes and past court precedents. Another is a dynamic link prediction task where we predict the hidden interdependence structure in the law. We make quantitative and qualitative comparisons among major machine learning models to obtain insights for future developments.

Inter-BIN: Interaction-Based Cross-Architecture IoT Binary Similarity Comparison

The big wave of Internet of Things (IoT) malware reflects the fragility of the current IoT ecosystem. Research has found that IoT malware can spread quickly on devices of different processer architectures, which leads our attention to cross-architecture binary similarity comparison technology. The goal of binary similarity comparison is to determine whether the semantics of two binary snippets is similar. Existing learning-based approaches usually learn the representations of binary code snippets individually and perform similarity matching based on the distance metric, without considering inter-binary semantic interactions. Moreover, they often rely on the large-scale external code corpus for instruction embeddings pre-training, which is heavyweight and easy to suffer the out-of-vocabulary (OOV) problem. In this paper, we propose an interaction-based cross-architecture IoT binary similarity comparison system, Inter-BIN. Our key insight is to introduce interaction between instruction sequences by co-attention mechanism, which can flexibly perform soft alignment of semantically related instructions from different architectures. And we design a lightweight multi-feature fusion-based instruction embedding method, which can avoid the heavy workload and the OOV problem of previous approaches. Extensive experiments show that Inter-BIN can significantly outperform state-of-the-art approaches on cross-architecture binary similarity comparison tasks of different input granularities. Furthermore, we present an IoT malware function matching dataset from real network environments, CrossMal, containing 1,878,437 cross-architecture reuse function pairs. Experimental results on CrossMal prove that Inter-BIN is practical and scalable on real-world binary similarity comparison collections.

Interrogating a National Narrative with GPT-2

In this lesson, you will learn how to apply a Generative Pre-trained Transformer language model to a large-scale corpus so that you can locate broad themes and trends within written text.

The Arousal Video Game AnnotatIoN (AGAIN) Dataset

How can we model affect in a general fashion, across dissimilar tasks, and to which degree are such general representations of affect even possible? To address such questions and enable research towards general affective computing, this paper introduces The Arousal video Game AnnotatIoN (AGAIN) dataset. AGAIN is a large-scale affective corpus that features over 1,100 in-game videos (with corresponding gameplay data) from nine different games, which are annotated for arousal from 124 participants in a first-person continuous fashion. Even though AGAIN is created for the purpose of investigating the generality of affective computing across dissimilar tasks, affect modelling can be studied within each of its 9 specific interactive games. To the best of our knowledge AGAIN is the largest -- over 37 hours of annotated video and game logs -- and most diverse publicly available affective dataset based on games as interactive affect elicitors.

A construção de corpus de larga escala da fala bilíngue de crianças e da fala bilíngue dirigida à criança, anotado e alinhado aos arquivos de áudio: desafios, soluções e implicações para a pesquisa

ABSTRACT The BiRCh Project (The Corpus of Bilingual Russian Child Speech) involves collecting a longitudinal audio corpus of Russian spoken by children and their families in Russia, Ukraine, Germany, the U.S., and Canada. We are building a large-scale corpus based on a subset of this data, the “Parsed and Audio-aligned Corpus of Bilingual Russian Child and Child-directed Speech (BiRCh)” with two basic components: (1) 1-million-word transcripts which are time-aligned with the audio speech signal and fully textsearchable, and (2) a 500K-word morphologically annotated and parsed portion of the transcripts, also audio-aligned. We are using this corpus to investigate various phenomena in the linguistic input and the developmental trajectory of heritage bilinguals, e.g., case, gender, passives, impersonals, politeness markers, disfluencies, and discourse markers. This article focuses on the challenges and solutions of the BiRCh development and the implications for research on the richly annotated data provided by the corpus.

The persuasive essays for rating, selecting, and understanding argumentative and discourse elements (PERSUADE) corpus 1.0.

This paper introduces the Persuasive Essays for Rating, Selecting, and Understanding Argumentative and Discourse Elements (PERSUADE) corpus.The PERSUADE corpus is large-scale corpus of writing with annotated discourse elements. The goal of the corpus is to spur the development of new, open-source scoring algorithms that identify discourse elements in argumentative writing to open new avenues for the development of automatic writing evaluation systems that focus more specifically on the semantic and organizational elements of student writing.

New Vietnamese Corpus for Machine Reading Comprehension of Health News Articles

Machine reading comprehension is a natural language understanding task where the computing system is required to read a text and then find the answer to a specific question posed by a human. Large-scale and high-quality corpora are necessary for evaluating machine reading comprehension models. Furthermore, machine reading comprehension (MRC) for the health sector has potential for practical applications; nevertheless, MRC research in this domain is currently scarce. This article presents UIT-ViNewsQA, a new corpus for the Vietnamese language to evaluate MRC models for the healthcare textual domain. The corpus consists of 22,057 human-generated question-answer pairs. Crowd-workers create the questions and answers on a collection of 4,416 online Vietnamese healthcare news articles, where the answers are textual spans extracted from the corresponding articles. We introduce a process for creating a high-quality corpus for the Vietnamese machine reading comprehension task. Linguistically, our corpus accommodates diversity in question and answer types. In addition, we conduct experiments and compare the effectiveness of different MRC methods based on the neural networks and transformer architectures. Experimental results on our corpus show that the MRC system based on ALBERT architecture outperforms the neural network architectures and the BERT-based approach, an exact match score of 65.26% and an F1-score of 84.89%. The best machine model achieves about 10.90% F1-score less efficiently than humans, which proves that exploring machine models on UIT-ViNewsQA to surpass humans is challenging for researchers in the future. Our corpus is publicly available on our website: http://nlp.uit.edu.vn/datasets for research purposes.

Computer Vision and Internet Meme Genealogy: An Evaluation of Image Feature Matching as a Technique for Pattern Detection

ABSTRACT Internet memes are a fundamental aspect of digital culture. Despite being individual expressions, they vastly transcend the individual level as windows into and vehicles for wide-stretching social, cultural, and political narratives. Empirical research into meme culture is thriving, yet particularly compartmentalized. In the humanities and social sciences, most efforts involve in-depth linguistic and visual analyses of mostly handpicked examples of memes, begging the question on the origins and meanings of those particular expressions. In technical disciplines, such as computer science, efforts are focused on the large-scale identification and classification of meme images, as well as patterns of “viral” spread at scale. This contribution aims to bridge the chasm between depth and scale by introducing a three-step approach suitable for “computational grounded theoretical” studies in which (1) an automated procedure establishes formal links between meme images drawn from a large-scale corpus paving the way for (2) network analysis to infer patterns of relatedness and spread, and (3) practically classifying visually related images in file folders for the purpose of further local, hermeneutic analysis. The procedure is demonstrated and evaluated on two datasets: an artificially constructed, structured dataset and a naturally harvested unstructured dataset. Future horizons and domains of application are discussed.

Song authorship attribution: a lyrics and rhyme based approach

In this work, we apply authorship attribution to a large-scale corpus of song lyrics. As a sub-category of poetry, song lyrics embody cultural elements as well as stylistic attributes that are not present in prose. We draw attention to special characteristics such as repetitive sound patterns and rhyme based structures in lyrics that can be key to ownership, and present opportunities that cannot be employed for authorship attribution of other types of text such as tweets, emails, and blog posts. We first create a new balanced, large-scale data set of 12,000 song lyrics from 120 different artists. We propose CNN models for authorship attribution on this song lyric data set, in order to use structural information included in the lyrics, similarly to image classification. We conduct experiments at the character and sub-word levels that mostly reflect positional information. In addition, we use phoneme level features, which intrinsically involve attributes such as repetitions, rhyme, and meter, and represent elements unique to verse-based textual compositions. We attempt to discover idiosyncratic features and consequently author and genre associations by working with variants of CNN architectures that have been successfully used in other text classification domains. Our architecture choice results in a particular focus on lyric attributes residing in neighboring regions, since CNNs fail to apprehend long term textual dependencies. Finally, we empirically evaluate our results in comparison with the findings of previous test classification research from different domains.

Twieng: A Multi-Domain Twi-English Parallel Corpus for Machine Translation of the Twi Language, A Low-Resource African Language

A Twi-English parallel corpus is certainly an important resource for Machine Translation of Twi (ISO 639-3), a Low- Resource Language (LRL) which is mainly spoken in Ghana and Ivory Coast. Currently large-scale multidomain Twi- English parallel corpus is still unavailable partly due to the difficulties and the arduous efforts required in its design. A digital Twi lexicon curated purposely for linguistic research is also not available. In this paper, we present TWIENG – Twi English corpus, a large-scale multi-domain Twi-English parallel corpus and Twi lexicon, a digital Twi Dictionary. We discuss the data collection methodology, translation, alignment and compilation of the Twi-English parallel sentences and the technology we used to compile and host the corpus. Today’s parallel corpora are crawled from the web using web crawlers, the sentence pairs are processed, aligned, tokenized and compiled to create the corpus. We crawled English sentences from Ghanaian indigenous electronic news portals, Ghanaian Parliamentary Hansards, standard literature and also used crowdsourcing. The sentences are translated by professional translators and linguists, then aligned, tokenized and compiled. The corpus is curated using the sketch engine, a corpus manager and analysis software developed by Lexical Computing Limited. The corpus is manually evaluated by Twi professional linguists. The Corpus has 5,419 parallel sentences which were curled from local news portals, Ghana Parliament Hansard, The New Testament of the Twi Bible and through crowdsourcing via social media sites. CCS CONCEPTS • Computing Methodologies • Artificial Intelligence • Natural Language Processing

LANGUAGE EXPOSURE PREDICTS CHILDREN'S PHONETIC PATTERNING: EVIDENCE FROM LANGUAGE SHIFT.

Although understanding the role of the environment is central to language acquisition theory, rarely has this been studied for children's phonetic development, and receptive and expressive language experiences in the environment are not distinguished. This last distinction may be crucial for child speech production in particular, because production requires coordination of low-level speech-motor planning with high-level linguistic knowledge. In this study, the role of the environment is evaluated in a novel way-by studying phonetic development in a bilingual community undergoing rapid language shift. This sociolinguistic context provides a naturalistic gradient of the amount of children's exposure to two languages and the ratio of expressive to receptive experiences. A large-scale child language corpus encompassing over 500 hours of naturalistic South Bolivian Quechua and Spanish speech was efficiently annotated for children's and their caregivers' bilingual language use. These estimates were correlated with children's patterns in a series of speech production tasks. The role of the environment varied by outcome: children's expressive language experience best predicted their performance on a coarticulation-morphology measure, while their receptive experience predicted performance on a lower-level measure of vowel variability. Overall these bilingual exposure effects suggest a pathway for children's role in language change whereby language shift can result in different learning outcomes within a single speech community. Appropriate ways to model language exposure in development are discussed.

Correlating Automated and Human Evaluation of Code Documentation Generation Quality

Automatic code documentation generation has been a crucial task in the field of software engineering. It not only relieves developers from writing code documentation but also helps them to understand programs better. Specifically, deep-learning-based techniques that leverage large-scale source code corpora have been widely used in code documentation generation. These works tend to use automatic metrics (such as BLEU, METEOR, ROUGE, CIDEr, and SPICE) to evaluate different models. These metrics compare generated documentation to reference texts by measuring the overlapping words. Unfortunately, there is no evidence demonstrating the correlation between these metrics and human judgment. We conduct experiments on two popular code documentation generation tasks, code comment generation and commit message generation, to investigate the presence or absence of correlations between these metrics and human judgments. For each task, we replicate three state-of-the-art approaches and the generated documentation is evaluated automatically in terms of BLEU, METEOR, ROUGE-L, CIDEr, and SPICE. We also ask 24 participants to rate the generated documentation considering three aspects (i.e., language, content, and effectiveness). Each participant is given Java methods or commit diffs along with the target documentation to be rated. The results show that the ranking of generated documentation from automatic metrics is different from that evaluated by human annotators. Thus, these automatic metrics are not reliable enough to replace human evaluation for code documentation generation tasks. In addition, METEOR shows the strongest correlation (with moderate Pearson correlation r about 0.7) to human evaluation metrics. However, it is still much lower than the correlation observed between different annotators (with a high Pearson correlation r about 0.8) and correlations that are reported in the literature for other tasks (e.g., Neural Machine Translation [ 39 ]). Our study points to the need to develop specialized automated evaluation metrics that can correlate more closely to human evaluation metrics for code generation tasks.

Cartooning and sexism in the time of Covid-19: Metaphors and metonymies in the Arab mind.

Using a large-scale corpus of 706 coronavirus cartoons by male and female Arab artists, this study takes a fresh and more cognitive look at sexism in multimodal discourse. Specifically, it examines the role of salience and grammar (and hence of metaphor and metonymy) in gender bias and/or in discrimination against women. It argues that both men and women are vulnerable to the influence of stereotypical and outdated beliefs that create unconscious bias. But this raises the crucial issue of whether we can speak of 'overt' sexism in images. Issues around terminology and conceptualization are thus also investigated. Importantly, this paper makes the following contributions to feminist and cross-cultural pragmatics: (i) it brings a distinctly Arabic perspective to gender and language; (ii) it expands socio-cognitive pragmatics beyond spoken and written communication; (iii) it shows a close coupling between an Arabic grammar and other aspects of culture; and (iv) it has the potential for impact beyond academia, specifically in the sphere of coronavirus care or of health communication.

Book Language and Its Implications for Children’s Language, Literacy, and Development

The onset of literacy marks a significant change in children’s development. Written language is more complex than everyday conversation, and even books targeted at preschoolers contain more varied words and more complex syntax than child-directed speech does. We review the nature and content of children’s book language, focusing on recent large-scale corpus analyses that systematically compared written and spoken language. We argue that exposure to book language provides opportunities for learning words and syntactic constructions that are only rarely encountered in speech and that, in turn, this rich experience drives further developments in language and literacy. Moreover, we speculate that the range, variety, depth, and sophistication of book language provide key input that promotes children’s social and emotional development. Becoming literate changes things, and researchers need to better understand how and why reading experience shapes people’s minds and becomes associated with a range of skills and abilities across the life span.

Frame Detection in German Political Discourses: How Far Can We Go Without Large-Scale Manual Corpus Annotation?

Frame Detection in German Political Discourses: How Far Can We Go Without Large-Scale Manual Corpus Annotation?

Text Revision By On-the-Fly Representation Optimization

Text revision refers to a family of natural language generation tasks, where the source and target sequences share moderate resemblance in surface form but differentiate in attributes, such as text formality and simplicity. Current state-of-the-art methods formulate these tasks as sequence-to-sequence learning problems, which rely on large-scale parallel training corpus. In this paper, we present an iterative in-place editing approach for text revision, which requires no parallel data. In this approach, we simply fine-tune a pre-trained Transformer with masked language modeling and attribute classification. During inference, the editing at each iteration is realized by two-step span replacement. At the first step, the distributed representation of the text optimizes on the fly towards an attribute function. At the second step, a text span is masked and another new one is proposed conditioned on the optimized representation. The empirical experiments on two typical and important text revision tasks, text formalization and text simplification, show the effectiveness of our approach. It achieves competitive and even better performance than state-of-the-art supervised methods on text simplification, and gains better performance than strong unsupervised methods on text formalization. Our code and model are released at https://github.com/jingjingli01/OREO.

Self-Supervised Knowledge Assimilation for Expert-Layman Text Style Transfer

Expert-layman text style transfer technologies have the potential to improve communication between members of scientific communities and the general public. High-quality information produced by experts is often filled with difficult jargon laypeople struggle to understand. This is a particularly notable issue in the medical domain, where layman are often confused by medical text online. At present, two bottlenecks interfere with the goal of building high-quality medical expert-layman style transfer systems: a dearth of pretrained medical-domain language models spanning both expert and layman terminologies and a lack of parallel corpora for training the transfer task itself. To mitigate the first issue, we propose a novel language model (LM) pretraining task, Knowledge Base Assimilation, to synthesize pretraining data from the edges of a graph of expert- and layman-style medical terminology terms into an LM during self-supervised learning. To mitigate the second issue, we build a large-scale parallel corpus in the medical expert-layman domain using a margin-based criterion. Our experiments show that transformer-based models pretrained on knowledge base assimilation and other well-established pretraining tasks fine-tuning on our new parallel corpus leads to considerable improvement against expert-layman transfer benchmarks, gaining an average relative improvement of our human evaluation, the Overall Success Rate (OSR), by 106%.

Explaining variance in writers’ use of demonstratives: A corpus study demonstrating the importance of discourse genre

Demonstratives such as this and that are among the most frequently used words in texts. But what are the factors that determine whether a writer uses one demonstrative form (proximal this) or another (distal that)? Here we report a large-scale corpus analysis in three written genres to empirically contrast theories based on differences in referent activation and prominence with a recent proposal suggesting that genre is the main driver of written demonstrative variance. We consistently observe that discourse genre is indeed the main predictor of writers’ demonstrative variation in English text. More specifically, a clear preference for distal demonstratives is found when the addressee is considered more prominent in the given discourse setting (as in news reports), whereas an overall preference for proximal demonstratives is observed when the knowledgeable writer feels more responsibility for the produced discourse themselves, as in an expository context (e.g. wikipedia texts). In such expository contexts, proximal demonstratives hence indicate that the referent is psychologically situated near the writer, whereas in interactional and narrative discourse the writer uses distal demonstratives to reach out to the addressee. These findings shed new theoretical light on some of the most frequently used and studied words in human language.