Large-scale Corpus Research Articles

The Challenges of Large-Scale, Web-Based Language Datasets: Word Length and Predictability Revisited.

Language research has come to rely heavily on large-scale, web-based datasets. These datasets can present significant methodological challenges, requiring researchers to make a number of decisions about how they are collected, represented, and analyzed. These decisions often concern long-standing challenges in corpus-based language research, including determining what counts as a word, deciding which words should be analyzed, and matching sets of words across languages. We illustrate these challenges by revisiting "Word lengths are optimized for efficient communication" (Piantadosi, Tily, & Gibson, 2011), which found that word lengths in 11 languages are more strongly correlated with their average predictability (or average information content) than their frequency. Using what we argue to be best practices for large-scale corpus analyses, we find significantly attenuated support for this result and demonstrate that a stronger relationship obtains between word frequency and length for a majority of the languages in the sample. We consider the implications of the results for language research more broadly and provide several recommendations to researchers regarding bestpractices.

Weakly-Supervised Hierarchical Models for Predicting Persuasive Strategies in Good-faith Textual Requests

Modeling persuasive language has the potential to better facilitate our decision-making processes. Despite its importance, computational modeling of persuasion is still in its infancy, largely due to the lack of benchmark datasets that can provide quantitative labels of persuasive strategies to expedite this line of research. To this end, we introduce a large-scale multi-domain text corpus for modeling persuasive strategies in good-faith text requests. Moreover, we design a hierarchical weakly-supervised latent variable model that can leverage partially labeled data to predict such associated persuasive strategies for each sentence, where the supervision comes from both the overall document-level labels and very limited sentence-level labels. Experimental results showed that our proposed method outperformed existing semi-supervised baselines significantly. We have publicly released our code at https://github.com/GT-SALT/Persuasion_Strategy_WVAE.

Improving Neural Machine Translation Model with Deep Encoding Information

Availability of very high computational power along with the development of deep neural network (DNN) technology has enabled rapid progress of machine translation technology. The powerful representation ability of the deep neural network also enables the neural machine translation technology (NMT) to exploit the available large-scale bilingual parallel corpus as well as the computing power to provide a highly effective translation model. Nevertheless, the existing neural machine translation models only utilize the top layer encoder information, whereas the information available in deeper encoding layers is often ignored. This significantly constrains the performance of the translation model. To address this issue, in this paper, we propose a novel neural machine translation model which can fully exploit the deep encoding information. The core idea is to use different ways of aggregating the information from different encoding layers. We further design three different aggregation strategies including parallel layer, multi-layer, and dynamic layer encoding information aggregations. Three translation models are correspondingly trained and compared with the baseline transformer model for the Chinese-to-English translation task. The experimental results indicate that the BLEU-4 score of the proposed model has been increased by 0.89 compared with that of the benchmark model. Experiments demonstrate the effectiveness of the proposed method.

Reality bites: How the pandemic has begun to shape the way we, metaphorically, see the world

Since the onset of the coronavirus pandemic, there have been thousands of articles on the use of metaphor to describe the crisis. A Google search yields more than 7000 hits. Indeed, an avalanche of metaphors has already been used to describe the Covid-19 pandemic. From war and oceanic metaphors to the dreaded phrase ‘ramping up’, the language and images used by politicians, journalists, scientific experts, commentators, artists, comedians, and meme-makers to understand the crisis are not neutral constructs. But far more disappointing than the use of inappropriate or politically incorrect metaphors is the fact that there is no single scholarly article that systematically studies the influence of the pandemic on cognition, especially metaphorical conceptualization. That is, how has the coronavirus pandemic changed the meaning of home, love, Halloween, social media platforms, and so on for us? Technically speaking, how has the crisis triggered, prompted, or simply facilitated the selection and employment of particular conceptual metaphors or their linguistic and non-linguistic manifestations? This article, based on a large-scale corpus of political cartoons, aims to answer that question – how the pandemic itself becomes a metaphor. I show that various different metaphor targets (including WAR, ISRAEL, TURKEY, DONALD TRUMP, BENJAMIN NETANYAHU, ARAB DICTATORS, THE KUWAITI NATIONAL ASSEMBLY, THE JORDANIAN CABINET, THE IRANIAN REGIME, MILITANT GROUPS, POVERTY, RACISM, CALENDAR, CORRUPTION, INJUSTICE, RUMORS, POLITICAL ASSASSINATIONS, THE GLOBE, THE UNION JACK, THE EUROPEAN UNION, THE US FAR RIGHT, THE 50 STARS OF THE AMERCAN FLAG, US-RUSSIAN RELATIONS, HOUSEWIVES, PERSONAL BELONGINGS, LOVE, QURBAN BAYRAM, CHRISTMAS, HALLOWEEN, ARABIC DRAMA, SCHOOL BAG, AWARENESS, and SOCIAL MEDIA COMPANIES such as Facebook, among many others) are all explained with reference to coronavirus-related terms or the source domain of CORONAVIRUS/VACCINATION. The sheer frequency of occurrences of CORONAVIRUS metaphors (a total of 175 out of 497 relevant multimodal cartoons, that is, more than 35%) demonstrates and makes a case for the necessity to examine the effect of context, in particular topical news and physical circumstances, in the cognitive linguistic study of creative metaphor. In short, the results provide initial evidence that new viruses and diseases such as Covid-19 have a negative and significant effect on cognition (or shape societies’ worldviews).

How metaphoremes emerge

Abstract According to the dynamic view of metaphor, the complexities of metaphorical expressions are emergent products of language use. However, this view lacks an explicit mechanism to account for the process. This paper puts forward a model named single-scope integration network with entrenchment (SINE), and uses if-then rules in the model to explain the temporal order and regularities that the metaphoremes of a metaphor should follow in their emergence. The validity of the model is tested in the case studies of Chinese verb metaphors, which reveal four if-then rules that govern the metaphoreme emergence of Chinese verb metaphors. These if-then rules are obtained via the analysis of the occurrence order of metaphoremes by performing DepCluster, a machine learning tool for collostruction generation, over a large-scale diachronic corpus. The case studies demonstrate that the proposed model is applicable to Chinese verb metaphors.

Language and cultural cognition

Abstract In this paper, we analyze a large-scale corpus of Arab cartoons to measure the correspondence between grammatical gender in Arabic and personified gender in images. The results show that the effect is very strong for males (a near-perfect relationship between the two, grammatical and visual depiction), but the reverse is the case for females (the grammatical description is almost the opposite in perceived meaning of the graphical depiction). It can be a substantive cartoon effect. That is, there is more ambiguity in images depicting females due to some implicit cultural effect (i.e., males/gendered maleness dominates even in the text in ‘male-centric’ cultures). We look at the implications of this androcentric behavior for understanding the complex set of relationships linking language, thought, and culture. Such research will aid both gender studies and cognition scholarship based on multimodal stimuli.

Understanding Offline Password-Cracking Methods: A Large-Scale Empirical Study

Researchers proposed several data-driven methods to efficiently guess user-chosen passwords for password strength metering or password recovery in the past decades. However, these methods are usually evaluated under ad hoc scenarios with limited data sets. Thus, this motivates us to conduct a systematic and comparative investigation with a very large-scale data corpus for such state-of-the-art cracking methods. In this paper, we present the large-scale empirical study on password-cracking methods proposed by the academic community since 2005, leveraging about 220 million plaintext passwords leaked from 12 popular websites during the past decade. Specifically, we conduct our empirical evaluation in two cracking scenarios, i.e., cracking under extensive-knowledge and limited-knowledge. The evaluation concludes that no cracking method may outperform others from all aspects in these offline scenarios. The actual cracking performance is determined by multiple factors, including the underlying model principle along with dataset attributes such as length and structure characteristics. Then, we perform further evaluation by analyzing the set of cracked passwords in each targeting dataset. We get some interesting observations that make sense of many cracking behaviors and come up with some suggestions on how to choose a more effective password-cracking method under these two offline cracking scenarios.

Beyond base and collocate

Abstract Support verb constructions figure among the most frequently investigated topics in the literature on collocation. So far, most studies of this kind have focused on bipartite structures, consisting of a verbal collocate and a nominal base. Accordingly, the analysis of how support verbs are distributed has concentrated almost exclusively on the lexical control exerted by the base. In this article, we draw attention towards the influence exerted by the participation of verb and noun in more complex patterns of lexical co-occurrence. We contend that the distribution of the support verb collocate is contingent not only on the base noun but also on other elements of the lexical context. This highlights the need to enrich the theoretical framework of collocation analysis with the additional descriptive category of ‘second-order collocate’. The proposal is illustrated with two case studies using a large-scale web corpus of English.

Vocal development in a large-scale crosslinguistic corpus.

This study evaluates whether early vocalizations develop in similar ways in children across diverse cultural contexts. We analyze data from daylong audio recordings of 49 children (1-36months) from five different language/cultural backgrounds. Citizen scientists annotated these recordings to determine if child vocalizations contained canonical transitions or not (e.g., "ba" vs. "ee"). Results revealed that the proportion of clips reported to contain canonical transitions increased with age. Furthermore, this proportion exceeded 0.15 by around 7months, replicating and extending previous findings on canonical vocalization development but using data from the natural environments of a culturally and linguistically diverse sample. This work explores how crowdsourcing can be used to annotate corpora, helping establish developmental milestones relevant to multiple languages and cultures. Lower inter-annotator reliability on the crowdsourcing platform, relative to more traditional in-lab expert annotators, means that a larger number of unique annotators and/or annotations are required, and that crowdsourcing may not be a suitable method for more fine-grained annotation decisions. Audio clips used for this project are compiled into a large-scale infant vocalization corpus that is available for other researchers to use in future work.

Establishment of Parallel Text Corpus of Equipment Manufacturing Industry Based on Data Mining Technology

In the era of language big data, traditional data analysis methods can’t analyze semi-structured or unstructured data such as text, but all the contents in the equipment manufacturing corpus belong to text data. The equipment manufacturing corpus is a linguistic information base for legal activities and equipment manufacturing research, which aims to study equipment manufacturing and collect equipment manufacturing cases. At present, the construction of legal database in China is not perfect, and there are still many problems. In this paper, a method based on template transformation is proposed to automatically acquire parallel corpus on the Internet, and a method based on the number of transformation patterns and the retrieval and sorting of transformation patterns is adopted to verify bilingual parallel texts. This system can build a large-scale parallel corpus of equipment manufacturing industry by automatically acquiring a large number of parallel texts from the Internet.

Sentiment-Aware Emoji Insertion Via Sequence Tagging

Due to the booming popularity of online social networks, emojis have been widely used in online communication. As nonverbal language units, emojis help to convey emotions and express feelings. In this article, we focus on the sentiment-aware emoji insertion task, which predicts multiple emojis and their positions in a sentence conditioned on the plain texts and sentiment polarities. To facilitate future research in this field, we construct a large-scale emoji insertion corpus named “MultiEmoji,” which contains 420 000 English posts with at least one emoji per post. We formulate the insertion process as a sequence tagging task and apply a BERT-BiLSTM-CRF model to the insertion of emojis. Extensive experiments illustrate that our model outperforms existing methods by a large margin.

On the Relationships Between the Grammatical Genders of Inanimate Nouns and Their Co-Occurring Adjectives and Verbs

AbstractWe use large-scale corpora in six different gendered languages, along with tools from NLP and information theory, to test whether there is a relationship between the grammatical genders of inanimate nouns and the adjectives used to describe those nouns. For all six languages, we find that there is a statistically significant relationship. We also find that there are statistically significant relationships between the grammatical genders of inanimate nouns and the verbs that take those nouns as direct objects, as indirect objects, and as subjects. We defer deeper investigation of these relationships for future work.

Smart(er) Citations

Smart(er) Citations

The Role of Parental Input in the Early Acquisition of Japanese Politeness Distinctions

Japanese polite language (teineigo) varies with the speaker-addressee relationship as well as social norms. Descriptive studies have found that young Japanese children use polite-speech early in development. This claim was experimentally tested in 3- to 6-year-old Japanese children and correct use of polite verb forms was found even in the youngest children. The early acquisition of these verb forms is surprising, because there is a Japanese social norm that parental speech to children is mostly not polite, so it is not clear how children acquire the knowledge of how to use polite forms. To examine this, a large scale corpus analysis of polite language was performed using a probabilistic measure of the intended addressee. We confirmed that parental speech is mostly not polite, but parents also produced a substantial amount of polite language that varied appropriately with addressees and this can help to explain the early use of polite speech in Japanese children under experimental conditions.

Analysis on an Auto Increment Detection System of Chinese Disaster Weibo Text

With the rapid development of Internet information technology, the advantages of social media in terms of speed, content, form, and effect of communication are becoming increasingly significant. In recent years, more and more researchers have paid attention to the special value and role of social media tools in disaster information emergency management. Weibo is the most widely used Chinese social media tool. To effectively mine and apply the emergency function of disaster situation microblogs, a disaster situation information discovery and collection system capable of online incremental identification and collection are constructed for massive and disordered disaster microblog text streams. First, based on the deep learning- trained word vector model and a large-scale corpus, an unsupervised short-text feature representation method of disaster situation Weibo information is developed. According to the experimental results of the feature combination test and the training set scale test, the SVM algorithm was selected for disaster microblog information classification, which realized effective identification of disaster situation micro-bloggings. Then, the temporal information similarity and geographic information similarity are used to improve the single text similarity algorithm, and a Chinese disaster event online real-time detection model is constructed. Furthermore, the disaster-affected areas can be achieved in real-time based on the detection results. By crawling and classifying the micro-bloggings from the disaster-affected areas, it is possible to realize the incremental identification and collection of online disaster situation Weibo information. Finally, the empirical analysis of disaster events such as the “Leshan Earthquake” shows that the real- time intelligent identification and collection system for disaster situation Weibo micro-bloggings developed in this paper can obtain large-scale and useful data for disaster emergency management, which proving that this system is effective and efficient.

Simulating Speech Error Patterns Across Languages and Different Datasets.

Children’s speech acquisition is influenced by universal and language-specific forces. Some speech error patterns (or phonological processes) in children’s speech are observed in many languages, but the same error pattern may have different effects in different languages. We aimed to explore phonological effects of the same speech error patterns across different languages, target audiences and discourse modes, using a novel method for large-scale corpus investigation. As an additional aim, we investigated the face validity of five different phonological effect measures by relating them to subjective ratings of assumed effects on intelligibility, as provided by practicing speech-language pathologists. Six frequently attested speech error patterns were simulated in authentic corpus data: backing, fronting, stopping, /r/-weakening, cluster reduction and weak syllable deletion—each simulation resulting in a “misarticulated” version of the original corpus. Phonological effects were quantified using five separate metrics of phonological complexity and distance from expected target forms. Using Swedish child-speech data as a reference, phonological effects were compared between this reference and a) child speech in Norwegian and English, and b) data representing different modes of discourse (spoken/written) and target audiences (adults/children) in Swedish. Of the speech error patterns, backing—the one atypical pattern of those included—was found to cause the most detrimental effects, across languages as well as across modes and speaker ages. However, none of the measures reflects intuitive rankings as provided by clinicians regarding effects on intelligibility, thus corroborating earlier reports that phonological competence is not translatable into levels of intelligibility.

Construction of a database for the correct and incorrect use of considerate expressions, for the formation of a Japanese considerate expression dictionary

Professor Masaki Yamaoka, based at the Department of Humanities, Faculty of Letters, Soka University in Japan is leading a highly collaborative and multidisciplinary team that features researchers from academic institutions around the world, with a view to clarifying the actual situation of considerate expressions. It is hoped that this will make real contributions to anthropological fields that provide a more complete picture of what it means to be human.<br/> Some considerate expressions are common to all languages, while others are unique to a specific language. One that is common to many languages is the use of 'maybe' and 'may' which have the effect of making what is said less forceful or certain. By being less emphatic, it is thought that a consideration for the other person's feelings is sometimes part of the language process. It is a usage called 'hedge' in linguistics. For example, in English, 'it might be better to' is a kind of hedge. We can use this phrase as a considerate expression like that 'it might be better to leave here' instead of 'Leave here.'<br/> It is worth bearing in mind that, until now, research into considerate expressions has almost exclusively been conducted with researchers rather than the general public. Laboratory conditions can have the effect of providing an incomplete picture, so Yamaoka wants to make the research more reliable by collecting data on the actual usage of considerate expressions by members of the general public. A large-scale corpus of written and spoken language has already been developed in Japanese, Professor Yamaoka in the process of collecting as many examples of considerate expressions as possible and storing them in the database. The Japanese considerate expression dictionary can be composed by selecting information which is beneficial to Japanese learners from the database. In addition, Professor Yamaoka would also like to publish this database among researchers for the convenience of those who are engaged in research on considerate expressions in the future.

Improved Landcover Classification using Online Spectral Data Hallucination

We deal with the problem of information fusion driven satellite remote sensing (RS) image/scene classification and propose a generic hallucination architecture considering that all the available sensor information is present during training while some of the image modalities may be absent while testing. It is well-known that different sensors are capable of capturing complementary information for a given geographical area, and a classification module incorporating information from all the sources are expected to produce an improved performance as compared to considering only a subset of the modalities. However, the classical classifier systems inherently require all the features used to train the module to be present for the test instances as well, which may not always be possible for typical remote sensing applications (say, disaster management). As a remedy, we provide a robust solution in terms of a hallucination module that can approximate the missing modalities from the available ones during the decision-making stage. In order to ensure better knowledge transfer during modality hallucination, we explicitly incorporate concepts of knowledge distillation for the purpose of exploring the privileged (side) information in our framework and subsequently introduce an intuitive modular training approach. The proposed network is evaluated extensively on a large-scale corpus of PAN-MS image pairs (scene recognition) as well as on a benchmark hyperspectral image dataset (image classification) where we follow different experimental scenarios and find that the proposed hallucination based module indeed is capable of capturing the multi-source information, albeit the explicit absence of some of the sensor information, and aid in improved scene characterization.

Cloud-based intelligent self-diagnosis and department recommendation service using Chinese medical BERT

With the rapid development of hospital informatization and Internet medical service in recent years, most hospitals have launched online hospital appointment registration systems to remove patient queues and improve the efficiency of medical services. However, most of the patients lack professional medical knowledge and have no idea of how to choose department when registering. To instruct the patients to seek medical care and register effectively, we proposed CIDRS, an intelligent self-diagnosis and department recommendation framework based on Chinese medical Bidirectional Encoder Representations from Transformers (BERT) in the cloud computing environment. We also established a Chinese BERT model (CHMBERT) trained on a large-scale Chinese medical text corpus. This model was used to optimize self-diagnosis and department recommendation tasks. To solve the limited computing power of terminals, we deployed the proposed framework in a cloud computing environment based on container and micro-service technologies. Real-world medical datasets from hospitals were used in the experiments, and results showed that the proposed model was superior to the traditional deep learning models and other pre-trained language models in terms of performance.

Pretraining Techniques for Sequence-to-Sequence Voice Conversion

Sequence-to-sequence (seq2seq) voice conversion (VC) models are attractive owing to their ability to convert prosody. Nonetheless, without sufficient data, seq2seq VC models can suffer from unstable training and mispronunciation problems in the converted speech, thus far from practical. To tackle these shortcomings, we propose to transfer knowledge from other speech processing tasks where large-scale corpora are easily available, typically text-to-speech (TTS) and automatic speech recognition (ASR). We argue that VC models initialized with such pretrained ASR or TTS model parameters can generate effective hidden representations for high-fidelity, highly intelligible converted speech. In this work, we examine our proposed method in a parallel, one-to-one setting. We employed recurrent neural network (RNN)-based and Transformer based models, and through systematical experiments, we demonstrate the effectiveness of the pretraining scheme and the superiority of Transformer based models over RNN-based models in terms of intelligibility, naturalness, and similarity.