Corpus-based Models Research Articles

A case study on decompounding in Indian language IR

Abstract Decompounding is an essential preprocessing step in text-processing tasks such as machine translation, speech recognition, and information retrieval (IR). Here, the IR issues are explored from five viewpoints. (A) Does word decompounding impact the Indian language IR? If yes, to what extent? (B) Can corpus-based decompounding models be used in the Indian language IR? If yes, how? (C) Can machine learning and deep learning-based decompounding models be applied in the Indian language IR? If yes, how? (D) Among the different decompounding models (corpus-based, hybrid machine learning-based, and deep learning-based), which provides the best effectiveness in the IR domain? (E) Among the different IR models, which provides the best effectiveness from the IR perspective? This study proposes different corpus-based, hybrid machine learning-based, and deep learning-based decompounding models in Indian languages (Marathi, Hindi, and Sanskrit). Moreover, we evaluate the effectiveness of each activity from an IR perspective only. It is observed that the different decompounding models improve IR effectiveness. The deep learning-based decompounding models outperform the corpus-based and hybrid machine learning-based models in Indian language IR. Among the different deep learning-based models, the Bi-LSTM-A model performs best and improves mean average precision (MAP) by 28.02% in Marathi. Similarly, the Bi-RNN-A model improves MAP by 18.18% and 6.1% in Hindi and Sanskrit, respectively. Among the retrieval models, the In_expC2 model outperforms others in Marathi and Hindi, and the BB2 model outperforms others in Sanskrit.

Animacy effects in the English genitive alternation: comparing native speakers and EFL learner judgments with corpus data

Abstract Recent years have seen a heightened interest in the interface between language use and cognition in language learners. In this study, we investigate this interface further by conducting a rating task experiment on the intuitions of 25 native speakers and 101 low–intermediate to advanced learners of English as a Foreign Language regarding the acceptability of the genitive variants (the beauty of nature/nature’s beauty) in different contexts. These ratings were then compared against existing corpus-based statistical models that predict which variant is most likely in spoken language use with two mixed-effects linear regression models. The first model focused on the animacy of the possessor in particular, which has been found to have a different effect on native speakers and EFL learners in language use, whereas the second model tested how the ratings relate to the predictions as a whole. Results show that there is a larger discrepancy between language use and intuitions of low-proficiency learners compared to native speakers, which is partially because animate, collective, and inanimate possessors affect the intuitions and the language use of learners differently.

Determining the importance of frequency and contextual diversity in the lexical organization of multiword expressions.

Corpus-based models of lexical strength have called into question the role of word frequency as an organizing principle of the lexicon, revealing that contextual and semantic diversity measures provide a closer fit to lexical behavior data (Adelman et al., 2006; Jones et al., 2012). Contextual diversity measures modify word frequency by ignoring word repetition in context, while semantic diversity measures consider the semantic consistency of contextual word occurrence. Recent research has shown that a better account of lexical organization data is provided by socially based measures of semantic diversity, which encode the communication patterns of individuals across discourses (Johns, 2021b). While most research on contextual diversity has focused on single words, recent corpus-based and experimental evidence suggests that an integral part of language use involves recurrent and more structurally complex units, such as multiword phrases and idioms. The aim of the present work was to determine if contextual and semantic diversity drive lexical organization at the level of multiword units (here, operationalized as idiomatic expressions), in addition to single words. To this end, we analyzed normative ratings of familiarity for 210 English idioms (Libben & Titone, 2008) using a set of contextual, semantic, and socially based diversity measures that were computed from a 55-billion word corpus of Reddit comments. The results confirm the superiority of diversity measures over frequency for multiword expressions, suggesting that multiword units, such as idiomatic phrases, show similar lexical organization dynamics as single words. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

A corpus-based versus experimental examination of word- and character-frequency effects in Chinese reading: Theoretical implications for models of reading.

Chinese words consist of a variable number of characters that are normally written in continuous lines, without the blank spaces that are used to separate words in most alphabetic writing systems. These conventions raise questions about the relative roles of character versus whole-word processing in word identification, and how words are segmented from strings of characters for the purpose of their identification and saccade targeting. The present article attempts to address these questions by reporting an eye-movement experiment in which 60 participants read a corpus of sentences containing two-character target words that varied in terms of their overall frequency and the frequency of their initial characters. We examine participants' eye movements using both corpus-based statistical models and more standard analyses of our target words. In addition to documenting how key lexical variables influence eye movements and highlighting a few discrepancies between the results obtained using our two statistical approaches, our experiment shows that high-frequency initial characters can actually slow word identification. We discuss the theoretical significance of this finding and others for current models of Chinese reading, and then describe a new computational model of eye-movement control during the reading of Chinese. Finally, we report simulations showing that this model can account for our findings. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

How Document Sampling and Vocabulary Pruning Affect the Results of Topic Models

Abstract Topic modeling enables researchers to explore large document corpora. Large corpora, however, can be extremely costly to model in terms of time and computing resources. In order to circumvent this problem, two techniques have been suggested: (1) to model random document samples, and (2) to prune the vocabulary of the corpus. Although frequently applied, there has been no systematic inquiry into how the application of these techniques affects the respective models. Using three empirical corpora with different characteristics (news articles, websites, and Tweets), we systematically investigated how different sample sizes and pruning affect the resulting topic models in comparison to models of the full corpora. Our inquiry provides evidence that both techniques are viable tools that will likely not impair the resulting model. Sample-based topic models closely resemble corpus-based models if the sample size is large enough (> 10,000 documents). Moreover, extensive pruning does not compromise the quality of the resultant topics.

Corpus-Based Paraphrase Detection Experiments and Review

Paraphrase detection is important for a number of applications, including plagiarism detection, authorship attribution, question answering, text summarization, text mining in general, etc. In this paper, we give a performance overview of various types of corpus-based models, especially deep learning (DL) models, with the task of paraphrase detection. We report the results of eight models (LSI, TF-IDF, Word2Vec, Doc2Vec, GloVe, FastText, ELMO, and USE) evaluated on three different public available corpora: Microsoft Research Paraphrase Corpus, Clough and Stevenson and Webis Crowd Paraphrase Corpus 2011. Through a great number of experiments, we decided on the most appropriate approaches for text pre-processing: hyper-parameters, sub-model selection—where they exist (e.g., Skipgram vs. CBOW), distance measures, and semantic similarity/paraphrase detection threshold. Our findings and those of other researchers who have used deep learning models show that DL models are very competitive with traditional state-of-the-art approaches and have potential that should be further developed.

FoodBase corpus: a new resource of annotated food entities.

The existence of annotated text corpora is essential for the development of public health services and tools based on natural language processing (NLP) and text mining. Recently organized biomedical NLP shared tasks have provided annotated corpora related to different biomedical entities such as genes, phenotypes, drugs, diseases and chemical entities. These are needed to develop named-entity recognition (NER) models that are used for extracting entities from text and finding their relations. However, to the best of our knowledge, there are limited annotated corpora that provide information about food entities despite food and dietary management being an essential public health issue. Hence, we developed a new annotated corpus of food entities, named FoodBase. It was constructed using recipes extracted from Allrecipes, which is currently the largest food-focused social network. The recipes were selected from five categories: ‘Appetizers and Snacks’, ‘Breakfast and Lunch’, ‘Dessert’, ‘Dinner’ and ‘Drinks’. Semantic tags used for annotating food entities were selected from the Hansard corpus. To extract and annotate food entities, we applied a rule-based food NER method called FoodIE. Since FoodIE provides a weakly annotated corpus, by manually evaluating the obtained results on 1000 recipes, we created a gold standard of FoodBase. It consists of 12 844 food entity annotations describing 2105 unique food entities. Additionally, we provided a weakly annotated corpus on an additional 21 790 recipes. It consists of 274 053 food entity annotations, 13 079 of which are unique. The FoodBase corpus is necessary for developing corpus-based NER models for food science, as a new benchmark dataset for machine learning tasks such as multi-class classification, multi-label classification and hierarchical multi-label classification. FoodBase can be used for detecting semantic differences/similarities between food concepts, and after all we believe that it will open a new path for learning food embedding space that can be used in predictive studies.

A Lexical Resource-Constrained Topic Model for Word Relatedness

Word relatedness computation is an important supporting technology for many tasks in natural language processing. Traditionally, there have been two distinct strategies for word relatedness measurement: one utilizes corpus-based models, whereas the other leverages external lexical resources. However, either solution has its strengths and weaknesses. In this paper, we propose a lexical resource-constrained topic model to integrate the two complementary strategies effectively. Our model is an extension of probabilistic latent semantic analysis, which automatically learns word-level distributed representations forward relatedness measurement. Furthermore, we introduce generalized expectation maximization (GEM) algorithm for statistical estimation. The proposed model not merely inherit the advantage of conventional topic models in dimension reduction, but it also refines parameter estimation by using word pairs that are known to be related. The experimental results in different languages demonstrate the effectiveness of our model in topic extraction and word relatedness measurement.

CLASENTI

Arabic text sentiment analysis suffers from low accuracy due to Arabic-specific challenges (e.g., limited resources, morphological complexity, and dialects) and general linguistic issues (e.g., fuzziness, implicit sentiment, sarcasm, and spam). The limited resources problem requires efforts to build new and improved Arabic corpora and lexica. We propose a class-specific sentiment analysis (CLASENTI) framework. The framework includes a new annotation approach to build multi-faceted Arabic corpus and lexicon allowing for simultaneous annotation of different facets, including domains, dialects, linguistic issues, and polarity strengths. Each of these facets has multiple classes (e.g., the nine classes representing dialects found in the Arab world). The new corpus and lexicon annotations facilitate the development of new class-specific classification models and polarity strength calculation. For the new sentiment classification models, we propose a hybrid model combining corpus-based and lexicon-based models. The corpus-based model has two interrelated phases to build; (1) full-corpus classification models for all facets; and (2) class-specific models trained on filtered subsets of the corpus according to the performances of the full-corpus models. To calculate polarity strengths, the lexicon-based model filters the annotated lexicon based on the specific classes of the domain and dialect. As a case study, we collect and annotate 15274 reviews from various sources, including surveys, Facebook comments, and Twitter posts, pertaining to governmental services. In addition, we develop a new web-based application to apply the proposed framework on the case study. CLASENTI framework reaches up to 95% accuracy and 93% F1-Score surpassing the best-known sentiment classifiers implemented in Scikit-learn library that achieve 82% accuracy and 81% F1-Score for Arabic when tested on the same dataset.

Embodied sound design

Abstract Embodied sound design is a process of sound creation that involves the designer’s vocal apparatus and gestures. The possibilities of vocal sketching were investigated by means of an art installation. An artist–designer interpreted several vocal self-portraits and rendered the corresponding synthetic sketches by using physics-based and concatenative sound synthesis. Both synthesis techniques afforded a broad range of artificial sound objects, from concrete to abstract, all derived from natural vocalisations. The vocal-to-synthetic transformation process was then automated in SEeD, a tool allowing to set and play interactively with physics- or corpus-based sound models. The voice-driven process and tool, developed and evaluated through design exercises, show how an embodied sound sketching system can work in supporting the externalisation of sonic concepts.

The language of smell: Connecting linguistic and psychophysical properties of odor descriptors

The olfactory sense is a particularly challenging domain for cognitive science investigations of perception, memory, and language. Although many studies show that odors often are difficult to describe verbally, little is known about the associations between olfactory percepts and the words that describe them. Quantitative models of how odor experiences are described in natural language are therefore needed to understand how odors are perceived and communicated. In this study, we develop a computational method to characterize the olfaction-related semantic content of words in a large text corpus of internet sites in English. We introduce two new metrics: olfactory association index (OAI, how strongly a word is associated with olfaction) and olfactory specificity index (OSI, how specific a word is in its description of odors). We validate the OAI and OSI metrics using psychophysical datasets by showing that terms with high OAI have high ratings of perceived olfactory association and are used to describe highly familiar odors. In contrast, terms with high OSI have high inter-individual consistency in how they are applied to odors. Finally, we analyze Dravnieks’s (1985) dataset of odor ratings in terms of OAI and OSI. This analysis reveals that terms that are used broadly (applied often but with moderate ratings) tend to be olfaction-unrelated and abstract (e.g., “heavy” or “light”; low OAI and low OSI) while descriptors that are used selectively (applied seldom but with high ratings) tend to be olfaction-related (e.g., “vanilla” or “licorice”; high OAI). Thus, OAI and OSI provide behaviorally meaningful information about olfactory language. These statistical tools are useful for future studies of olfactory perception and cognition, and might help integrate research on odor perception, neuroimaging, and corpus-based linguistic models of semantic organization.

Pitting corpus-based classification models against each other: a case study for predicting constructional choice in written Estonian

AbstractIn the context of constructional alternatives, we may assume that speakers’ choice between alternative forms is influenced by a multitude of factors. At the moment, multivariate statistical classification modelling seems to be the best tool available to capture this knowledge quantitatively. There is a vast array of techniques available. In this paper, two distinct modelling techniques are applied – logistic regression and naïve discriminative learning – to predict the choice between two constructional alternatives in written Estonian. One of the central questions in statistical modelling concerns the evaluation of model fit. It is proposed that for linguistic analysis, the performance of alternative corpus-based models can be evaluated by, first, pitting them against each other and second, pitting them against experimental data. Previous work on modelling constructional and lexical choice has focused on one of the two aspects. The present paper takes this line of analysis further by combining the two approaches.

Corpus-Based Semantic Models of the Noun Phrases Containing Words with ‘Person’ Marker

Abstract The mechanism underlying constructing of lexically correct sequences of words is an object of attention both in theoretical and applied fields of linguistics. This paper reveals some aspects of modelling the patterns of semantic valence in noun phrases of NN (Noun+Noun) structure, one or both components of which contain the ‘person’ semantic tag. The research is based on the Corpus of Ukrainian and performed with the help of automatic language processing.

Quantitative computational syntax: some initial results

In the computational study of human intelligence, the language sciences are in the unique position of resting both on sophisticated theories and representations and on large amounts of observational data available for many languages. In this paper, we discuss some recent results, where large-scale, data-intensive computational modelling techniques are used to address fundamental linguistic questions on the quantitative properties of abstract grammatical representations. Specifically, we present a programme of research exemplified in three case studies to identify the causes of frequency differentials. In the area of word order, we discuss work that investigates whether typological and corpus frequencies are systematically correlated to abstract syntactic structures and to higher-level structural principles of minimisation and efficiency. In the area of verb meaning, corpus-based computational models are discussed that investigate how frequencies are correlated to well-known lexical effects in causative alternations and morphological marking. The large corpus-based, cross-linguistic component of the work and the abstract grammatical hypotheses on word order and verb meaning provide new empirical and computational evidence to the important debate on language variation, its extent and its limits and illustrate how to bring corpus-based computational methodology to bear on theoretical syntactic issues. In so doing, we help reduce the current gap between theoretical and computational linguistics.

Machine Meets Man: Evaluating the Psychological Reality of Corpus-based Probabilistic Models

AbstractLinguistic convention typically allows speakers several options. Evidence is accumulating that the various options are preferred in different contexts, yet the criteria governing the selection of the appropriate form are often far from obvious. Most researchers who attempt to discover the factors determining a preference rely on the linguistic analysis and statistical modeling of data extracted from large corpora. In this paper, we address the question of how to evaluate such models and explicitly compare the performance of a statistical model derived from a corpus with that of native speakers in selecting one of six Russian TRY verbs. Building on earlier work we trained a polytomous logistic regression model to predict verb choice given the sentential context. We compare the predictions the model makes for 60 unseen sentences to the choices adult native speakers make in those same sentences. We then look in more detail at the interplay of the contextual properties and model computationally how individual differences in assessing the importance of contextual properties may impact the linguistic knowledge of native speakers. Finally, we compare the probability the model assigns to encountering each of the six verbs in the 60 test sentences to the acceptability ratings the adult native speakers give to those sentences. We discuss the implications of our findings for both usage-based theory and empirical linguistic methodology.

A novel family of IC-based similarity measures with a detailed experimental survey on WordNet

This paper introduces a novel family of ontology-based similarity measures based on the Information Content (IC) theory, a detailed state of the art, a large experimental survey into ontology-based similarity measures on WordNet, and a new comparison between intrinsic and corpus-based IC models. Our experiments are based on our implementation of a large set of similarity measures, intrinsic and corpus-based IC models, which are evaluated on two known datasets and three different WordNet versions. The new measures are called weighted Jiang–Conrath distance (wJ&Cdist) and similarity (wJ&Csim), cosine-normalized Jiang–Conrath similarity (cosJ&Csim) and cosine-normalized weighted Jiang–Conrath similarity (coswJ&Csim). Two of our similarity measures outperform the state-of-the-art measures on the RG65 dataset, and one of them obtains the third overall score on all the datasets and evaluated WordNet versions. The cosine-normalized similarity measures are a non-linear normalization of the classic Jiang–Conrath (J&C) distance and the new wJ&C distance. On the other hand, the wJ&C distance is a generalization of the classic J&C distance which is based on the length of the shortest path between concepts within an IC-based weighted graph. Our measures are based on two not previously considered notions: (1) a generalization of the classic J&C distance to any type of taxonomy, based on an IC-based weighted graph derived from the conditional probabilities between child and parent concepts, and (2) a non-linear normalization function that converts the ontology-based semantic distances into similarity functions. Finally, the corpus-based IC models based on the Resnik method obtain rivaling results as regards the state-of-the-art intrinsic IC models, when they are used with some unexplored WordNet-based frequency files. Therefore, this latter fact allows us to reconsider some previous conclusions about the outperformance of the intrinsic IC models over the corpus-based ones.

A new family of information content models with an experimental survey on WordNet

This paper introduces a new family of intrinsic and corpus-based Information Content (IC) models for ontology-based similarity measures based on the IC theory, a detailed state of the art, an experimental survey of IC models and IC-based similarity measures on WordNet, and a comparison between intrinsic and corpus-based IC models. The family of IC models is made up of five intrinsic IC models, called CondProbHypo, CondProbUniform, CondProbLeaves, CondProbLogistic, and CondProbCosine, and one corpus-based IC model called CondProbCorpus which completes the family. The proposed IC models rely on two previously unconsidered notions: (1) the preservation of the probabilistic structure of the taxonomy associated to the conditional probabilities between child and parent concepts, and (2) the explicit consideration of a cognitive similarity notion in the definition of the IC model. The family of IC models defines a new method for the proposal of new intrinsic IC models based on the exploration of other alternatives for the intrinsic estimation of the conditional probabilities between child and parent concepts. Our work is inspired by an unexplored relationship between the Jiang–Conrath distance and a shortest path on an IC-based weighted graph, derived from the conditional probabilities between concepts, as well as certain cognitive evidence about the perception distance between concepts. The new IC models obtain results comparable to the state of the art and satisfy a set of well-founded structure axioms. In addition, we prove that most of intrinsic IC models and IC-based similarity measures do not show a significant statistical difference as regards a baseline corpus-based IC model and the Jiang–Conrath similarity, with the exception of the overall outperformance shown by the Sánchez et al. (2012) IC model and the cosJ&C similarity measure, which has recently been introduced by the authors.

Integrating words that refer to typical sequences of events.

The organisation and processing of event concepts in semantic memory is an important issue in language processing and memory research. The present study tested whether pairs of words denoting events that can plausibly occur in sequence (marinate-grill) generate expectancies for a target that denotes a subsequently occurring event (chew). In Experiment 1, two events that tend to occur sequentially primed the third. In Experiment 2, the individual primes (i.e., marinate and grill separately) did not prime their related event targets. Experiments 1 and 2 used a lexical-decision task on the target. Therefore, information from both primes must be integrated to sufficiently activate knowledge of the subsequently occurring target. This is the first study to demonstrate priming among words denoting sequentially occurring events. In Experiment 3, a relatedness decision task, processing of these event triplets was facilitated when the first two event words were presented in a temporally correct order compared with when their order was reversed. These findings are not predicted by spreading activation theory and cannot be simulated by corpus-based models that do not include order-sensitive measures. We interpret the results as evidence for the role of situation models and the use of world knowledge during online language comprehension, even in the absence of sentential contexts.

Murder in the Arboretum: Comparing Character Models to Personality Models

Interactive Narrative often involves dialogue with virtual dramatic characters. In this paper we compare two kinds of models of character style: one based on models derived from the Big Five theory personality, and the other derived from a corpus-based method applied to characters and films from the IMSDb archive. We apply these models to character utterances for a pilot narrative-based outdoor augmented reality game called Murder in the Arboretum. We use an objective quantitative metric to estimate the quality of a character model, with the aim of predicting model quality without perceptual experiments. We show that corpus-based character models derived from individual characters are often more detailed and specific than personality based models, but that there is a strong correlation between personality judgments of original character dialogue and personality judgments of utterances generated for Murder in the Arboretum that use the derived character models.

Las tecnologías del habla en las lenguas románicas ibéricas

Abstract This paper reviews the development of speech technologies for Iberian Romance languages (Catalan, Galician, Spanish and Portuguese) with a special emphasis on the phonetic knowledge involved in the three core areas in the field: text-to-speech synthesis, automatic speech recognition and spoken language systems. Several important technological advances have taken place during the last decades and there is a substantial base of active researchers in the areas of engineering and computer science. However, the trend towards the use of automatic learning techniques and corpus-based models has seriously reduced the role of linguistics, favoring a reduction in time in the development of applications over basic research into the nature of the speech code.