Natural Language Words Research Articles

Communication increases category structure and alignment only when combined with cultural transmission

Abstract The semantic categories labeled by words in natural languages are used for communication with others, and learned by observing the productions of others who learned them in the same way. Do these processes of communication and cultural transmission affect the structure of category systems and their alignment across speakers? We examine novel category systems that emerge from communication, cultural transmission, and both processes combined. Communication alone leads to category systems that vary widely in their communicative effectiveness, and are no more structured or aligned than those created by individuals. When combined with cultural transmission, communication speeds up convergence on a learnable number of structured, aligned categories that are consistently communicatively effective. However, cultural transmission without communication eventually has similar results. Communication appears to be neither necessary nor sufficient for creating semantic category systems that are robustly effective for communication. Furthermore, category systems that emerge from cultural transmission are more aligned across speakers than the systems created by individuals, suggesting that cultural transmission allows individuals to coordinate their semantic systems more effectively than they can through shared perceptual biases alone.

Fuzzy rule based unsupervised sentiment analysis from social media posts

In this paper, we compute the sentiment of social media posts using a novel set of fuzzy rules involving multiple lexicons and datasets. The proposed fuzzy system integrates Natural Language Processing techniques and Word Sense Disambiguation using a novel unsupervised nine fuzzy rule based system to classify the post into: positive, negative or neutral sentiment class. We perform a comparative analysis of our method on nine public twitter datasets, three sentiment lexicons, four state-of-the-art approaches for unsupervised Sentiment Analysis and one state-of-the-art method for supervised machine learning. Traditionally, Sentiment Analysis of twitter data is performed using a single lexicon. Our results can give an insight to researchers to choose which lexicon is best for social media. The fusion of fuzzy logic with lexicons for sentiment classification provides a new paradigm in Sentiment Analysis. Our method can be adapted to any lexicon and any dataset (two-class or three-class sentiment). The experiments on benchmark datasets yield higher performance for our approach as compared to the state-of-the-art.

Are you really cursing? Neural processing of taboo words in native and foreign language

The use of socially opprobrious words (taboo words) is a cross-cultural phenomenon occurring between individuals from almost all social extractions. The neurocognitive correlates of using taboo words in the native language (L1) as compared to their use in a second (L2) language are largely unknown. We used fMRI to investigate the processing of taboo and non-taboo stimuli in monolinguals (Experiment 1) and highly proficient bilinguals (Experiment 2) engaged in lexical decision tasks. We report that for L1 socio-pragmatic knowledge is automatically conveyed and taboo words are processed with less effort than non-taboo words. For L2 the processing of taboo words is more effortful and engages additional structures (anterior cingulate cortex, insula) involved in social-norm representation and evaluation. Our results contribute to understand the interface between language and social-norm processing indicating that lexical processing is affected by socio-pragmatic knowledge, but only when the speaker has a contextual use of the language.

The Small Words of Film Spoken Language for Second Language Learning

Although being scripted, film dialogues are claimed to mirror natural spoken language. Exposing second language (L2) learners to instances of authentic language is reported to be stimulating and enjoyable. Amongst others, natural spoken discourse is hallmarked by discourse markers, which are small words, or prefabricated units, which constitute the bulk of native-like conversation. Helping students become acquainted with the small words of natural language can increase their perceived proficiency. In light of these argumentations, this paper is aimed at presenting a trial lesson with sixteen young adults who participated in a 2-hour class. During the class, they became acquainted with discourse markers, which were sourced from film dialogues. Students had firstly to infer their meanings and propose coherent translation candidates by recurring to their interlanguage. Then, they were prompted to search for translations in online language platforms. The paper findings highlight that the trial lesson was not only enjoyable and stimulating, but students felt that their L2 knowledge increased. Furthermore, they were stimulated by the inferring tasks and appreciated the word search. In some instances, students' inferences outperformed dictionary results and online suggestions.

Grammar of protein domain architectures

From an abstract, informational perspective, protein domains appear analogous to words in natural languages in which the rules of word association are dictated by linguistic rules, or grammar. Such rules exist for protein domains as well, because only a small fraction of all possible domain combinations is viable in evolution. We employ a popular linguistic technique, n-gram analysis, to probe the "proteome grammar"-that is, the rules of association of domains that generate various domain architectures of proteins. Comparison of the complexity measures of "protein languages" in major branches of life shows that the relative entropy difference (information gain) between the observed domain architectures and random domain combinations is highly conserved in evolution and is close to being a universal constant, at ∼1.2 bits. Substantial deviations from this constant are observed in only two major groups of organisms: a subset of Archaea that appears to be cells simplified to the limit, and animals that display extreme complexity. We also identify the n-grams that represent signatures of the major branches of cellular life. The results of this analysis bolster the analogy between genomes and natural language and show that a "quasi-universal grammar" underlies the evolution of domain architectures in all divisions of cellular life. The nearly universal value of information gain by the domain architectures could reflect the minimum complexity of signal processing that is required to maintain a functioning cell.

Multi-Scale Deformable CNN for Answer Selection

The answer selection task is one of the most important issues within the automatic question answering system, and it aims to automatically find accurate answers to questions. Traditional methods for this task use manually generated features based on tf-idf and n-gram models to represent texts, and then select the right answers according to the similarity between the representations of questions and the candidate answers. Nowadays, many question answering systems adopt deep neural networks such as convolutional neural network (CNN) to generate the text features automatically, and obtained better performance than traditional methods. CNN can extract consecutive n-gram features with fixed length by sliding fixed-length convolutional kernels over the whole word sequence. However, due to the complex semantic compositionality of the natural language, there are many phrases with variable lengths and be composed of non-consecutive words in natural language, such as these phrases whose constituents are separated by other words within the same sentences. But the traditional CNN is unable to extract the variable length n-gram features and non-consecutive n-gram features. In this paper, we propose a multi-scale deformable convolutional neural network to capture the non-consecutive n-gram features by adding offset to the convolutional kernel, and also propose to stack multiple deformable convolutional layers to mine multi-scale n-gram features by the means of generating longer n-gram in higher layer. Furthermore, we apply the proposed model into the task of answer selection. Experimental results on public dataset demonstrate the effectiveness of our proposed model in answer selection.

Which Variables Should I Log?

Developers usually depend on inserting logging statements into the source code to collect system runtime information. Such logged information is valuable for software maintenance. A logging statement usually prints one or more variables to record vital system status. However, due to the lack of rigorous logging guidance and the requirement of domain-specific knowledge, it is not easy for developers to make proper decisions about which variables to log. To address this need, in this work, we propose an approach to recommend logging variables for developers during development by learning from existing logging statements. Different from other prediction tasks in software engineering, this task has two challenges: 1) Dynamic labels – different logging statements have different sets of accessible variables, which means in this task, the set of possible labels of each sample is not the same. 2) Out-of-vocabulary words – identifiers’ names are not limited to natural language words and the test set usually contains a number of program tokens which are out of the vocabulary built from the training set and cannot be appropriately mapped to word embeddings. To deal with the first challenge, we convert this task into a representation learning problem instead of a multi-label classification problem. Given a code snippet which lacks a logging statement, our approach first leverages a neural network with an RNN (recurrent neural network) layer and a self-attention layer to learn the proper representation of each program token, and then predicts whether each token should be logged through a unified binary classifier based on the learned representation. To handle the second challenge, we propose a novel method to map program tokens into word embeddings by making use of the pre-trained word embeddings of natural language tokens. We evaluate our approach on 9 large and high-quality Java projects. Our evaluation results show that the average MAP of our approach is over 0.84, outperforming random guess and an information-retrieval-based method by large margins.

Lexical Access in the Processing of Word Boundary Ambiguity

Language ambiguity results from, among other things, the vagueness of the syntactic structure of phrases and whole sentences. Numerous types of syntactic ambiguity are associated with the placement of the phrase boundary. A special case of the segmentation problem is the phenomenon of word boundary ambiguities; in spoken natural language words coalesce, making it possible to interpret them in different ways (e.g., a name vs. an aim). The purpose of the study was to verify whether the two meanings of words with boundary ambiguities are activated, or whether it is a case of semantic context priming. The study was carried out using the cross-modality semantic priming paradigm. Sentences containing phrases with word boundary ambiguities were presented in an auditory manner to the participants. Immediately after, they performed a visual lexical decision task. Results indicate that both meanings of the ambiguity are automatically activated — independently of the semantic context. When discussing the results I refer to the autonomous and interactive models of parsing, and show other possible areas of research concerning word boundary ambiguities.

Multilingual Interface for C++

The multilingual interface for C++project aims to help students at the early stages to learn programming while bypassing the language barrier. Additionally, it can aid in the transfer of skills gained by the students to C++. The proposed interface utilizes the pre-processor commands of C++ and offers a simplified syntax and translate the keywords to natural language.The problem however is that keywords from some natural languages cannot be written in ASCII and there are multiple words for some keywords.The outcome of the experiment reveals that the presence of multiple natural language words to cater for a specific traditional programming language keyword causes confusion among novice programmers. The Syntax and keywords were formulated by a three phased workshop on school children and teachers.The multilingual interface for C++ was evaluated by a study on KG to 7thGrade children and novice programmers.The findings showed that natural language-based programming is not a complete solution for the difficulties faced by novice programming language learners. But, it could assist in elimination of certain syntax errors, while adding to the complexity of natural language constructs.

Diminished brain responses to second-language words are linked with native-language literacy skills in dyslexia

Dyslexia is characterized by poor reading skills, yet often also difficulties in second-language learning. The differences between native- and second-language speech processing and the establishment of new brain representations for spoken second language in dyslexia are not, however, well understood. We used recordings of the mismatch negativity component of event-related potential to determine possible differences between the activation of long-term memory representations for spoken native- and second-language word forms in Finnish-speaking 9–11-year-old children with or without dyslexia, studying English as their second language in school. In addition, we sought to investigate whether the bottleneck of dyslexic readers' second-language learning lies at the level of word representations or smaller units and whether the amplitude of mismatch negativity is correlated with native-language literacy and related skills. We found that the activation of brain representations for familiar second-language words, but not for second-language speech sounds or native-language words, was weaker in children with dyslexia than in typical readers. Source localization revealed that dyslexia was associated with weak activation of the right temporal cortex, which has been previously linked with word-form learning. Importantly, the amplitude of the mismatch negativity for familiar second-language words correlated with native-language literacy and rapid naming scores, suggesting a close link between second-language processing and these skills.

Visual exploration and comparison of word embeddings

Word embeddings are distributed representations for natural language words, and have been wildly used in many natural language processing tasks. The word embedding space contains local clusters with semantically similar words and meaningful directions, such as the analogy. However, there are different training algorithms and text corpora, which both have a different impact on the generated word embeddings. In this paper, we propose a visual analytics system to visually explore and compare word embeddings trained by different algorithms and corpora. The word embedding spaces are compared from three aspects, i.e., local clusters, semantic directions and diachronic changes, to understand the similarity and differences between word embeddings.

Low-Power Sparse Hyperdimensional Encoder for Language Recognition

Editor’s note: Online learning for data analysis, categorization and anomaly detection has become a key technique in a range of adaptive embedded applications. In this article the authors propose a very low power encoder design using a sparse, hyperdimensional representation of letters and words in natural language and they show that this representation and their design can be used with high energy efficiency for a language recognition task. — Axel Jantsch, Royal Institute of Technology Electronics, Computer and Software Systems

Developmental Constraints on Learning Artificial Grammars with Fixed, Flexible and Free Word Order.

Human learning, although highly flexible and efficient, is constrained in ways that facilitate or impede the acquisition of certain systems of information. Some such constraints, active during infancy and childhood, have been proposed to account for the apparent ease with which typically developing children acquire language. In a series of experiments, we investigated the role of developmental constraints on learning artificial grammars with a distinction between shorter and relatively frequent words (‘function words,’ F-words) and longer and less frequent words (‘content words,’ C-words). We constructed 4 finite-state grammars, in which the order of F-words, relative to C-words, was either fixed (F-words always occupied the same positions in a string), flexible (every F-word always followed a C-word), or free. We exposed adults (N = 84) and kindergarten children (N = 100) to strings from each of these artificial grammars, and we assessed their ability to recognize strings with the same structure, but a different vocabulary. Adults were better at recognizing strings when regularities were available (i.e., fixed and flexible order grammars), while children were better at recognizing strings from the grammars consistent with the attested distribution of function and content words in natural languages (i.e., flexible and free order grammars). These results provide evidence for a link between developmental constraints on learning and linguistic typology.

The Effect of Zipfian Frequency Variations on Category Formation in Adult Artificial Language Learning

ABSTRACTSuccessful language acquisition hinges on organizing individual words into grammatical categories and learning the relationships between them, but the method by which children accomplish this task has been debated in the literature. One proposal is that learners use the shared distributional contexts in which words appear as a cue to their underlying category structure. Indeed, recent research using artificial languages has demonstrated that learners can acquire grammatical categories from this type of distributional information. However, artificial languages are typically composed of a small number of equally frequent words, while words in natural languages vary widely in frequency, complicating the distributional information needed to determine categorization. In a series of three experiments we demonstrate that distributional learning is preserved in an artificial language composed of words that vary in frequency as they do in natural language, along a Zipfian distribution. Rather than depending on the absolute frequency of words and their contexts, the conditional probabilities that words will occur in certain contexts (given their base frequency) is a better basis for assigning words to categories; and this appears to be the type of statistic that human learners utilize.

A Study of the Interface of Word and Phrase in Natural Language -with a Focus on Japanese and Korean Noun Connecting System-

A Study of the Interface of Word and Phrase in Natural Language -with a Focus on Japanese and Korean Noun Connecting System-

Cross-lingual neighborhood effects in generalized lexical decision and natural reading.

The present study assessed intra- and cross-lingual neighborhood effects, using both a generalized lexical decision task and an analysis of a large-scale bilingual eye-tracking corpus (Cop, Dirix, Drieghe, & Duyck, 2016). Using new neighborhood density and frequency measures, the general lexical decision task yielded an inhibitory cross-lingual neighborhood density effect on reading times of second language words, replicating van Heuven, Dijkstra, and Grainger (1998). Reaction times for native language words were not influenced by neighborhood density or frequency but error rates showed cross-lingual neighborhood effects depending on target word frequency. The large-scale eye movement corpus confirmed effects of cross-lingual neighborhood on natural reading, even though participants were reading a novel in a unilingual context. Especially second language reading and to a lesser extent native language reading were influenced by lexical candidates from the nontarget language, although these effects in natural reading were largely facilitatory. These results offer strong and direct support for bilingual word recognition models that assume language-independent lexical access. (PsycINFO Database Record

Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation

We introduce a new class of models called multiresolution recurrent neural networks, which explicitly model natural language generation at multiple levels of abstraction. The models extend the sequence-to-sequence framework to generate two parallel stochastic processes: a sequence of high-level coarse tokens, and a sequence of natural language words (e.g. sentences). The coarse sequences follow a latent stochastic process with a factorial representation, which helps the models generalize to new examples. The coarse sequences can also incorporate task-specific knowledge, when available. In our experiments, the coarse sequences are extracted using automatic procedures, which are designed to capture compositional structure and semantics. These procedures enable training the multiresolution recurrent neural networks by maximizing the exact joint log-likelihood over both sequences. We apply the models to dialogue response generation in the technical support domain and compare them with several competing models. The multiresolution recurrent neural networks outperform competing models by a substantial margin, achieving state-of-the-art results according to both a human evaluation study and automatic evaluation metrics. Furthermore, experiments show the proposed models generate more fluent, relevant and goal-oriented responses.

A Comparative Study to Understanding about Poetics Based on Natural Language Processing

This paper tries to find out five poets’ (Thomas Hardy, Wilde, Browning, Yeats, and Tagore) differences and similarities through analyzing their works on nineteenth Century by using natural language understanding technology and word vector model. Firstly, we collect enough poems from these five poets, build five corpus respectively, and calculate their high-frequency words, by using Natural Language Processing method. Then, based on the word vector model, we calculate the word vectors of the five poets’ high-frequency words, and combine the word vectors of each poet into one vector. Finally, we analyze the similarity between the combined word vectors by using the hierarchical clustering method. The result shows that the poems of Hardy, Browning, and Wilde are similar; the poems of Tagore and Yeats are relatively close—but the gap between the two is relatively large. In addition, we evaluate the stability of our approach by altering the word vector dimension, and try to analyze the results of clustering in a literary (poetic) perspective. Yeats and Tagore possessed a kind of mysticism poetics thought, while Hardy, Browning, and Wilde have the elements of realism combined with tragedy and comedy. The results are similar comparing to those we get from the word vector model.

Contract Interpretation: Brief Overview from the Standpoint of International and Trans-National Instruments

The role of contract in modern social and economic order can hardly be overestimated. Contract constitutes the legal form of economic exchange and in the same time it is an instrument for self-regulation which is used by agents in the market. Therefore contract bears much resemblance to the state law. Just as the state law, contract provides some obligatory rules which must be followed. And just as the provisions of state law, the terms of a contract are put in words of natural language. Since natural language allows for much ambiguity and polysemy, the problem of finding out the exact meaning of contract terms constitutes an important issue of modern law of contracts.Contract interpretation is a traditional topic within the realm of contract law. That is why the issue is always in the spotlight of researchers’ attention. Amongst contemporary publications the works of Catherine Mitchell, Steve J Cornelius, Lord Justice Lewison and David Hughes address the issue.Paper objective. The main objective of the paper is to figure out the most modern tendencies relating to contract interpretation. This goal is achieved through the analysis of influential international and trans-national authorities such as Vienna Convention on Contracts for the International Sale of Goods (CISG), UNIDROIT Principles of International Commercial Contracts (PICC), Principles of European Contract Law (PECL), Draft Common Frame of Reference (DCFR), Common European Sales Law (CESL).First of all the paper provides a brief overview of two opposite approaches to the interpretation of contract, namely objective theory of interpretation and subjective theory of interpretation which are also known as reliance theory and theory of will. The author states pros and cons of each theory and then goes on to analyzing how those two theories are reconciled in PICC, PECL, DCFR, and CESL. Also the author pays attention to different factors which one has to take into account while interpreting the contract. In this context some brief comparative observations are made.The main trend in evolution of modern rules on contract interpretation is that the boundary dividing subjective and objective interpretation is being more and more erased. Those two approaches are now combined in one complex solution of the problem. This solution can be worded as a kind of rebuttable presumption according to which the contract must be construed from the standpoint of hypothetical reasonable persons put in the same factual situation as the parties, unless there are firm evidences that there was an actual mutual intention of the parties to attach the contract some different meaning (i.e. different from the meaning which can be inferred by using the first approach).

Using Uniform-Design GEP for Part-of-Speech Tagging

In natural language processing (NLP), a crucial subsystem in a wide range of applications is a part-of-speech (POS) tagger, which labels (or classifies) unannotated words of natural language with POS labels corresponding to categories such as noun, verb or adjective. This paper proposes a model of uniform-design genetic expression programming (UGEP) for POS tagging. UGEP is used to search for appropriate structures in function space of POS tagging problems. After the evolution of sequence of tags, GEP can find the best individual as solution. Experiments on Brown Corpus show that (1) in closed lexicon tests, UGEP model can get higher accuracy rate of 98.8% which is much better than genetic algorithm model, neural networks and hidden Markov model (HMM) model.; (2) in open lexicon tests, the proposed model can also achieve higher accuracy rate of 97.4% and a high accuracy rate on unknown words of 88.6%.