Natural Language Words Research Articles

Research on the Application of NLP Artificial Intelligence Tools in University Natural Language Processing

Natural language formal analysis theory has created brilliant achievements on the basis of previous studies. However, with the development of computing power and the advent of the deep learning boom, some people believe that the rule-based rationalist method is outdated, and deep learning that relies on massive data can truly realize artificial intelligence. When traditional natural language is directly transplanted to text language, the short content of natural language will cause data sparseness and result in deviation of calculation results. This paper proposes a new natural language similarity measurement method by using NLP artificial intelligence tools. This method first preprocesses short texts, then builds a complex network model for natural language, calculates the complex network feature values of natural language words, and then uses NLP artificial intelligence tools to calculate the semantic similarity between natural language words, and then combines natural language Semantic similarity is defined to calculate the similarity between natural languages.

Quantum semantics of text perception

The paper presents quantum model of subjective text perception based on binary cognitive distinctions corresponding to words of natural language. The result of perception is quantum cognitive state represented by vector in the qubit Hilbert space. Complex-valued structure of the quantum state space extends the standard vector-based approach to semantics, allowing to account for subjective dimension of human perception in which the result is constrained, but not fully predetermined by input information. In the case of two distinctions, the perception model generates a two-qubit state, entanglement of which quantifies semantic connection between the corresponding words. This two-distinction perception case is realized in the algorithm for detection and measurement of semantic connectivity between pairs of words. The algorithm is experimentally tested with positive results. The developed approach to cognitive modeling unifies neurophysiological, linguistic, and psychological descriptions in a mathematical and conceptual structure of quantum theory, extending horizons of machine intelligence.

Are current elicitation techniques for barriers and enablers confounded with motivation? How natural language may hinder theory-guided research.

The objective of this study was to compare standard elicitation techniques for barriers and enablers for physical activity and sleep behaviours, to an alternative approach whereby participants were told to only consider the literal meanings of the words prevent/enable. Randomized controlled design. College students were randomized to either a standard methods group (n=177) (what prevents you from doing behaviour X) or a vignette group (n=176) to encourage them to think of the literal meaning of the words prevent/enable. Responses were then codified by two blinded researchers. Students reported significantly different types of control beliefs between groups. Those in the standard group reported significantly more overall beliefs (p's<.05, except sleep/enable), suggesting poorer discrimination in interpreting what was meant by 'prevent' and 'enable'. This study demonstrates when self-efficacy-related control beliefs are elicited, natural language words such as 'prevent' and 'enable' have the potential to confuse people about the intent of the question.

A expressão de singularidades em gêneros narrativos análise da classificação de fanfictions

The objective was to understand how fanfiction narrative genres are classified in repositories, describing the users perceptions and motivations to accomplishthe indexing. The research is descriptive and exploratory, with the Intensive Direct Observation method in virtual groups of fans who write and consume fanfictions;the data supported a qualitative analysis of the fanfictions narrative genres used in theirindexingin digital repositories. The analysis demonstrated that many of narrative genres used in the repositories express concern with the representation of gender, sexual and physical aspects. Furthermore, natural language words, neologisms and other foreignwords are re-semantized to represent narrative genres. It was concluded that narrative genres are used to classify the stories, to express singularities andtogive representation to demands of marginalized groups of the publishing market.

Supplemental Material for Modeling Word and Morpheme Order in Natural Language as an Efficient Trade-Off of Memory and Surprisal

Supplemental Material for Modeling Word and Morpheme Order in Natural Language as an Efficient Trade-Off of Memory and Surprisal

Semantic Detection of Targeted Attacks Using DOC2VEC Embedding

The targeted attack is one of the social engineering attacks. The detection of this type of attack is considered a challenge as it depends on semantic extraction of the intent of the attacker. However, previous research has primarily relies on the Natural Language Processing or Word Embedding techniques that lack the context of the attacker's text message. Based on Sentence Embedding and machine learning approaches, this paper introduces a model for semantic detection of targeted attacks. This model has the advantage of encoding relevant information, which helps to improve the performance of the multi-class classification process. Messages will be categorized based on the type of security rule that the attacker has violated. The suggested model was tested using a dialogue dataset taken from phone calls, which was manually categorized into four categories. The text is pre-processed using natural language processing techniques, and the semantic features are extracted as Sentence Embedding vectors that are augmented with security policy sentences. Machine Learning algorithms are applied to classify text messages. The experimental results show that sentence embeddings with doc2vec achieved high prediction accuracy 96.8%. So, it outperformed the method applied to the same dialog dataset.

Ambiguity and context learning in signalling games

Abstract Lexical ambiguity is present in many natural languages, but ambiguous words and phrases do not seem to be advantageous. Therefore, the presence of ambiguous words in natural language warrants explanation. We justify the existence of ambiguity from the perspective of context dependence. The main contribution of the paper is that we constructed a context learning process such that each interlocutor can infer their opponent’s private belief from the conversation. A sufficient condition for successful learning is provided. Furthermore, for cases in which learning fails, we investigate how the interlocutors choose among degrees of ambiguous expressions through an adaptive learning process. Lastly, we apply our model in the lattice network, demonstrating that structural evolution favours ambiguity as well.

A Computational Framework Based on Ensemble Deep Neural Networks for Essential Genes Identification.

Essential genes contain key information of genomes that could be the key to a comprehensive understanding of life and evolution. Because of their importance, studies of essential genes have been considered a crucial problem in computational biology. Computational methods for identifying essential genes have become increasingly popular to reduce the cost and time-consumption of traditional experiments. A few models have addressed this problem, but performance is still not satisfactory because of high dimensional features and the use of traditional machine learning algorithms. Thus, there is a need to create a novel model to improve the predictive performance of this problem from DNA sequence features. This study took advantage of a natural language processing (NLP) model in learning biological sequences by treating them as natural language words. To learn the NLP features, a supervised learning model was consequentially employed by an ensemble deep neural network. Our proposed method could identify essential genes with sensitivity, specificity, accuracy, Matthews correlation coefficient (MCC), and area under the receiver operating characteristic curve (AUC) values of 60.2%, 84.6%, 76.3%, 0.449, and 0.814, respectively. The overall performance outperformed the single models without ensemble, as well as the state-of-the-art predictors on the same benchmark dataset. This indicated the effectiveness of the proposed method in determining essential genes, in particular, and other sequencing problems, in general.

Miller's monkey updated: Communicative efficiency and the statistics of words in natural language

Is language designed for communicative and functional efficiency? G. K. Zipf famously argued that shorter words are more frequent because they are easier to use, thereby resulting in the statistical law that bears his name. Yet, G. A. Miller showed that even a monkey randomly typing at a keyboard, and intermittently striking the space bar, would generate “words” with similar statistical properties. Recent quantitative analyses of human language lexicons (Piantadosi et al., 2012) have revived Zipf's functionalist hypothesis. Ambiguous words tend to be short, frequent, and easy to articulate in language production. Such statistical findings are commonly interpreted as evidence for pressure for efficiency, as the context of language use often provides cues to overcome lexical ambiguity. In this study, we update Miller's monkey thought experiment to incorporate empirically motivated phonological and semantic constraints on the creation of words. We claim that the appearance of communicative efficiency is a spandrel (Gould & Lewontin, 1979), as lexicons formed without the context of language use or reference to communication or efficiency exhibit comparable statistical properties. Furthermore, the updated monkey model provides a good fit for the growth trajectory of English as recorded in the Oxford English Dictionary. Focusing on the history of English words since 1900, we show that lexicons resulting from the monkey model provide a better embodiment of communicative efficiency than the actual lexicon of English. We conclude by arguing for the need to go beyond correlational statistics and to seek direct evidence for the mechanisms that underlie principles of language design.

Bridging Semantic Gaps between Natural Languages and APIs with Word Embedding

Developers increasingly rely on text matching tools to analyze the relation between natural language words and APIs. However, semantic gaps, namely textual mismatches between words and APIs, negatively affect these tools. Previous studies have transformed words or APIs into low-dimensional vectors for matching; however, inaccurate results were obtained due to the failure of modeling words and APIs simultaneously. To resolve this problem, two main challenges are to be addressed: the acquisition of massive words and APIs for mining and the alignment of words and APIs for modeling. Therefore, this study proposes Word2API to effectively estimate relatedness of words and APIs. Word2API collects millions of commonly used words and APIs from code repositories to address the acquisition challenge. Then, a shuffling strategy is used to transform related words and APIs into tuples to address the alignment challenge. Using these tuples, Word2API models words and APIs simultaneously. Word2API outperforms baselines by 10-49.6 percent of relatedness estimation in terms of precision and NDCG. Word2API is also effective on solving typical software tasks, e.g., query expansion and API documents linking. A simple system with Word2API-expanded queries recommends up to 21.4 percent more related APIs for developers. Meanwhile, Word2API improves comparison algorithms by 7.9-17.4 percent in linking questions in Question&Answer communities to API documents.

Text analysis of DNS queries for data exfiltration protection of computer networks

The paper proposes effective method of computer network protection from data exfiltration by the system of domain names. Data exfiltration by Domain Name System (DNS) is an approach to conceal the transfer of confidential data to remote adversary using data encapsulation into the requesting domain name. The DNS requests that transfer stolen information from a host infected by malicious software to an external host controlled by a malefactor are considered. The paper proposes a method of detecting such DNS requests based on text classification of domain names by convolutional neural network. The efficiency of the method is based on assumption that domain names exploited for data exfiltration differ from domain names formed from words of natural language. To classify the requests in convolutional neural network the use of character embedding for representing the string of a domain name is proposed. Quality evaluation of the trained neural network used for recognition of data exfiltration through domain name system using ROC-analysis is performed.The paper presents the software architecture used for deployment of trained neural network into existing infrastructure of the domain name system targeting practical computer networks protection from data exfiltration. The architecture implies creation of response policy zones for blocking of individual requests, classified as malicious.

Sentiment Analysis of Children and Youth Literature: Is There a Pollyanna Effect?

If the words of natural human language possess a universal positivity bias, as assumed by Boucher and Osgood’s (1969) famous Pollyanna hypothesis and computationally confirmed for large text corpora in several languages (Dodds et al., 2015), then children and youth literature (CYL) should also show a Pollyanna effect. Here we tested this prediction applying an unsupervised vector space model-based sentiment analysis tool called SentiArt (Jacobs, 2019) to two CYL corpora, one in English (372 books) and one in German (500 books). Pitching our analysis at the sentence level, and assessing semantic as well as lexico-grammatical information, both corpora show the Pollyanna effect and thus add further evidence to the universality hypothesis. The results of our multivariate sentiment analyses provide interesting testable predictions for future scientific studies of literature.

DPWord2Vec: Better Representation of Design Patterns in Semantics

With the plain text descriptions of design patterns, developers could better learn and understand the definitions and usage scenarios of design patterns. To facilitate the automatic usage of these descriptions, e.g., recommending design patterns by free-text queries, design patterns and natural languages should be adequately associated. Existing studies usually use texts in design pattern books as the representations of design patterns to calculate similarities with the queries. However, this way is problematic. Lots of information of design patterns may be absent from design pattern books and many words would be out of vocabulary due to the content limitation of these books. To overcome these issues, a more comprehensive method should be constructed to estimate the relatedness between design patterns and natural language words. Motivated by Word2Vec, in this study, we propose DPWord2Vec that embeds design patterns and natural language words into vectors simultaneously. We first build a corpus containing more than 400 thousand documents extracted from design pattern books, Wikipedia, and Stack Overflow. Next, we redefine the concept of context window to associate design patterns with words. Then, the design pattern and word vector representations are learnt by leveraging an advanced word embedding method. The learnt design pattern and word vectors can be universally used in textual description based design pattern tasks. An evaluation shows that DPWord2Vec outperforms the baseline algorithms by 17.1%-96.5% in measuring the similarities between design patterns and words in terms of Spearman's rank correlation coefficient. Moreover, we adopt DPWord2Vec on two typical design pattern tasks. In the design pattern tag recommendation task, the DPWord2Vec based method outperforms two state-of-the-art algorithms by 6.6% and 32.7% respectively when considering Recall@10. In the design pattern selection task, DPWord2Vec improves the existing methods by 6.5%-70.7% in terms of MRR.

Normalization of Kazakh language words

Subject of Research. Models and existing algorithms for normalization of natural language words are considered. The paper describes algorithms for automatic selection of the basic principles for a number of natural languages and possible ways of the normal word form synthesis for the Kazakh language. The research is aimed at creation of a complete classification for the Kazakh language ending system and development of a normalization algorithm for words based on the proposed classification approach for endings and suffixes. Method. Word formation analysis by applying endings for all Kazakh language parts of speech was carried out; a classification of endings and suffixes was presented. The paper discusses all kinds of placement options for endings and suffixes. The total number of various suffixes is 26 526 units and the endings is 3 565 units. All considered types are lexically and semantically valid, but some of them are not applicable. Only those, that are most commonly used, are added to the affix base. The order, that the affixes are added to the base, is presented using sets. Thus, the base is correctly selected. The study does not examine word-forming suffixes, as they change the word stem and contextual interpretation. Basically, word-forming suffixes are added to nouns. Main Results. A complete classification system for endings and suffixes of the Kazakh language has been developed. Deterministic finite automata for various parts of speech are created using all possible options, adding suffixes and endings, taking into account the morphological and lexical features of the Kazakh language grammar. A lexicon-free stemming algorithm is developed using the proposed classification system for endings of the Kazakh language. A normalization system has been implemented, proving the operability of the developed algorithm without a dictionary. The algorithm implementation was tested on the Kazakh language corpus. Punctuation and stop words were initially removed from the specified corpus. Practical Relevance. The results of the work can find application in the text analysis and normalization (lemmatization), as well as in information retrieval systems, in machine translation from the Kazakh language, and other applied problems.

Using word embeddings in abstracts to accelerate metallocene catalysis polymerization research

Natural language processing (NLP) and word embeddings trained neural networks were investigated as a more efficient method to extract useful information on catalytic polymerizations. Thousands of abstracts on metallocene-catalyzed polymerizations were accessed through journal Application Programming Interfaces. These abstracts were then used to create a group of related models to produce word embeddings, making use of the word2vec algorithm. This algorithm turns vocabulary into high dimensional vectors using unsupervised training. These vectors can then be used to show relationships between chemicals, suggest catalysts and activators combinations, understand acronyms, and categorize chemical compounds based on their reagent classification. We hypothesize that one can determine which areas of metallocene catalysis are understudied by comparing the predicted abstract and catalysts combinations with those found in existing abstracts, thereby guiding research to major breakthroughs as scientific literature continues to grow.

The use of number words in natural language obeys Weber's law.

It has been suggested that the origins of number words can be traced back to an evolutionarily ancient approximate number system, which represents quantities on a compressed scale and complies with Weber's law. Here, we use a data-driven computational model, which learns to predict 1 event (a word in a text corpus) from associated events, to characterize verbal behavior relative to number words in natural language, without appeal to perception. We show that the way humans use number words in spontaneous language reliably depends on numerical ratio-a clear signature of Weber's law-thus, perfectly mirroring the human and nonhuman psychophysical performance in comparative judgments of numbers. Most notably, the adherence to Weber's law is robustly replicated in a wide range of different languages. Together, these findings suggest that the everyday use of number words in language rests upon a preverbal approximate number system, which would affect the handling of numerical information not only at the input level but also at the level of verbal production. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Contextual word representations

Advances in how programs treat natural language words have a big impact in AI.

Relations between semiotic resources in mathematics tasks: a source of students' difficulties

ABSTRACT Tackling mathematics tasks often involves reading and interpreting different semiotic resources such as natural language (words), mathematical notation, and images. This study aims to enhance knowledge of how meaning relations, in the form of “cohesive ties” between and within different semiotic resources, are related to how difficult it is to read and solve mathematics tasks. Using 133 tasks from PISA mathematics tests and 354 tasks from the annual Swedish National Test in mathematics, statistical analyses found relations between the presence of different types of cohesive ties and measures of how difficult the tasks were to read and solve. The results reveal a difficulty aspect related to the extent to which a task has cohesive ties, of any kind, and that non-mathematics-specific reading demand is not part of this difficulty aspect. An implication is that mathematics teaching should also focus on the identification of cohesive relations in the text of tasks.

Natural Language Word Prediction Model Based on Multi-Window Convolution and Residual Network

In this paper, we propose MCNN-ReMGU model based on multi-window convolution and residual-connected minimal gated unit (MGU) network for the natural language word prediction. First, the convolution kernels with different sizes are used to extract the local feature information of different graininess between the word sequences. Then, the extracted features are fed to the residual-connected MGU network. Finally, the prediction results are output by the SoftMax layer. Through the residual-connection processing of MGU network in the model, not only the problems of vanishing gradient and network degradation are effectively solved, but also the long-term dependence between word sequences is effectively extracted to predict the next word accurately. Meanwhile, the introduction of the convolution kernel in a convolutional neural network (CNN) enables the feature information between word sequences to be extracted more fully. The experimental results on the Penn Treebank and WikiText-2 datasets show that the proposed method has certain advantages in the word prediction task.

Supervised Word Sense Disambiguation with Recurrent Neural Network Model

Disambiguating words is a branch of artificial intelligence that deals with natural language processing. The dissatisfaction of the motive of the word deals with the polysemy of the ambiguous word, processing a single word in natural language, having two or more meanings where the corresponding context discriminates the meaning. Humans are intelligent enough to derive the meaning of the word because they are a biological neural network. Computers can be trained in such a way that they should function similarly to biological neural networks. There are four different suggested approaches to clutter as the knowledge-dependent approach and the machine learning based models which are further classified as supervised, semi-supervised and unpublished learning models. The purpose of this research is to improve better communication between computers and humans. The discussed model used a supervised learning approach with recurrent neural networks.