Semantic Information Of Words Research Articles

Joint extraction of wheat germplasm information entity relationship based on deep character and word fusion

The verified text data of wheat varieties is an important component of wheat germplasm information. To automatically obtain a structured description of the phenotypic and genetic characteristics of wheat varieties, the aim at solve the issues of fuzzy entity boundaries and overlapping relationships in unstructured wheat variety approval data, WGIE-DCWF (joint extraction model of wheat germplasm information entity relationship based on deep character and word fusion) was proposed. The encoding layer of the model deeply fused word semantic information and character information using the Transformer encoder of BERT. This allowed for the cascading fusion of contextual semantic feature information to achieve rich character vector representation and improve the recognition ability of entity features. The triple extraction layer of the model established a cascading pointer network, extracted the head entity, extracted the tail entity according to the relationship category, and decoded the output triplet. This approach improved the model’s capability to extract overlapping relationships. The experimental results demonstrated that the WGIE-DCWF model performed exceptionally well on both the WGD (wheat germplasm dataset) and the public dataset DuIE. The WGIE-DCWF model not only achieved high performance on the evaluation datasets but also demonstrated good generalization. This provided valuable technical support for the construction of a wheat germplasm information knowledge base and is of great significance for wheat breeding, genetic research, cultivation management, and agricultural production.

Aspect Sentiment Triplet Extraction Based on Deep Relationship Enhancement Networks

The task of aspect-based sentiment analysis (ASBA) is to identify all the sentiment analyses expressed by specific aspect words in the text. How to identify specific objects (i.e., aspect words), describe the modifiers of the specific objects (i.e., opinion words), and judge the sentiment analysis expressed by opinion words (sentimental classification) in one step has become a focus of research in ASBA. ASTE (Aspect Sentiment Triplet Extraction) based on DREN (Deep Relationship Enhancement Networks) has been proposed in this paper. It aims to extract the aspect words and opinion words in the review text in one-step. They can judge the sentiment analysis expressed by the opinion words. Therefore, the study defines ten kinds of word relations; then, the study uses the parts of the speech feature, syntactic feature, relative position feature and tree distance relative feature to enhance the word representation relationship, which enriches the table of information in the relational matrix. Secondly, based on the word representation of BERT and GCN, the structural information of the texts are extracted; then, further extraction of higher-level word semantic information and word relationship information through SWDA (Sliding Window Dilated Attention) occurs, as SWDA can capture the multi-granularity relationship in words. Finally, the experimental results show that the proposed method is effective.

Adversarial Multi-task Learning for Efficient Chinese Named Entity Recognition

Named entity recognition (NER) is a fundamental task for information extraction applications. NER is challenging because of semantic ambiguities in academic literature, especially for non-Latin languages. Besides word semantic information, recognizing Chinese named entities needs to consider word boundary information, as words contained in Chinese texts are not separated with spaces. Leveraging word boundary information could help to determine entity boundaries and thus improve entity recognition performance. In this article, we propose to combine word boundary information and semantic information for named entity recognition based on multi-task adversarial learning. Specifically, we learn commonly shared boundary information of entities from multiple kinds of tasks, including Chinese word segmentation (CWS), part-of-speech (POS) tagging, and entity recognition, with adversarial learning. We learn task-specific semantic information of words from these tasks and combine the learned boundary information with the semantic information to improve entity recognition with multi-task learning. We then propose a compression method based on improved clustering to accelerate the proposed model. We conduct extensive experiments on four public benchmark datasets and two private datasets, compared with state-of-the-art baseline models, and the experimental results demonstrate that our model achieves considerable performance improvements on various evaluation datasets.

A Novel Discrimination Structure for Assessing Text Semantic Similarity

<p>Discrimination of semantic textual similarity refers to comparing the similarity between two or more entities (including words, short texts and documents) through certain strategies to obtain a specific quantitative similarity value. Traditional research put more experience into the similarity calculation of the original text content, using the matching degree or distance of characters or words as the yardstick to judge whether the text pairs are similar. However, there are still some problems to be solved in the following aspects: the key points of sentence meaning and word semantics, which play important role in the semantic expression of natural language, are not well integrated into the similarity discrimination, and the interactive features between texts are not fully utilized. To solve the above problems, this paper proposes a novel discrimination structure based on the Siamese Network model and the idea of text matching. In this structure, we introduce sentence meaning key information and word semantic information to realize the extraction of word interaction feature information, and then we realize the text vector representation by using Siamese BiLSTM. The experimental results showed that the accuracy of the proposed model is higher than that of the basic models.</p> <p> </p>

Research on Text Similarity Measurement Hybrid Algorithm with Term Semantic Information and TF-IDF Method

TF-IDF (term frequency-inverse document frequency) is one of the traditional text similarity calculation methods based on statistics. Because TF-IDF does not consider the semantic information of words, it cannot accurately reflect the similarity between texts, and semantic information enhanced methods distinguish between text documents poorly because extended vectors with semantic similar terms aggravate the curse of dimensionality. Aiming at this problem, this paper advances a hybrid with the semantic understanding and TF-IDF to calculate the similarity of texts. Based on term similarity weighting tree (TSWT) data structure and the definition of semantic similarity information from the HowNet, the paper firstly discusses text preprocess and filter process and then utilizes the semantic information of those key terms to calculate similarities of text documents according to the weight of the features whose weight is greater than the given threshold. The experimental results show that the hybrid method is better than the pure TF-IDF and the method of semantic understanding at the aspect of accuracy, recall, and F1-metric by different K-means clustering methods.

Improving biomedical word representation with locally linear embedding

Distributed word representation, usually obtained through calculation from large corpora, has been widely used in biomedical text because of its effectiveness in representing word semantic information. High-quality and meaningful biomedical words enable doctors to obtain the gist of information and knowledge in a short time to make clinical decisions quickly. Currently, the distributed word representation ignores the influence of the word embedding geometric structure obtained through calculation on the word semantic information and cannot accurately represent the word information, thus affecting the representation effect of biomedical text. To solve the above problems, we propose a biomedical word embedding framework based on manifold learning. Our work provides new perspectives for representing biomedical word embedding, which is the key concept in biomedical natural language processing tasks. First, the distributed word representation model is used to obtain the pretrained word embedding, and then the manifold learning is used to re-embed the pretrained word embedding. To verify the validity of the proposed framework in the biomedical domain, we evaluate the algorithm by using biomedical texts. Experimental results show that the proposed method can effectively improve the results of electronic health record classification and semantic similarity.

Relation Extraction using Language Model Based on Knowledge Graph

Relation extraction is an important task in natural language processing (NLP). The existing methods generally pay more attention on extracting textual semantic information from text, but ignore the relation contextual information from existed relations in datasets, which is very important for the performance of relation extraction task. In this paper, we represent each individual entity as a embedding based on entities and relations knowledge graph, which encodes the relation contextual information between the given entity pairs and relations. Besides, inspired by the impressive performance of language models recently, we used the language model to leverage word semantic information, in which word semantic information can be better captured than word embedding. The experimental results on SemEval2010 Task 8 dataset showed that the F1-score of our proposed method improved nearly 3% compared with the previous methods.

Incorporating Biterm Correlation Knowledge into Topic Modeling for Short Texts

Abstract The prevalence of short texts on the Web has made mining the latent topic structures of short texts a critical and fundamental task for many applications. However, due to the lack of word co-occurrence information induced by the content sparsity of short texts, it is challenging for traditional topic models like latent Dirichlet allocation (LDA) to extract coherent topic structures on short texts. Incorporating external semantic knowledge into the topic modeling process is an effective strategy to improve the coherence of inferred topics. In this paper, we develop a novel topic model—called biterm correlation knowledge-based topic model (BCK-TM)—to infer latent topics from short texts. Specifically, the proposed model mines biterm correlation knowledge automatically based on recent progress in word embedding, which can represent semantic information of words in a continuous vector space. To incorporate external knowledge, a knowledge incorporation mechanism is designed over the latent topic layer to regularize the topic assignment of each biterm during the topic sampling process. Experimental results on three public benchmark datasets illustrate the superior performance of the proposed approach over several state-of-the-art baseline models.

Learning Improved Semantic Representations with Tree-Structured LSTM for Hashtag Recommendation: An Experimental Study

A hashtag is a type of metadata tag used on social networks, such as Twitter and other microblogging services. Hashtags indicate the core idea of a microblog post and can help people to search for specific themes or content. However, not everyone tags their posts themselves. Therefore, the task of hashtag recommendation has received significant attention in recent years. To solve the task, a key problem is how to effectively represent the text of a microblog post in a way that its representation can be utilized for hashtag recommendation. We study two major kinds of text representation methods for hashtag recommendation, including shallow textual features and deep textual features learned by deep neural models. Most existing work tries to use deep neural networks to learn microblog post representation based on the semantic combination of words. In this paper, we propose to adopt Tree-LSTM to improve the representation by combining the syntactic structure and the semantic information of words. We conduct extensive experiments on two real world datasets. The experimental results show that deep neural models generally perform better than traditional methods. Specially, Tree-LSTM achieves significantly better results on hashtag recommendation than standard LSTM, with a 30% increase in F1-score, which indicates that it is promising to utilize syntactic structure in the task of hashtag recommendation.

Sentiment Analysis of Comment Texts Based on BiLSTM

With the rapid development of Internet technology and social networks, a large number of comment texts are generated on the Web. In the era of big data, mining the emotional tendency of comments through artificial intelligence technology is helpful for the timely understanding of network public opinion. The technology of sentiment analysis is a part of artificial intelligence, and its research is very meaningful for obtaining the sentiment trend of the comments. The essence of sentiment analysis is the text classification task, and different words have different contributions to classification. In the current sentiment analysis studies, distributed word representation is mostly used. However, distributed word representation only considers the semantic information of word, but ignore the sentiment information of the word. In this paper, an improved word representation method is proposed, which integrates the contribution of sentiment information into the traditional TF-IDF algorithm and generates weighted word vectors. The weighted word vectors are input into bidirectional long short term memory (BiLSTM) to capture the context information effectively, and the comment vectors are better represented. The sentiment tendency of the comment is obtained by feedforward neural network classifier. Under the same conditions, the proposed sentiment analysis method is compared with the sentiment analysis methods of RNN, CNN, LSTM, and NB. The experimental results show that the proposed sentiment analysis method has higher precision, recall, and F1 score. The method is proved to be effective with high accuracy on comments.

Word meaning in word identification during reading: Co-occurrence-based semantic neighborhood density effects

ABSTRACTIdentifying individual words is an essential part of the reading process that should occur first so that understanding the structural relations between words and comprehending the sentence as a whole may take place. Therefore, lexical processing (or word identification) has received much attention in the literature, with many researchers exploring the effects of different aspects of word representation (orthographic, phonological, and semantic information of words) in word identification. While the influence of many orthographic and phonological factors in normal reading are well researched and understood (Rayner, 1998, 2009), the effect of semantic characteristics of a word in its identification has received relatively less attention. A complete account of lexical processing during normal reading requires understanding the role of word meaning in lexical processing. Currently, little is understood about whether and how the meaning of an individual word is extracted during early stages of word identification. This article primarily focuses on how word meaning contributes to the process of word identification.

Method of determining keywords for English texts based on DKPro Core

The approaches to search of keywords in text that are divided into two linguistic and statistical categories are considered. Linguistic methods are based on the meaning of words, especially using ontologies and semantic information of words. Unfortunately, these methods are resource-intensive in the early stages - development of ontologies, for example, is very time-consuming process. It is proposed a new method for determining the keywords based on finding connections between word forms of the English text with the instrumental capabilities of package DKPro Core. The method, which illustrated with examples of analysis, aimed at solving problems of efficient processing of text documents - indexing, abstracting, clustering and classification. As a result of theoretical and experimental studies it is found that the developed method found more keywords, specified by the author of the text, compared to analogues. In addition, the proposed method without additional filters at least 5 times reduces the number of stop words among the top ten important (key) words. The results can be used to improve the accuracy of the content analysis of the site and raise the site position in search results. Unlike the existing methods the proposed method of determining the keywords based on the use of additional information about complex relationships between members of the English sentence. For the functional implementation of text analyzer it is selected the popular linguistic package DKPro Core. Experimental studies of theoretical substantiation of method are proved its quality advantages in comparison with known analogues.

HPS: High precision stemmer

Research into unsupervised ways of stemming has resulted, in the past few years, in the development of methods that are reliable and perform well. Our approach further shifts the boundaries of the state of the art by providing more accurate stemming results. The idea of the approach consists in building a stemmer in two stages. In the first stage, a stemming algorithm based upon clustering, which exploits the lexical and semantic information of words, is used to prepare large-scale training data for the second-stage algorithm. The second-stage algorithm uses a maximum entropy classifier. The stemming-specific features help the classifier decide when and how to stem a particular word.In our research, we have pursued the goal of creating a multi-purpose stemming tool. Its design opens up possibilities of solving non-traditional tasks such as approximating lemmas or improving language modeling. However, we still aim at very good results in the traditional task of information retrieval. The conducted tests reveal exceptional performance in all the above mentioned tasks. Our stemming method is compared with three state-of-the-art statistical algorithms and one rule-based algorithm. We used corpora in the Czech, Slovak, Polish, Hungarian, Spanish and English languages. In the tests, our algorithm excels in stemming previously unseen words (the words that are not present in the training set). Moreover, it was discovered that our approach demands very little text data for training when compared with competing unsupervised algorithms.

Unconscious semantic processing? No evidence for extracting the semantics of words during interocular suppression.

Continuous Flash Suppression (CFS) has been used as a method to probe unconscious processing of visually suppressed stimuli. One line of research pertains to the extent to which semantic information of words is processed in the absence of awareness. Since CFS was introduced, it has been reported that neutral words break suppression faster than negative words (Yang and Yeh, 2011) (and vice versa (Sklar et al. , 2013)) and that congruent prime-target relations influence suppression time (Costello et al., 2009), but do not influence electrophysiological correlates of semantic congruency (Kang et al., 2011). In this study, we used the breaking CFS paradigm to probe whether words break suppression faster than pseudo words and whether suppression time of words is influenced by its word frequency. In all experiments, participants were instructed to indicate the location of the initially suppressed words as fast as possible upon breakthrough. In Experiment 1, we found neither evidence for word type (real vs. pseudo words) nor word frequency influencing suppression time. In Experiment 2, we scrambled pseudo words to eliminate readability of pseudo words, included an inverted condition (to control for familiarity) and used a test-retest design (to examine the consistency of suppression times). Again, no effect of word type nor word frequency was found. Moreover, no effect of inversion was found and test-retest reliability was low. A control experiment was conducted to verify that reliable data could be obtained with this CFS paradigm. The target was a white disc of which the radius was manipulated. Bigger discs broke suppression faster than smaller discs indicating a basic stimulus manipulation can indeed yield reliable results in the predicted direction. In conclusion, our results provide no evidence for the extraction of semantic information in the absence of awareness induced by interocular suppression. Meeting abstract presented at VSS 2014

Multiple neural mechanisms for coloring words in synesthesia

Grapheme-color synesthesia is a phenomenon in which achromatic letters/digits automatically induce particular colors. When multiple letters are integrated into a word, some synesthetes perceive that all those letters are changed into the same color, reporting lexical color to that word. Previous psychological studies found several “rules” that determine those lexical colors. The colors to most words are determined by the first letters of the words, while some words in ordinal sequences have their specific colors. Recent studies further reported the third case where lexical colors might be influenced by semantic information of words. Although neural mechanisms determining those lexical colors remained unknown, here we identified three separate neural systems in the synesthete's brain underlying three rules for illusory coloring of words. In addition to the occipito-temporal and parietal regions previously found to be associated with the grapheme-color synesthesia, neural systems for lexical coloring extended to linguistic areas in the left inferior frontal and anterior temporal regions that were engaged in semantic analyses of words. Those results indicate an involvement of wider and higher neural networks than previously assumed in a production of synesthetic colors to visual stimuli and further showed a multiplicity of synesthetic mechanisms represented in the single brain.

단어 의미 정보를 활용하는 이용자 자연어 질의 유형의 효율적 분류

For question-answering system, question analysis module finds the question points from user`s natural language questions, classifies the question types, and extracts some useful information for answer. This paper proposes a question type classifying technique based on focus words extracted from questions and word semantic information, instead of complicated rules or huge knowledge resources. It also shows how to find the question type without focus words, and how useful the synonym or postfix information to enhance the performance of classifying module.