Chinese Short Text Research Articles

RSVN: A RoBERTa Sentence Vector Normalization Scheme for Short Texts to Extract Semantic Information

With the explosive growth in short texts on the Web and an increasing number of Web corpora consisting of short texts, short texts are playing an important role in various Web applications. Entity linking is a crucial task in knowledge graphs and a key technology in the field of short texts that affects the accuracy of many downstream tasks in natural language processing. However, compared to long texts, the entity-linking task of Chinese short text is a challenging problem due to the serious colloquialism and insufficient contexts. Moreover, existing methods for entity linking in Chinese short text underutilize semantic information and ignore the interaction between label information and the original short text. In this paper, we propose a RoBERTa sentence vector normalization scheme for short texts to fully extract the semantic information. Firstly, the proposed model utilizes RoBERTa to fully capture contextual semantic information. Secondly, the anisotropy of RoBERTa’s output sentence vectors is revised by utilizing the standard Gaussian of flow model, which enables the sentence vectors to more precisely characterize the semantics. In addition, the interaction between label embedding and text embedding is employed to improve the NIL entity classification. Experimental results demonstrate that the proposed model outperforms existing research results and mainstream deep learning methods for entity linking in two Chinese short text datasets.

Entity Linking Method for Chinese Short Text Based on Siamese-Like Network

Entity linking plays a fundamental role in knowledge engineering and data mining and is the basis of various downstream applications such as content analysis, relationship extraction, question and answer. Most existing entity linking models rely on sufficient context for disambiguation but do not work well for concise and sparse short texts. In addition, most of the methods use pre-training models to directly calculate the similarity between the entity text to be disambiguated and the candidate entity text, and do not dig deeper into the relationship between them. This article proposes an entity linking method for Chinese short texts based on Siamese-like networks to address the above shortcomings. In the entity disambiguation task, the features of the Siamese-like network are used to deeply parse the semantic relationships in the text and make full use of the feature information of the entity text to be disambiguated, capturing the interdependent features within the sentences through an attention mechanism, aiming to find out the most critical elements in the entity text description. The experimental demonstration on the CCKS2019 dataset shows that the F1 value of the method reaches 87.29%, increase of 11.02% compared to the F1 value(that) of the baseline method, fully validating the superiority of the model.

Chinese Short Text Classification by ERNIE Based on LTC_Block

Short text classification, an important direction of the basic research of natural language processing, has extensive applications. Its effect depends on feature extraction methods and feature representation methods. This paper proposed an LTC_Block-based short text classification model named ERNIE to classify Chinese short texts and extract semantics in the corpus to address the polysemy problem in the text. In this model, LTC_Block, a double-channel structural unit composed of BiLSTM and TextCNN, was used to extract the contextual sequences and overall features of semantics, and residual connection was used to integrate features and further classify short texts. Experiments on two different datasets showed that ERNIE achieved a better classification effect than mainstream models, proving its feasibility and effectiveness.

The short texts classification based on neural network topic model

Aiming at the low effectiveness of short texts feature extraction, this paper proposes a short texts classification model based on the improved Wasserstein-Latent Dirichlet Allocation (W-LDA), which is a neural network topic model based on the Wasserstein Auto-Encoder (WAE) framework. The improvements of W-LDA are as follows: Firstly, the Bag of Words (BOW) input in the W-LDA is preprocessed by Term Frequency–Inverse Document Frequency (TF-IDF); Subsequently, the prior distribution of potential topics in W-LDA is replaced from the Dirichlet distribution to the Gaussian mixture distribution, which is based on the Variational Bayesian inference; And then the sparsemax function layer is introduced after the hidden layer inferred by the encoder network to generate a sparse document-topic distribution with better topic relevance, the improved W-LDA is named the Sparse Wasserstein-Variational Bayesian Gaussian mixture model (SW-VBGMM); Finally, the document-topic distribution generated by SW-VBGMM is input to BiGRU (Bidirectional Gating Recurrent Unit) for the deep feature extraction and the short texts classification. Experiments on three Chinese short texts datasets and one English dataset represent that our model is better than some common topic models and neural network models in the four evaluation indexes (accuracy, precision, recall, F1 value) of text classification.

Aesthetic and Implication Analysis of the Traditional Poetic Environment Based on Natural Language Emotion Analysis.

The uniqueness of aesthetic implication in Zhou Dynasty poetics lies in that it is the basic forming stage of the concept of formal beauty of the whole Chinese nation, and the aesthetic implication of the Zhou Dynasty poetics art has fundamental significance for the whole ancient Chinese aesthetic implication theory. In the discipline of natural language processing, text emotion analysis is a crucial topic. Artificial neural network research is where the idea of “deep learning” (DL) first emerged. In view of the problems that semantic information is easy to be lost and emotional information may be ignored in the traditional Chinese short text emotion analysis model, this paper introduces the AM (attention mechanism) and proposes a CNN-LSTM (convolutional neural network-long short-term memory) poetic aesthetic implication analysis method based on self-attention. For the IL (input layer), word vectors trained by Word2Vec are used and then input into the CNN-LSTM joint model. Then, the output of the joint model is weighted and summed by self-attention and finally input into the Softmax classifier, so as to realize the emotion classification of the text. By creating and putting into practise pertinent comparative experiments, the usefulness of the proposed model is confirmed. The outcomes demonstrate that this model outperforms the other three comparison models for the quantification of evaluation indices in terms of overall performance. The accuracy and F1 of this paper are 93.362% and 90.886%, respectively, which are higher than other models.

Word-Level and Pinyin-Level Based Chinese Short Text Classification

Short text classification is an important branch of Natural Language Processing. Although CNN and RNN have achieved satisfactory results in the text classification tasks, they are difficult to apply to the Chinese short text classification because of the data sparsity and the homophonic typos problems of them. To solve the above problems, word-level and Pinyin-level based Chinese short text classification model is constructed. Since homophones have the same Pinyin, the addition of Pinyin-level features can solve the homophonic typos problem. In addition, due to the introduction of more features, the data sparsity problem of short text can be solved. In order to fully extract the deep hidden features of the short text, a deep learning model based on BiLSTM, Attention and CNN is constructed, and the residual network is used to solve the gradient disappearance problem with the increase of network layers. Additionally, considering that the complex deep learning network structure will increase the text classification time, the Text Center is constructed. When there is a new text input, the text classification task can be quickly realized by calculating the Manhattan distance between the embedding vector of it and the vectors stored in the Text Center. The Accuracy, Precision, Recall and F1 of the proposed model on the simplifyweibo_4_moods dataset are 0.9713, 0.9627, 0.9765 and 0.9696 respectively, and those on the online_shopping_10_cats dataset are 0.9533, 0.9416, 0.9608 and 0.9511 respectively, which are better than that of the baseline method. In addition, the classification time of the proposed model on simplifyweibo_4_moods and online_shopping_10_cats is 0.0042 and 0.0033 respectively, which is far lower than that of the baseline method.

Pseudo-siamese networks with lexicon for Chinese short text matching

Short text matching is one of the fundamental technologies in natural language processing. In previous studies, most of the text matching networks are initially designed for English text. The common approach to applying them to Chinese is segmenting each sentence into words, and then taking these words as input. However, this method often results in word segmentation errors. Chinese short text matching faces the challenges of constructing effective features and understanding the semantic relationship between two sentences. In this work, we propose a novel lexicon-based pseudo-siamese model (CL2 N), which can fully mine the information expressed in Chinese text. Instead of utilizing a character-sequence or a single word-sequence, CL2 N augments the text representation with multi-granularity information in characters and lexicons. Additionally, it integrates sentence-level features through single-sentence features as well as interactive features. Experimental studies on two Chinese text matching datasets show that our model has better performance than the state-of-the-art short text matching models, and the proposed method can solve the error propagation problem of Chinese word segmentation. Particularly, the incorporation of single-sentence features and interactive features allows the network to capture the contextual semantics and co-attentive lexical information, which contributes to our best result.

An emotional classification method of Chinese short comment text based on ELECTRA

Chinese short comment texts have the characteristics of feature sparseness, interlacing, irregularity, etc., which makes it difficult to fully grasp the overall emotional tendency of users. In response to such problem, the text proposes a new method based on ELECTRA and hybrid neural network. This method can more accurately capture the emotional features of the text, improve the classification effect, enhance the evaluation feedback mechanism, and facilitate user decision-making. First, in the embedding layer, ELECTRA model is used to replace BERT model, which can avoid the inconsistency of the mask training and fine-tuning process of the traditional pre-training model. Then, in the training layer, the self-attention mechanism and the BiLSTM are selected to obtain the fine-grained semantic representation information of the review text more comprehensively. Finally, in the output layer, the softmax classifier classifies the input corpus according to the sentiment characteristics of the Chinese short text. The experimental results show that the proposed model has an efficiently improvement in accuracy and there are some discoveries about the training effect of the pre-training model on text sentiment analysis tasks.

A Multi-Classification Sentiment Analysis Model of Chinese Short Text Based on Gated Linear Units and Attention Mechanism

Sentiment analysis of social media texts has become a research hotspot in information processing. Sentiment analysis methods based on the combination of machine learning and sentiment lexicon need to select features. Selected emotional features are often subjective, which can easily lead to overfitted models and poor generalization ability. Sentiment analysis models based on deep learning can automatically extract effective text emotional features, which will greatly improve the accuracy of text sentiment analysis. However, due to the lack of a multi-classification emotional corpus, it cannot accurately express the emotional polarity. Therefore, we propose a multi-classification sentiment analysis model, GLU-RCNN, based on Gated Linear Units and attention mechanism. Our model uses the Gated Linear Units based attention mechanism to integrate the local features extracted by CNN with the semantic features extracted by the LSTM. The local features of short text are extracted and concatenated by using multi-size convolution kernels. At the classification layer, the emotional features extracted by CNN and LSTM are respectively concatenated to express the emotional features of the text. The detailed evaluation on two benchmark datasets shows that the proposed model outperforms state-of-the-art approaches.

Improved Chinese Short Text Classification Method Based on ERNIE_BiGRU Model

This paper is aimed at traditional word embedding models and Bidirectional Encoder Representations from Transformers (BERT) that cannot learn text semantic knowledge, as well as convolutional neural network (CNN) and Bidirectional long short-term memory (BiLSTM) unable to distinguish the importance of words, proposing an improved Chinese short text classification method based on ERNIE_BiGRU model. Firstly, learning text knowledge and information through the Enhanced Representation through Knowledge Integration (ERNIE) enhances the model’s semantic representation capabilities. Secondly, considering that CNN can only extract local features of the text while ignoring the semantic relevance between contextual information, and the Bidirectional Gating Recurrent Unit (BiGRU) is simpler, has fewer network parameters and faster calculation speed than the BiLSTM, the combination of CNN and BiGRU enables the model to capture both local phrase-level features and contextual structure information. Finally, according to the importance of features, the attention mechanism is used to assign different weights to improve the classification effect of the model. The experimental results show that the ERNIE_CNN_BiGRU_Attention (ECBA) model used in this paper has achieved good results in the task of Chinese short text classification.

Chinese Text Summarization Generation Algorithm based on ERNIE-GEN with Few-shot Learning

In view of the problems of exposure bias and insufficient annotation data in the existing Chinese short text summary algorithms, a new pre-training short text summary algorithm named ERNIE-GEN-CTS based on ERNIE-GEN is proposed. This paper uses pre-training language model to realize the double attention of word level and word level for short text, and fills the sub block with semantic information to optimize the exposure bias problem. After the pre-training model of large-scale Chinese corpus, the results are improved in LCSTS data set after the task tuning of the downstream summary, and the validity and stability of the model are verified.

Grammar guided embedding based Chinese long text sentiment classification

AbstractAlthough the state‐of‐the‐art sentiment classification approaches, such as LSTM and TextCNN, have achieved a good performance on Chinese short text sentiment analysis, the Chinese long text sentiment classification is still a challenge because of the sentiment change problem and the long text structure problem. Therefore, we propose a grammar guided embedding model (GGE) and a novel Chinese long text sentiment classification framework. First, the part‐of‐speech (POS) tags are introduced as the Chinese long text grammar guided information which can help classification approaches to model the Chinese long text structure and the important structure of sentiment change. Second, we proposed a simple GGE training method which considers the combination representation of word sequence and POS sequence. Finally, we proposed a unified framework which combines our novel GGE with TextCNN. Experiment results show that after using GGE, the model outperforms the state‐of‐the‐art approaches. At the same time, we also found that the GGE achieves the model converge faster, that is, it can achieve better results than without GGE when there is only a small amount of training data. Thus, we believe that the GGE can help machines better understand human language sentiment expression structure.

LET: Linguistic Knowledge Enhanced Graph Transformer for Chinese Short Text Matching

Chinese short text matching is a fundamental task in natural language processing. Existing approaches usually take Chinese characters or words as input tokens. They have two limitations: 1) Some Chinese words are polysemous, and semantic information is not fully utilized. 2) Some models suffer potential issues caused by word segmentation. Here we introduce HowNet as an external knowledge base and propose a Linguistic knowledge Enhanced graph Transformer (LET) to deal with word ambiguity. Additionally, we adopt the word lattice graph as input to maintain multi-granularity information. Our model is also complementary to pre-trained language models. Experimental results on two Chinese datasets show that our models outperform various typical text matching approaches. Ablation study also indicates that both semantic information and multi-granularity information are important for text matching modeling.

RETRACTED ARTICLE: Research on online marketing effects based on multi-model fusion and artificial intelligence algorithms

With the popularity of the network, more and more users like to express their opinions and exchange through the network media. At present, compared with English text sentiment classification, the research on Chinese text sentiment classification is not deep enough. Therefore, sentiment classification of Chinese short text has great research value. This paper studies the influence of word vector, network structure, training method and loss function on deep learning text classification model. Firstly, the influence of word vector on the accuracy of deep learning text classification model is analyzed. It is found that the deep learning text classification model can obtain higher accuracy by using the pre trained word vector. Considering the fact that the wheel in textcnn will lose the important function information, this paper improves the wheel method and obtains the K-maxcnn network. Compared with the original network, k-max-cnn network and four improved DCNN networks can achieve higher accuracy. Secondly, aiming at the instability of the training results of deep learning, this paper proposes a line by line learning training method, which is “difficult at the beginning but simple later”, and proves that this method makes the results more stable through experiments. Finally, using word vector and network model, and using line by line learning training method, some deep learning models are trained and text classification is established, which is applied to text emotion classification. The experimental results show that the accuracy of the proposed method is improved by 0.79%, 2.85% and 2.05% compared with the most accurate model.

A dual deep neural network with phrase structure and attention mechanism for sentiment analysis

Sentiment analysis of short texts is difficult for their simplicity and compactness. This goes a step further when it comes to the Chinese texts. Although deep learning achieved better accuracy in sentiment analysis, there is a lack of explain-ability. Thus, this paper evaluates the effectiveness of techniques for sentiment analysis of Chinese short financial texts with deep learning. For this, we built a Chinese short financial texts corpus (CSFC) and designed an ablation experiment. Beside the CFSC, we used a Chinese review collection and an English short-text repository in the experiment for comparison. There are five techniques involved. They are the Pinyin, the segmentation, the lexical analysis, the phrase structure and the attention mechanism. As results, we found that the phrase structure and the attention mechanism are two of the best. Therefore, the best model in the experiment is called a Phrase Structure and Attention-based Deep network model (PhraSAD). Moreover, to improve the classification accuracy on neutral data, we use a dual classifier strategy for 3-class problems. Experimental results showed that PhraSAD outperformed all other compared models on all experimental datasets.

Research on Short Text Classification Based on TextCNN

The TextCNN model is widely used in text classification tasks. It has become a comparative advantage model due to its small number of parameters, low calculation, and fast training speed. However, training a convolutional neural network requires a large amount of sample data. In many cases, there are not enough data sets as training samples. Therefore, this paper proposes a Chinese short text classification model based on TextCNN, which uses back translation to achieve data augment and compensates for the lack of training data. The experimental data shows that our proposed model has achieved good results.

An LSTM&Topic-CNN Model for Classification of Online Chinese Medical Questions

In recent years, people’s interest in health question and answer (Q&A) websites has been growing with the development of the internet technologies. How to seek appropriate professional medical information among the massive data has become the focus of all patients. Therefore, it is vital to obtain reasonable predictions and automatic recommendations on the basis of patients’ keyword descriptions of their health status and question intention. The key to solving this problem is to achieve automatic text classification of health questions. This paper considered a feature fusion model for the classification of Chinese short texts on medical health Q&A websites by combining the text features and topic features. Firstly, we generated the text word vector by word embedding method and obtained the text features under Long Short-Term Memory (LSTM) model. Given the difficulty in determination of topic numbers, we conducted a sub-sample experiment to obtain the few optimal topic numbers under which the classification performances were good. Then we extracted the topic features and used the one-dimensional convolution idea of the Convolutional Neural Network (CNN) model for topic feature filtering. Finally, we combined the two features together subtly for text classification. Two experiments were conducted to illustrate our model in terms of recall rate, precision, and F1 value when the datasets were from different online medical Q&A websites. Results showed that the LSTM&Topic-CNN model could efficiently enhance the classification effect of Chinese medical health question texts.

Short Text Sentiment Analysis Based on Multi-Channel CNN With Multi-Head Attention Mechanism

In view of the limited text features of short texts, features of short texts should be mined from various angles, and multiple sentiment feature combinations should be used to learn the hidden sentiment information. A novel sentiment analysis model based on multi-channel convolutional neural network with multi-head attention mechanism (MCNN-MA) is proposed. This model combines word features with part of speech features, position features and dependency syntax features separately to form three new combined features, and inputs them into the multi-channel convolutional neural network, as well as integrates the multi-head attention mechanism to more fully learn the sentiment information in the text. Finally, experiments are carried out on two Chinese short text data sets. The experimental results show that the MCNN-MA model has a higher classification accuracy and a relatively low training time cost compared with other baseline models.

Research on sentiment classification of Chinese short text

Sentiment classification can be widely used in various fields because of the value it has gained, and has received extensive attention in recent years. No matter the traditional methods based on language rules, machine statistical learning, or the current popular deep learning have achieved great success in this respect, but up to now, because of the particularity of Chinese, there are still few methods for Chinese text sentiment classification. Moreover, the traditional method is time-consuming and labor-consuming, with poor versatility. The neural network method saves resources and is generally applicable, but the application of language knowledge is not sufficient. For this reason, we propose a method combining traditional method with neural network, which combines sentiment lexicon, attention mechanism and stacked BILSTM. Word vectors are generated using word2vec, and the context semantics are captured multiple times by stacked BILSTM to get richer semantic information, for sentiment classification, sentiment words often have more influence on the result of classification, so we use sentiment lexicon which contains rich sentiment word to monitor the attention mechanism, and make the attention model pay more attention to sentiment words. Combining the advantages of traditional methods and neural networks, we use the proposed method to do the sentiment binary classification experiments on two data sets. The experimental results show that our proposed method is superior to the previous method.

Chinese Short Text Classification with Mutual-Attention Convolutional Neural Networks

The methods based on the combination of word-level and character-level features can effectively boost performance on Chinese short text classification. A lot of works concatenate two-level features with little processing, which leads to losing feature information. In this work, we propose a novel framework called Mutual-Attention Convolutional Neural Networks, which integrates word and character-level features without losing too much feature information. We first generate two matrices with aligned information of two-level features by multiplying word and character features with a trainable matrix. Then, we stack them as a three-dimensional tensor. Finally, we generate the integrated features using a convolutional neural network. Extensive experiments on six public datasets demonstrate improved performance of our new framework over current methods.