Chinese Text Similarity Research Articles

CTSARF: A Chinese Text Similarity Analysis Model based on Residual Fusion

Semantic textual similarity analysis is a fundamental task in the field of natural language processing. Both representational and interactive models are extensively employed for this purpose. However, representational models often suffer from issues such as semantic defocus, semantic shift, and low accuracy. In contrast, interactive models can accurately capture the semantic focus of a text and extract ample semantic information, but hindered by high time costs and low retrieval efficiency. To address these challenges, this paper introduces a Chinese Text Similarity Analysis model based on Residual Fusion (CTSARF). This model effectively implements the semantic feature fusion of representational and interactive models through a constant mapping method using residual networks. Therefore CTSARF can achieve both high accuracy and retrieval efficiency simultaneously. In this study, we employ the cosine-based loss function CoSENT and the fine-tuned pre-trained model Mengzi to optimize performance in text similarity analysis tasks. Additionally, we utilize a data augmentation method based on text editing to enhance the robustness of CTSARF when evaluating text similarity on unbalanced datasets. We conducted optimization comparison experiments, benchmark model comparison experiments, and ablation experiments of the CTSARF model on four Chinese datasets. The results demonstrate improvements in accuracy, recall, precision, and F1 scores, thereby highlighting the necessity and superiority of the model fusion method based on residual networks.

Attention-Based Overall Enhance Network for Chinese Semantic Textual Similarity Measure

ABSTRACT Semantic text similarity(STS) measure plays an important role in the practical application of natural language processing. However, due to the complexity of Chinese semantic comprehension and the lack of currently available Chinese text similarity dataset, present research on Chinese semantic text similarity still exists many limitations. In this paper, we construct a new private self-built Chinese semantic similarity (NCSS) dataset and propose a new method called Attention-based Overall Enhance Network (ABOEN) for measuring semantic textual similarity. This model takes advantage of convolutional neural network upon soft attention layers to capture more fine-grained interactive features between two sentences. Besides, inspired by the channel attention mechanism in image classification, we adopt a channel attention mechanism to enhance the critical overall interactive features between two sentences. The experimental results show that compared with other baseline models, the accuracy based on our model on the NCSS and LCQMC datasets has increased by 1.38% and 1.49%, respectively, which proves the effectiveness of our proposed model.

A Text Similarity Measurement Based on Semantic Fingerprint of Characteristic Phrases

Text similarity measurements are the basis for measuring the degree of matching between two or more texts. Traditional large-scale similarity detection methods based on a digital fingerprint have the advantage of high detection speed, which are only suitable for accurate detection. We propose a method of Chinese text similarity measurement based on feature phrase semantics. Natural language processing (NLP) technology is used to pre-process text and extract the keywords by the Term frequency-Inverse document frequency (TF-IDF) model and further screen out the feature words. We get the exact meaning of a word and semantic similarities between words and a HowNet semantic dictionary. We substitute concepts to get the feature phrases and generate a semantic fingerprint and calculate similarity. The experimental results indicate that the method proposed is superior in similarity detection in terms of its accuracy rate, recall rate, and F-value to the traditional and digital fingerprinting method.

Chinese Text Similarity Algorithm Based on Part-of-Speech Tagging and Word Vector Model

Chinese Text Similarity Algorithm Based on Part-of-Speech Tagging and Word Vector Model

Study of Chinese Text Similarity Based on Difference Factor in Word-Number

Text similarity calculation is the basic work in the application of Chinese information processing. A high-quality text similarity calculation method must be accurate and efficient, that is, it can be able to compare texts from the level of text natural language meaning, and arrive at the similarity distinction similar to artificial reading based on a full understanding of the author or text source semantic. At the same time, it should also be an efficient algorithm to save the processing time in facing large amount of text information to be processed. Through the research of many domestic and foreign literature, analysis and further research on current situation of similarity calculation, this paper intended to present a new method to improve the performance of similarity calculation, namely a Chinese text similarity algorithm based on word-number difference, which combined the traditional based on statistics and the narrow semantic method that meant the combination of the statistical efficiency and semantic accuracy. Combining the advantages of statistics and semantic category also means the necessity to face and overcome disadvantages of the two kinds of methods. This paper attempted to take the difference in word-number as the breakthrough point, took advantage of the diversity of Chinese word-number, combining with the word frequency, number and meaning, in order to successfully extend the word similarity calculation to the text similarity calculation. Finally, introduced the self built small text set as test object, compared similarity calculation of different methods in the laboratory environment. It shows that the similarity calculation method based on difference in word-number performances better than the traditional methods based on statistical and semantic. Through artificial comparison of the test results of research on this topic in accuracy and speed of segmentation, provide a new approach for Chinese text similarity calculation