Text Similarity Calculation Research Articles

Research on Automatic Scoring for Subjective Item of Database Domain Based on Database Synonyms Ontology

Nowadays, automatic scoring is an important way of teaching and examinations. However, there is no existing research on automatic scoring for subjective item of database domain both at home and abroad. According to the characteristics of database domain, we construct database domain synonyms ontology and proposed a text similarity calculation algorithm based on Hamming distance. Then we implement the automatic scoring for subjective item of database domain on the basis of ontology. In addition, in order to verify the accuracy and rationality of the algorithm, we take a specific subject as an example. The experiment results further illustrate the accuracy and efficiency of the proposed automatic scoring algorithm.

A Novel Method for Text Similarity Calculation

In view of the fact that traditional vector space model for text similarity calculation which does not take word order into consideration leads to bias, this paper puts forward a longest common subsequence and the traditional vector space model of combining text similarity calculation. This method takes the word order and word frequency information into account, using the texts of the longest common subsequence and substring of their information from all public records and the use of word order and word frequency in the text. The importance of similarity calculation is acknowledged, and the traditional vector space model in the calculation of the weight is used on the word frequency information. Some of the dataset collected through the web crawler are used in the proposed text similarity calculation method for testing, and the results proved the effectivity of the method.

A segment-based approach to clustering multi-topic documents

Document clustering has been recognized as a central problem in text data management. Such a problem becomes particularly challenging when document contents are characterized by subtopical discussions that are not necessarily relevant to each other. Existing methods for document clustering have traditionally assumed that a document is an indivisible unit for text representation and similarity computation, which may not be appropriate to handle documents with multiple topics. In this paper, we address the problem of multi-topic document clustering by leveraging the natural composition of documents in text segments that are coherent with respect to the underlying subtopics. We propose a novel document clustering framework that is designed to induce a document organization from the identification of cohesive groups of segment-based portions of the original documents. We empirically give evidence of the significance of our segment-based approach on large collections of multi-topic documents, and we compare it to conventional methods for document clustering.

Hybrid Algorithm for Improving Efficiency of Keywords based Search Engine

Web mining is the application of data mining techniques to discover interesting patterns from the Web. Web usage mining is the process of extracting useful information from server logs i.e users history. While discovering interesting patterns in multi agents the efficiency is decreased. In our paper presents hybrid algorithm is designed for the improvement of the efficiency of keywords based search engine. The model divides mining task into several parallel agents which coordinately work together, and the mining efficiency is improved greatly. The hybrid algorithm is Evolved from HITS, algorithm. Hybrid algorithm removes Link Farm pages in the expansion of root set, makes anchor text similarity calculation when crawling link page, and chooses pages by a brief conceptual analysis of page content. With the overcoming of the shortcomings of only text analysis or link analysis, Hybrid enhances the search engine in understanding the user interest and crawling more Web pages to meet the needs of the users.

An improved K-nearest-neighbor algorithm for text categorization

Text categorization is a significant tool to manage and organize the surging text data. Many text categorization algorithms have been explored in previous literatures, such as KNN, Naïve Bayes and Support Vector Machine. KNN text categorization is an effective but less efficient classification method. In this paper, we propose an improved KNN algorithm for text categorization, which builds the classification model by combining constrained one pass clustering algorithm and KNN text categorization. Empirical results on three benchmark corpora show that our algorithm can reduce the text similarity computation substantially and outperform the-state-of-the-art KNN, Naïve Bayes and Support Vector Machine classifiers. In addition, the classification model constructed by the proposed algorithm can be updated incrementally, and it has great scalability in many real-word applications.

An Improved Text Similarity Calculation Algorithm Based on VSM

Text similarity calculation is a key technology in the fields of text clustering, Web intelligent retrieval and natural language processing etc. Because the traditional text similarity calculation algorithm does not consider the affect of same feature words between texts, sometimes this algorithm may lead to inaccurate results. To solve this problem, this paper gives an improved text similarity calculation algorithm. Considering that the amount of same feature words reflects two texts’ similarity in some extent, the improved algorithm adds in the coverage measured parameter, which effectively reduces the interference of texts with lower similarity. The simulation and experimental results verify the improved algorithm’s correctness and effectiveness.

A Text Hybrid Clustering Algorithm Based on HowNet Semantics

Many existing text clustering algorithms overlook the semantic information between words and so they possess a lower accuracy of text similarity computation. A new text hybrid clustering algorithm (HCA) based on HowNet semantics has been proposed in this paper. It calculates the semantic similarity of words by using the words’ semantic concept description in HowNet and then combines it with the method of maximum weight matching of bipartite graph to calculate a semantic-based text similarity. Based on the new text similarity and by combining an improved genetic algorithm with k-medoids algorithm, HCA has been designed. The comparative experiments show that: 1) compared with two existing traditional clustering algorithms, HCA can get better quality and 2) when their text cosine similarity is replaced with the new semantic-based text similarity, all the qualities of the three clustering algorithms can be improved significantly.