TF-IDF Formula Research Articles

Composer classification using melodic combinatorial n-grams

In the present study, we investigate the supervised problem of composer classification. From a set of compositions and a set of composers, we seek to assign each composition to the correct composer using machine learning and natural language processing techniques. Our objective focused on using the n-gram technique to create vector representations of musical compositions and classify them using the Support Vector Machines (SVM) classifier on a term-frequency matrix composed of the vectors of the compositions. Our representation takes into account melodic relationships between instruments in polyphonic pieces. We extract n-grams in melodic direction, allowing us to go from one instrument to another in the process, which aims to generate more robust n-grams and a greater quantity of occurrences of n-grams. We evaluate different classification models using feature filtering and varying hyperparameters such as the TF-IDF formula, among others. We test our method on a dataset made of string quartets by composers Haydn and Mozart, achieving results that improves upon previous state-of-the-art results.

Research on Text Classification Techniques Based on Improved TF-IDF Algorithm and LSTM Inputs

Text classification is a technique that automatically classifies and labels text according to certain rules, and is widely used in sentiment analysis, intelligent recommendation systems and intelligent question and answer systems. Deep learning-based text classification methods can automatically identify and extract features in text that are useful for classification, so that it can analyse the text content directly, saving a lot of labour costs required for manual feature extraction. In this paper, the TF-IDF algorithm and the input structure of bidirectional LSTM was modified. Specifically, we rewrote the TF-IDF formula and processed the texts with a sliding window. The word2vec vector was used as the word embedding layer, and the text features were extracted using a combination of bidirectional LSTM and Text-CNN for classification prediction. Bidirectional LSTM treats texts as sequences to grasp information as a whole, while Text-CNN can extract local important features at the sentence level. As a result, good accuracy was achieved.

Improved algorithm of Context Graph based on feature selection

In order to solve the problem of low efficiency of traditional theme crawlers in searching theme pages, the crawling algorithm based on Context Graph was discussed. After analyzing the working principle and process of the algorithm, we introduced a new algorithm idea named feature selection algorithm. This new algorithm improved the original TF-IDF formula accordingly and solved the algorithm problems.

A refined method for computing bibliographic coupling strengths

Abstract We propose a new method for computing the bibliographic coupling strength between two documents. This new method is based on the TF-IDF formula from the field of information retrieval. It is shown that this formula is a valid alternative for the original formula introduced by Kessler and is, from a probabilistic point of view, a correction of the Vladutz-Cook formula. We further define a cosine based similarity formula generalizing the Sen-Gan coupling angle formula.

Comments Mining With TF-IDF: The Inherent Bias and Its Removal

Text mining have gained great momentum in recent years, with user-generated content becoming widely available. One key use is comment mining, with much attention being given to sentiment analysis and opinion mining. An essential step in the process of comment mining is text pre-processing; a step in which each linguistic term is assigned with a weight that commonly increases with its appearance in the studied text, yet is offset by the frequency of the term in the domain of interest. A common practice is to use the well-known tf-idf formula to compute these weights. This paper reveals the bias introduced by between-participants’ discourse to the study of comments in social media, and proposes an adjustment. We find that content extracted from discourse is often highly correlated, resulting in dependency structures between observations in the study, thus introducing a statistical bias. Ignoring this bias can manifest in a non-robust analysis at best and can lead to an entirely wrong conclusion at worst. We propose an adjustment to tf-idf that accounts for this bias. We illustrate the effects of both the bias and correction with with seven Facebook fan pages data, covering different domains, including news, finance, politics, sport, shopping, and entertainment.

News event evolution model based on the reading willingness and modified TF-IDF formula

News event evolution model based on the reading willingness and modified TF-IDF formula

Between-Participantss Discourse Bias in Comments Classification: Adjusting Tf-idf

Text mining and natural language processing have gained great momentum in recent years, with user-generated content becoming widely available. One key use is comment classification, with much attention being given to sentiment analysis and opinion mining. An essential step in the process of comment classification is text pre-processing; a step in which each linguistic term is assigned with a weight that commonly increases with its appearance in the studied text, yet is offset by the frequency of the term in the domain of interest. A common practice is to use the well known tf-idf formula to compute these weights.This paper reveals the bias introduced by between-participants' discourse to the study of comments in social media, and proposes a correction. We find that content extracted from between-participants' discourse is often highly correlated, resulting in dependency structures between observations in the study. Ignoring this bias can manifest in a non-robust analysis at best, and can lead to an entirely wrong conclusion at worst. We propose a statistical correction to tf-idf that accounts for this bias. We illustrate the effects of both the bias and correction with real data from Facebook.