Macro-average F1 Research Articles

Automatic prediction of news intent for search queries

PurposeThe purpose of this paper is to predict news intent by exploring contextual and temporal features directly mined from a general search engine query log.Design/methodology/approachFirst, a ground-truth data set with correctly marked news and non-news queries was built. Second, a detailed analysis of the search goals and topics distribution of news/non-news queries was conducted. Third, three news features, that is, the relationship between entity and contextual words extended from query sessions, topical similarity among clicked results and temporal burst point were obtained. Finally, to understand the utilities of the new features and prior features, extensive prediction experiments on SogouQ (a Chinese search engine query log) were conducted.FindingsNews intent can be predicted with high accuracy by using the proposed contextual and temporal features, and the macro average F1 of classification is around 0.8677. Contextual features are more effective than temporal features. All the three new features are useful and significant in improving the accuracy of news intent prediction.Originality/valueThis paper provides a new and different perspective in recognizing queries with news intent without use of such large corpora as social media (e.g. Wikipedia, Twitter and blogs) and news data sets. The research will be helpful for general-purpose search engines to address search intents for news events. In addition, the authors believe that the approaches described here in this paper are general enough to apply to other verticals with dynamic content and interest, such as blog or financial data.

Extensive Experimental Evaluation of Self-Organizing Maps for Automatic Classification of a Multi-Class Multi-Label Corpus

This paper aims at bridging the gap between feature selection and feature space size by utilizing both square and non-square self-organizing maps under different configuration scenarios for classifying a multi-class multi-label corpus, the Reuters Mod Apte’ split. The selection of non-square maps is based on a heuristic process for finding the suitable size for the self-organizing map. Vector construction is based on a simple, yet effective procedure aiming at transforming the vectors from multi-label to uni-label. The proposed solution improves classification efficiency not only in terms of accuracy but also in computational resources needed and time for training. Extensive experiments were conducted, using different configurations regarding map and vector sizes, and training cycles, also utilizing context words, to assess their impact in the classifier’s performance. Furthermore, an intelligent algorithm for label selection is being proposed, aiming to show that the neighboring nodes on the map affect the selection of labels for a specific node. According to the experiments conducted, our approach achieves 10% increase in Macro-Average F1 scores, $30\times $ decrease in vector dimensionality, and approximately $34\times $ smaller maps when compared to the baseline scenario.

Unsupervised feature selection method based on latent Dirichlet allocation model and mutual information

To solve the category-deficiency and the tendency of selecting low-frequency words in feature selection process based on Mutual Information(MI),the method named LDA-σ was presented.Firstly,the latent topics were extracted by the Latent Dirichlet Allocation(LDA) model,and then the standard deviation of "Word-Topic" MI was calculated as the feature evaluation function.When conducting feature selection and categorization in Reuters-21578,the micro average F1 of LDA-σ reached up to 0.909 6,and the highest macro average F1 of LDA-σ was 0.782 3,which were higher than that of other algorithms.The experimental results indicate that LDA-σ can be applied to feature selection in text sets.