Short Text Datasets Research Articles

Clustering Integrated Fusion Model Based on the Maximum Matching Problem

The maximum-weight perfect matching inverse issue in graph theory and text clustering are the two primary topics of this study. We suggest a novel approach to text clustering that makes use of self-encoders and BERT embeddings for feature extraction in order to increase clustering accuracy. According to experimental results, our technique enhances the clustering results greatly and performs well on numerous short text datasets. In the context of graph theory, we examine the unit paradigm inverse issue of maximum-weight perfect matching with value constraints and provide a robust polynomial-time method for its solution. In addition to effectively solving the maximum-weight perfect matching inverse issue, our technique can also produce the best weight vector configuration scheme for real-world uses. In conclusion, our work has advanced the domains of text clustering and graph theory significantly, offering fresh approaches and theoretical underpinnings for future investigations.

Short text classification using semantically enriched topic model

Modelling short text is challenging due to the small number of word co-occurrence and insufficient semantic information that affects downstream Natural Language Processing (NLP) tasks, for example, text classification. Gathering information from external sources is expensive and may increase noise. For efficient short text classification without depending on external knowledge sources, we propose Expressive Short text Classification (EStC). EStC consists of a novel document context-aware semantically enriched topic model called the Short text Topic Model (StTM) that captures words, topics and documents semantics in a joint learning framework. In StTM, the probability of predicting a context word involves the topic distribution of word embeddings and the document vector as the global context, which obtains by weighted averaging of word embeddings on the fly simultaneously with the topic distribution of words without requiring an additional inference method for the document embedding. EStC represents documents in an expressive (number of topics × number of word embedding features) embedding space and uses a linear support vector machine (SVM) classifier for their classification. Experimental results demonstrate that EStC outperforms many state-of-the-art language models in short text classification using several publicly available short text data sets.

ALGORITHM FOR DETECTION OF ABUSIVE CONTENT IN AUDIO CONTENT FOR IMPLEMENTATION IN OBJECT-ORIENTED INFORMATION SYSTEM

The paper proposes the basic principles of developing an object-oriented information system for detecting abusive content in Ukrainian-language audio content based on a new algorithm that uses statistical and neural network approaches to detect abusive content. Detecting offensive content in text and audio content is an urgent task, as it helps to create a safe and healthy environment for communication, especially in online platforms. Offensive content can harm the people who hear or read it and violate their rights. It can also have a negative impact on society, contributing to the spread of hatred and violence. To detect abusive speech in audio content, the proposed approach uses two key components: the use of dictionary methods and the analysis of the emotional tonality of utterances. A set of reviews was used as a dataset to determine the abusive component of the content, which was expanded by the authors by adding words of abuse. An object-oriented information system architecture written in the Python programming language in the PyCharm programming environment is proposed. The information system consists of a software module for training recurrent neural network models and further saving trained instances, and a software module for detecting abusive content in Ukrainian-language audio content using trained RNN models. Since the recurrent neural network is trained on a short text data set, the system is less efficient at identifying texts that have a larger number of words. In the example of the proposed approach, the accuracy of detecting offensive content is more than 90%. This means that the algorithm works correctly in the absence of offending highlights in the test data set. The results of the analysis of the effectiveness of the proposed approach show that in the vast majority of cases the conclusions regarding the acceptability of audio content based on the level of abuse are correct.

Co-occurrence word model for news media hotspot mining-text mining method design.

Currently, with the rapid growth of online media, more people are obtaining information from it. However, traditional hotspot mining algorithms cannot achieve precise and fast control of hot topics. Aiming at the problem of poor accuracy and timeliness in current news media hotspot mining methods, this paper proposes a hotspot mining method based on the co-occurrence word model. First, a new co-occurrence word model based on word weight is proposed. Then, for key phrase extraction, a hotspot mining algorithm based on the co-occurrence word model and improved smooth inverse frequency rank (SIFRANK) is designed. Finally, the Spark computing framework is introduced to improve the computing efficiency. The experimental outcomes expresses that the new word discovery algorithm discovered 16871 and 17921 new words in the Weibo Short News and Weibo Short Text datasets respectively. The heat weight values of the keywords obtained by the improved SIFRANK reaches 0.9356, 0.9991, and 0.6117. In the Covid19 Tweets dataset, the accuracy is 0.6223, the recall is 0.7015, and the F1 value is 0.6605. In the President-elects Tweets dataset, the accuracy is 0.6418, the recall is 0.7162, and the F1 value is 0.6767. After applying the Spark computing framework, the running speed has significantly improved. The text mining news media hotspot mining method based on the co-occurrence word model proposed in this study has improved the accuracy and efficiency of mining hot topics, and has great practical significance.

Contextual topic discovery using unsupervised keyphrase extraction and hierarchical semantic graph model

Recent technological advancements have led to a significant increase in digital documents. A document’s key information is generally represented by the keyphrases that provide the abstract description contained therein. With traditional keyphrase techniques, however, it is difficult to identify relevant information based on context. Several studies in the literature have explored graph-based unsupervised keyphrase extraction techniques for automatic keyphrase extraction. However, there is only limited existing work that embeds contextual information for keyphrase extraction. To understand keyphrases, it is essential to grasp both the concept and the context of the document. Hence, a hybrid unsupervised keyphrase extraction technique is presented in this paper called ContextualRank, which embeds contextual information such as sentences and paragraphs that are relevant to keyphrases in the keyphrase extraction process. We propose a hierarchical topic modeling approach for topic discovery based on aggregating the extracted keyphrases from ContextualRank. Based on the evaluation on two short-text datasets and one long-text dataset, ContextualRank obtains remarkable improvements in performance over other baselines in the short-text datasets.

An embedded feature selection approach for depression classification using short text sequences

Depression has become a serious mental health issue worldwide, particularly due to the rise of the Global Pandemic. Identifying depression of an individual from short texts shared in social media is a challenging task. The present work aims to select the optimal feature subset for classifying short texts for depression detection. By performing feature selection, it is possible to eliminate redundant and noisy features in high-dimensional datasets with small sample sizes. This can prevent the ”curse of dimensionality” and enhance the effectiveness of classification algorithms. However, current feature selection methods often focus on optimizing classification or clustering performance, while neglecting the stability of the selected features. This can lead to unstable results and make it challenging to identify meaningful and interpretable features. This paper introduces a novel embedded feature selection approach named Statistical Relevance Class Frequency based on Whale Optimization Algorithm (SRCF-WOA) for selecting feature subsets from short texts in social media. The proposed methodology extracts both the unigram features and composite features to capture the semantic and structural information. χ2.rcf (Chi-squared relevance class frequency) filter approach is applied to rank the extracted features to signify the importance of the features. WOA is adapted to retrieve the optimal subset of features with low-dimensional space using its high exploration and high exploitation capability. In the evaluation process, four benchmark short text datasets and two classifiers are used. The comparison shows that the proposed embedded feature selection method outperforms other algorithms in terms of accuracy and Fβ scores(β=0.5,1, and 2). The sensitivity analysis is carried out to check the robustness and stability of the proposed method. The findings indicate that the SRCF-WOA surpasses other methods on the majority of datasets, achieving the maximum classification accuracy while utilizing the minimal features. The statistical importance of these findings is further supported by the Analysis of Variance (ANOVA) F-test. Moreover, the proposed method strikes the optimal balance between classification accuracy and feature stability.

Evolution of online public opinions on major accidents: Implications for post-accident response based on social media network

Based on the incredible breadth and speed of information distribution within social media network, and continuous development of natural language processing techniques for sentiment and topic analyses, it is potential to access online public opinions on major accidents. An integrated framework including four-stage evolution model, lexicon-based sentiment analysis, and latent Dirichlet allocation (LDA)-based topic extraction was developed based upon social media data analysis. The explosion of Xiangshui eco-chemical industrial zone (EXEIZ) was taken as a case for investigating the evolution of online public opinions. The large accident domain (LAD)-based sentiment dictionary was established for the lexicon-based approach for sentiment determination. Effective methods (i.e., combination of short microblogs and comments into a new text, and the hybrid approach integrating perplexity with principal component analysis) were deployed for overcoming two typical shortcomings (i.e., inappropriate for short text dataset, and sensitive to the number of topics within a corpus) of the LDA-based approach for topic analysis and extraction. According to the four-stage evolution model, customized strategies and suggestions were provided for guiding and controlling online public opinions on major accidents, in order to decrease their negative impacts on the society. This enabled conducting targeted communication efforts among different stakeholders for the reduction of negative sentiments such as indignation, sadness, and threat, and the avoidance of social media crises. Although a major accident poses tangible threats to the public, it may improve their awareness for preventing from these threats. In case of appropriate measures in time, the focus tends to steer toward effective and prosocial solutions. It is helpful for sustaining and re-establishing the image of authorities, enterprises or individuals that are closely associated with the major accident.

A Fine-grained Chinese Short Text Classification Method Based on Capsule Networks

Chinese short-text classification is an essential primary research direction in natural language processing. The main problem facing the current Chinese short text classification task is how to effectively extract critical semantic feature information and determine the text category within a short text length. Aiming at the problem of feature information extraction of Chinese short texts, this paper introduces a novel sparse attention mechanism. It proposes a Chinese short text classification model RGSC combined with capsule networks. After the model generates word vectors through the RoBERTa pre-training model, the Bi-GRU model initially extracts text feature information. The profound text feature extraction module SC, composed of sparse attention mechanism and capsule networks, is used to further extract critical semantic feature information. The results of ablation experiments on the TouTiao Chinese short text dataset and performance comparison experiments with various models show that the RGSC model can effectively extract the essential text feature information while effectively reducing the irrelevant noise information contained in the text features, and obtain fine-grained text feature information.

WETM: A word embedding-based topic model with modified collapsed Gibbs sampling for short text

Short texts are a common source of knowledge, and the extraction of such valuable information is beneficial for several purposes. Traditional topic models are incapable of analyzing the internal structural information of topics. They are mostly based on the co-occurrence of words at the document level and are often unable to extract semantically relevant topics from short text datasets due to their limited length. Although some traditional topic models are sensitive to word order due to the strong sparsity of data, they do not perform well on short texts. In this paper, we propose a novel word embedding-based topic model (WETM) for short text documents to discover the structural information of topics and words and eliminate the sparsity problem. Moreover, a modified collapsed Gibbs sampling algorithm is proposed to strengthen the semantic coherence of topics in short texts. WETM extracts semantically coherent topics from short texts and finds relationships between words. Extensive experimental results on two real-world datasets show that WETM achieves better topic quality, topic coherence, classification, and clustering results. WETM also requires less execution time compared to traditional topic models.

Short Text Classification Based on Hierarchical Heterogeneous Graph and LDA Fusion

The proliferation of short texts resulting from the rapid advancements of social networks, online communication, and e-commerce has created a pressing need for short text classification in various applications. This paper presents a novel approach for short text classification, which combines a hierarchical heterogeneous graph with latent Dirichlet allocation (LDA) fusion. Our method first models the short text dataset as a hierarchical heterogeneous graph, which incorporates more syntactic and semantic information through a word graph, parts-of-speech (POS) tag graph, and entity graph. We then connected the representation of these three feature maps to derive a comprehensive feature vector for the text. Finally, we used the LDA topic model to adjust the feature weight, enhancing the effectiveness of short text extension. Our experiments demonstrated that our proposed approach has a promising performance in English short text classification, while in Chinese short text classification, although slightly inferior to the LDA + TF-IDF method, it still achieved promising results.

V-A3tS: A rapid text steganalysis method based on position information and variable parameter multi-head self-attention controlled by length

Text length varies in social networks, such as IMDB long text, Twitter short text, and long–short mixed text. For these complex situations, the time series, convolution, or fine-tuned BERT models are used in existing text steganalysis methods almost. However, these methods do not simultaneously consider higher detection accuracy and lower training time at the same time. To alleviate this dilemma, this paper proposes a novel text steganalysis method. First, the proposed method maps words into a semantic space containing position information. Second, a variable parameter attention layer scaled appropriately according to text length is designed, it achieves the purpose that the entire parameter amount of the model is not redundant and can ensure effective detection. Finally, the steganalysis features are enhanced by the residual linear layer. For long, short, and mixed text datasets, comparing experiments show that the proposed method has higher detection accuracy, fewer parameters, and shorter training time than existing methods. Among them, the advantage of this method is more obvious for long and mixed texts.

A Method of Sustainable Development for Three Chinese Short-Text Datasets Based on BERT-CAM

Considering the low accuracy of current short text classification (TC) methods and the difficulties they have with effective emotion prediction, a sustainable short TC (S-TC) method using deep learning (DL) in big data environments is proposed. First, the text is vectorized by introducing a BERT pre-training model. When processing language tasks, the TC accuracy is improved by removing a word from the text and using the information from previous words and the next words to predict. Then, a convolutional attention mechanism (CAM) model is proposed using a convolutional neural network (CNN) to capture feature interactions in the time dimension and using multiple convolutional kernels to obtain more comprehensive feature information. CAM can improve TC accuracy. Finally, by optimizing and merging bidirectional encoder representation from the transformers (BERT) pre-training model and CAM model, a corresponding BERT-CAM classification model for S-TC is proposed. Through simulation experiments, the proposed S-TC method and the other three methods are compared and analyzed using three datasets. The results show that the accuracy, precision, recall, F1 value, Ma_F and Mi_F are the largest, reaching 94.28%, 86.36%, 84.95%, 85.96%, 86.34% and 86.56, respectively. The algorithm’s performance is better than that of the other three comparison algorithms.

A semi-supervised short text sentiment classification method based on improved Bert model from unlabelled data

Short text information has considerable commercial value and immeasurable social value. Natural language processing and short text sentiment analysis technology can organize and analyze short text information on the Internet. Natural language processing tasks such as sentiment classification have achieved satisfactory performance under a supervised learning framework. However, traditional supervised learning relies on large-scale and high-quality manual labels and obtaining high-quality label data costs a lot. Therefore, the strong dependence on label data hinders the application of the deep learning model to a large extent, which is the bottleneck of supervised learning. At the same time, short text datasets such as product reviews have an imbalance in the distribution of data samples. To solve the above problems, this paper proposes a method to predict label data according to semi-supervised learning mode and implements the MixMatchNL data enhancement method. Meanwhile, the Bert pre-training model is updated. The cross-entropy loss function in the model is improved to the Focal Loss function to alleviate the data imbalance in short text datasets. Experimental results based on public datasets indicate the proposed model has improved the accuracy of short text sentiment recognition compared with the previous update and other state-of-the-art models.

A Drift-Sensitive Distributed LSTM Method for Short Text Stream Classification

Real-world applications especially in the fields of social media have produced massive short text streams. Unlike traditional normal texts, these data present the characteristics of short length, high-volume, high-velocity and variable data distribution etc, which lead to the issues of data sparsity and concept drift. It is hence very challenging for existing short text classification algorithms. Therefore, we propose a flexible Long Short-Term Memory (LSTM) ensemble network based short text stream classification approach, which is implemented in a distributed mode while maintaining the high-accuracy advantage of deep learning models. More specifically, external resource based short text embedding using a pretrained embedding model and CNN is first proposed for the solution to the data sparsity of short texts. Second, to adapt to the high-volume and high-velocity short text streams, a flexible LSTM network is developed and implemented in a distributed mode for classifying short text data streams. Third, a concept drift factor is introduced for adapting to the concept drifts caused by the changing of data distributions. Finally, experiments conducted on three real short text data sets demonstrate that as compared with several state-of-the-art short text (stream) classification approaches, the proposed approach can classify short text streams effectively and efficiently while adapting to concept drifts.

Advanced Hierarchical Topic Labeling for Short Text

Hierarchical Topic Modeling is the probabilistic approach for discovering latent topics distributed hierarchically among the documents. The distributed topics are represented with the respective topic terms. An unambiguous conclusion from the topic term distribution is a challenge for readers. The hierarchical topic labeling eases the challenge by facilitating an individual, appropriate label for each topic at every level. In this work, we propose a BERT-embedding inspired methodology for labeling hierarchical topics in short text corpora. The short texts have gained significant popularity on multiple platforms in diverse domains. The limited information available in the short text makes it difficult to deal with. In our work, we have used three diverse short text datasets that include both structured and unstructured instances. Such diversity ensures the broad application scope of this work. Considering the relevancy factor of the labels, the proposed methodology has been compared against both automatic and human annotators. Our proposed methodology outperformed the benchmark with an average score of 0.4185, 49.50, and 49.16 for cosine similarity, exact match, and partial match, respectively.

JACLNet:Application of adaptive code length network in JavaScript malicious code detection.

Currently, JavaScript malicious code detection methods are becoming more and more effective. Still, the existing methods based on deep learning are poor at detecting too long or too short JavaScript code. Based on this, this paper proposes an adaptive code length deep learning network JACLNet, composed of convolutional block RDCNet, BiLSTM and Transfrom, to capture the association features of the variable distance between codes. Firstly, an abstract syntax tree recombination algorithm is designed to provide rich syntax information for feature extraction. Secondly, a deep residual convolution block network (RDCNet) is designed to capture short-distance association features between codes. Finally, this paper proposes a JACLNet network for JavaScript malicious code detection. To verify that the model presented in this paper can effectively detect variable JavaScript code, we divide the datasets used in this paper into long text dataset DB_Long; short text dataset DB_Short, original dataset DB_Or and enhanced dataset DB_Re. In DB_Long, our method's F1 - score is 98.87%, higher than that of JSContana by 2.52%. In DB_Short, our method's F1-score is 97.32%, higher than that of JSContana by 7.79%. To verify that the abstract syntax tree recombination algorithm proposed in this paper can provide rich syntax information for subsequent models, we conduct comparative experiments on DB_Or and DB_Re. In DPCNN+BiLSTM, F1-score with abstract syntax tree recombination increased by 1.72%, and in JSContana, F1-score with abstract syntax tree recombination increased by 1.50%. F1-score with abstract syntax tree recombination in JACNet improved by 1.00% otherwise unused.

RSVN: A RoBERTa Sentence Vector Normalization Scheme for Short Texts to Extract Semantic Information

With the explosive growth in short texts on the Web and an increasing number of Web corpora consisting of short texts, short texts are playing an important role in various Web applications. Entity linking is a crucial task in knowledge graphs and a key technology in the field of short texts that affects the accuracy of many downstream tasks in natural language processing. However, compared to long texts, the entity-linking task of Chinese short text is a challenging problem due to the serious colloquialism and insufficient contexts. Moreover, existing methods for entity linking in Chinese short text underutilize semantic information and ignore the interaction between label information and the original short text. In this paper, we propose a RoBERTa sentence vector normalization scheme for short texts to fully extract the semantic information. Firstly, the proposed model utilizes RoBERTa to fully capture contextual semantic information. Secondly, the anisotropy of RoBERTa’s output sentence vectors is revised by utilizing the standard Gaussian of flow model, which enables the sentence vectors to more precisely characterize the semantics. In addition, the interaction between label embedding and text embedding is employed to improve the NIL entity classification. Experimental results demonstrate that the proposed model outperforms existing research results and mainstream deep learning methods for entity linking in two Chinese short text datasets.

Corpus Statistics Empowered Document Classification

In natural language processing (NLP), document classification is an important task that relies on the proper thematic representation of the documents. Gaussian mixture-based clustering is widespread for capturing rich thematic semantics but ignores emphasizing potential terms in the corpus. Moreover, the soft clustering approach causes long-tail noise by putting every word into every cluster, which affects the natural thematic representation of documents and their proper classification. It is more challenging to capture semantic insights when dealing with short-length documents where word co-occurrence information is limited. In this context, for long texts, we proposed Weighted Sparse Document Vector (WSDV), which performs clustering on the weighted data that emphasizes vital terms and moderates the soft clustering by removing outliers from the converged clusters. Besides the removal of outliers, WSDV utilizes corpus statistics in different steps for the vectorial representation of the document. For short texts, we proposed Weighted Compact Document Vector (WCDV), which captures better semantic insights in building document vectors by emphasizing potential terms and capturing uncertainty information while measuring the affinity between distributions of words. Using available corpus statistics, WCDV sufficiently handles the data sparsity of short texts without depending on external knowledge sources. To evaluate the proposed models, we performed a multiclass document classification using standard performance measures (precision, recall, f1-score, and accuracy) on three long- and two short-text benchmark datasets that outperform some state-of-the-art models. The experimental results demonstrate that in the long-text classification, WSDV reached 97.83% accuracy on the AgNews dataset, 86.05% accuracy on the 20Newsgroup dataset, and 98.67% accuracy on the R8 dataset. In the short-text classification, WCDV reached 72.7% accuracy on the SearchSnippets dataset and 89.4% accuracy on the Twitter dataset.

A new metric for feature selection on short text datasets

AbstractIn recent years, short texts are everywhere, especially in social media networks. Short text classification is an essential task for various applications related to the operations on short text documents. In many cases, using the entire feature set causes the high dimensionality problem in short text data. This problem reason of time‐consuming and negatively impacts the performance of classifiers. This study presents an effective feature selection algorithm called XY method, which represents the features on XY line and calculates the distance of a feature to the XY line. Also, a value named λ is calculated. According to this value, the terms are divided into different regions such as negative, positive, and third to determine their discrimination capability. The novel XY method aims to select as few terms as possible in the negative region. The proposed method is evaluated using four different short text datasets with Macro‐F1 success measure. In comparisons with other existing feature selection algorithms such as chi‐square, information gain, deviation from Poisson distribution, recently proposed max‐min ratio, and distinguishing feature selector demonstrate that the XY method achieves either better or competitive performance in significantly reduced various feature sizes.

Research of Text Classification Based on TF-IDF and CNN-LSTM

With the rapid development of deep learning, many deep learning models have been widely used in Natural Language Processing(NLP). The Long-Short Term memory network(LSTM) model and convolutional neural network(CNN) model can achieve high classification accuracy in text classification tasks. However, the high input dimension of text features and the need to train a large number of parameters in the deep learning model often take a lot of time. This paper uses Term Frequency-inverse Document Frequency(TF-IDF) to remove features with lower weights, extract key features in the text, extract the corresponding word vector through the Word2Vec model, and then input it into the CNN-LSTM model. We compared the model with CNN, LSTM, and LSTM-attention methods and found that the model can significantly reduce model parameters and training time in short and long text data sets. The model hardly loses accuracy in the long text, but the model will lose a certain amount of accuracy in short texts. This paper also proposes fusing original text features to make up for the accuracy loss caused by the TF-IDF feature extraction method.