Text Clustering Research Articles

Experimental study on short-text clustering using transformer-based semantic similarity measure

Sentence clustering plays a central role in various text-processing activities and has received extensive attention for measuring semantic similarity between compared sentences. However, relatively little focus has been placed on evaluating clustering performance using available similarity measures that adopt low-dimensional continuous representations. Such representations are crucial in domains like sentence clustering, where traditional word co-occurrence representations often achieve poor results when clustering semantically similar sentences that share no common words. This article presents a new implementation that incorporates a sentence similarity measure based on the notion of embedding representation for evaluating the performance of three types of text clustering methods: partitional clustering, hierarchical clustering, and fuzzy clustering, on standard textual datasets. This measure derives its semantic information from pre-training models designed to simulate human knowledge about words in natural language. The article also compares the performance of the used similarity measure by training it on two state-of-the-art pre-training models to investigate which yields better results. We argue that the superior performance of the selected clustering methods stems from their more effective use of the semantic information offered by this embedding-based similarity measure. Furthermore, we use hierarchical clustering, the best-performing method, for a text summarization task and report the results. The implementation in this article demonstrates that incorporating the sentence embedding measure leads to significantly improved performance in both text clustering and text summarization tasks.

TeC: A Novel Method for Text Clustering with Large Language Models Guidance and Weakly-Supervised Contrastive Learning

TeC: A Novel Method for Text Clustering with Large Language Models Guidance and Weakly-Supervised Contrastive Learning

Systematic Review: HIV, Aging, and Housing-A North American Perspective, 2012-2023.

Advances in anti-retroviral therapy (ART) have decreased mortality rates and subsequently led to a rise in the number of HIV-positive people living longer. The housing experiences of this new population of interest-older adults (50 years and older) living with HIV-are under-researched. Understanding the housing experiences and unmet needs of older people with HIV can better provide comprehensive care services for them. This study's systematic review evaluated the peer-reviewed literature reporting housing access/insecurity/assistance/options, housing impact, and unmet needs of older individuals living with HIV in North America from 2012 to 2023. Furthermore, Latent Semantic Analysis (LSA), a text-mining technique, and Singular Value Decomposition (SVD) for text clustering were utilized to examine unstructured data from the abstracts selected from the review. The goal was to allow for a better understanding of the relationships between terms in the articles and the identification of emerging public health key themes affecting older adults living with HIV. The results of text clustering yielded two clusters focusing on (1) improvements to housing and healthcare services access and policies and (2) unmet needs-social support, mental health, finance, food, and sexuality insecurities. Topic modeling demonstrated four topics, which we themed to represent (1) a holistic care approach; (2) insecurities-food, financial, sexuality, and other basic needs; (3) access to housing and treatment/care; and (4) homelessness and HIV-related health outcomes. Stable housing, food, and healthcare services access and availability are critical elements to incorporating comprehensive, holistic healthcare for older adults living with HIV. The aging population requires high-priority policies for accessible and equitable healthcare. Clinicians and policymakers should address individual barriers, adopt a patient-centered approach, increase doctor visits, provide competency training, ensure long-term follow-up, involve families, and improve patient education in care management, contributing to HIV/AIDS geriatric care models.

CrisisTransformers: Pre-trained language models and sentence encoders for crisis-related social media texts

Social media platforms play an essential role in crisis communication, but analysing crisis-related social media texts is challenging due to their informal nature. Transformer-based pre-trained models like BERT and RoBERTa have shown success in various NLP tasks, but they are not tailored for crisis-related texts. Furthermore, general-purpose sentence encoders are used to generate sentence embeddings, regardless of the textual complexities in crisis-related texts. Advances in applications like text classification, semantic search, and clustering contribute to the effective processing of crisis-related texts, which is essential for emergency responders to gain a comprehensive view of a crisis event, whether historical or real-time. To address these gaps in crisis informatics literature, this study introduces CrisisTransformers, an ensemble of pre-trained language models and sentence encoders trained on an extensive corpus of over 15 billion word tokens from tweets associated with more than 30 crisis events, including disease outbreaks, natural disasters, conflicts, and other critical incidents. We evaluate existing models and CrisisTransformers on 18 crisis-specific public datasets. Our pre-trained models outperform strong baselines across all datasets in classification tasks, and our best-performing sentence encoder improves the state-of-the-art by 17.43% in sentence encoding tasks. Additionally, we investigate the impact of model initialization on convergence and evaluate the significance of domain-specific models in generating semantically meaningful sentence embeddings. The models are publicly available at: https://huggingface.co/crisistransformers.

Hybrid topic modeling method based on dirichlet multinomial mixture and fuzzy match algorithm for short text clustering

Topic modeling methods proved to be effective for inferring latent topics from short texts. Dealing with short texts is challenging yet helpful for many real-world applications, due to the sparse terms in the text and the high dimensionality representation. Most of the topic modeling methods require the number of topics to be defined earlier. Similarly, methods based on Dirichlet Multinomial Mixture (DMM) involve the maximum possible number of topics before execution which is hard to determine due to topic uncertainty, and many noises exist in the dataset. Hence, a new approach called the Topic Clustering algorithm based on Levenshtein Distance (TCLD) is introduced in this paper, TCLD combines DMM models and the Fuzzy matching algorithm to address two key challenges in topic modeling: (a) The outlier problem in topic modeling methods. (b) The problem of determining the optimal number of topics. TCLD uses the initial clustered topics generated by DMM models and then evaluates the semantic relationships between documents using Levenshtein Distance. Subsequently, it determines whether to keep the document in the same cluster, relocate it to another cluster, or mark it as an outlier. The results demonstrate the efficiency of the proposed approach across six English benchmark datasets, in comparison to seven topic modeling approaches, with 83% improvement in purity and 67% enhancement in Normalized Mutual Information (NMI) across all datasets. The proposed method was also applied to a collected Arabic tweet and the results showed that only 12% of the Arabic short texts were incorrectly clustered, according to human inspection.

Innovative Cultural Transmission Paths Analysis for Local Opera Based on Data Mining

With the rapid development of society and cultural diversity, local opera, as a representative of traditional art, faces dual challenges of inheritance and innovation. The inheritance of traditional Chinese opera often relies on oral instruction, but in modern society, this method is no longer suitable for the rapidly changing cultural environment. The purpose of this study is to explore in depth the innovative cultural dissemination path of local opera through data mining techniques. This paper uses data mining techniques to explore and analyze the inheritance of innovative culture in local opera, and to study the inheritance paths. This paper proposes a data mining-based method for analyzing and identifying the paths of inheritance of innovative culture in local opera. This paper focuses on two aspects of technology clustering and technology theme association, and the research paths are divided into flexible clustering based on data enhancement and technology evolution path identification based on theme extraction. Firstly, the heritage changes of dating tunes are discussed. Secondly, the unique artistic esthetics of the dating tune is compared with the artistic characteristics of Henan opera and the three-stringed book. Through the research and learning experience of the Dai Yongqu, it is easier to thoroughly understand the artistic essence and cultural connotation of the Dai Yongqu, and that exposure to the Dai Yongqu is a process of touching traditional culture. In the text clustering section, the data co-occurrence frequency matrix is constructed based on the results of multiple clustering, transforming the problem of determining the number of clusters into a co-occurrence frequency problem with semantic information of the text, thus achieving the purpose of flexible clustering on demand. In a comparison with existing methods, the text representation method using data-enhanced coding achieves better semantic coding results. The flexible clustering method can differentiate the data better than the clustering method with a defined number of clusters and obtains clustering results with high intra-class similarity and high inter-class differentiation.

Lp-slam: language-perceptive RGB-D SLAM framework exploiting large language model

With the development of deep learning, a higher level of perception of the environment such as the semantic level can be achieved in the simultaneous localization and mapping (SLAM) domain. However, previous works did not achieve a natural-language level of perception. Therefore, LP-SLAM (Language-Perceptive RGB-D SLAM) is proposed that leverages large language models (LLMs). The texts in the scene can be detected by scene text recognition (STR) and mapped as landmarks with a task-driven selection. A text error correction chain (TECC) is designed with a similarity classification method, a two-stage memory strategy, and a text clustering method. The proposed architecture is designed to deal with the mis-detection and mis-recognition cases of STR and to provide accurate text information to the framework. The proposed framework takes input images and generates a 3D map with sparse point cloud and task-related texts. Finally, a natural user interface (NUI) is designed based on the constructed map and LLM, which gives position instructions based on users’ natural queries. The experimental results validated the proposed TECC design and the overall framework. We publish the virtual dataset with ground truth, as well as the source code for further research. https://github.com/GroupOfLPSLAM/LP_SLAM.

A Graph-Based Keyword Extraction Method for Academic Literature Knowledge Graph Construction

In this paper, we construct an academic literature knowledge graph based on the relationship between documents to facilitate the storage and research of academic literature data. Keywords are an important type of node in the knowledge graph. To solve the problem that there are no keywords in some documents for several reasons in the process of knowledge graph construction, an improved keyword extraction algorithm called TP-CoGlo-TextRank is proposed by using word frequency, position, word co-occurrence frequency, and a word embedding model. By combining the word frequency and position in the document, the importance of words is distinguished. By introducing the GloVe word-embedding model, which brings the external knowledge of documents into the TextRank algorithm, and combining the internal word co-occurrence frequency in the documents, the word-adjacency relationship is transferred non-uniformly. Finally, the words with the highest scores are combined into phrases if they are adjacent in the original text. The validity of the TP-CoGlo-TextRank algorithm is verified by experiments. On this basis, the Neo4j graph database is used to store and display the academic literature knowledge graph, to provide data support for research tasks such as text clustering, automatic summarization, and question-answering systems.

Driving patterns in connected environments: A case study of intersection-approaching behavior of professional and non-professional drivers

The in-vehicle communication provides promising opportunities to improve the road safety and traffic efficiency. Previous studies demonstrated that the professional drivers have better driving skills than the non-professional drivers who allocate more attention to secondary tasks. However, they may not be sensitive to the new in-vehicle technology. In addition, these qualitative studies failed to elaborate on the visual and response behavior differences among different driver groups (professional drivers such as taxi, bus, motorcoach, and non-professional drivers), and lacked the quantitative analysis of driving patterns in a new environment. This paper explores the differences in visual interaction, response characteristics, driving performance, and behavior patterns between the professional and non-professional drivers in the connected environment through a case study of intersection-approaching behavior using a driving simulator. More precisely, two driving scenarios (baseline and human–machine interface (HMI)) were designed in the driving simulator, and 65 participants, including 34 professional drivers and 31 non-professional drivers, completed the experiment. In the HMI scenario, the driver was provided with the signal light phase and phase transition remaining time of the current intersection. This paper also proposes a driving pattern extraction model based on the Bayesian non-parametric method combined with a text clustering algorithm to perform a quantitative description of the driving patterns. The results show that the professional drivers tend to interact less with the HMI compared with the non-professional drivers. Moreover, the professional drivers’ first gaze at the HMI occurs and responds earlier. The proposed driving model can effectively describe 7 patterns of intersection-approaching behavior. The connected information can significantly improve the efficiency of the intersection traffic and the driving behavior. However, the professional drivers are more responsive and behave more consistently. This study can provide insights into the development of personalized assisted driving systems, as the two driving populations differ in their interactions, responses, and behavioral patterns.

ВІД АНАЛІЗУ ТЕКСТУ ДО МОДЕЛЮВАННЯ ПРИРОДНОЇ МОВИ: КОМПЛЕКСНЕ ДОСЛІДЖЕННЯ

The article presents a comprehensive analysis of modern techniques and technologies used to analyze the tonality of textual data. Through a series of experiments, basic machine and deep learning algorithms such as sentiment analysis were developed to automatically detect and classify emotional tones in texts. Practical applications of these methods are also explored, ranging from social network monitoring to expert analysis and analysis of user reviews. Next, the article delves into the classification of textual data using modern machine and deep learning methods. This section of the paper reviews several classification models, including the Bayesian most-narrow classifier, support vector methods, and neural networks, highlighting their advantages and limitations. In addition, it emphasizes the importance of implementing text classification in various fields, including social network analysis, news article categorization, and automated document processing. In addition, clustering of similar textual data is considered for further analysis. Various clustering algorithms, such as k-means, hierarchical clustering, and spectral clustering, are compared, with special emphasis on their application to large text corpora. A demonstration of the practical application of text clustering, including data organization, topic search, and style identification in paper works, was also demonstrated. Moving on, the article considers the processing of individual thematic structures in textual data and its further analysis. An in-depth analysis of topic modeling techniques, such as the realized Dirichlet distribution (LDA) model, as well as its capabilities and limitations, is explored. Practical applications of topic modeling are demonstrated in various ways, including text collection analysis, news trend detection, and automatic document categorization. Finally, the article discusses some challenges and future prospects for the development of thematic modeling.

Deep Learning-Driven Public Opinion Analysis on the Weibo Topic about AI Art

The emergence of AI Art has ignited extensive debates on social media platforms. Various online communities have expressed their opinions on different facets of AI Art and participated in discussions with other users, leading to the generation of a substantial volume of data. Analyzing these data can provide useful insights into the public’s opinions on AI Art, enable the investigation of the origins of conflicts in online debates, and contribute to the sustainable development of AI Art. This paper presents a deep learning-driven framework for analyzing the characteristics of public opinion on the Weibo topic of AI Art. To classify the sentiments users expressed in Weibo posts, the linguistic feature-enhanced pre-training model (LERT) was employed to improve text representation via the fusion of syntactic features, followed by a bidirectional Simple Recurrent Unit (SRU) embedded with a soft attention module (BiSRU++) for capturing the long-range dependencies in text features, thus improving the sentiment classification performance. Furthermore, a text clustering analysis was performed across sentiments to capture the nuanced opinions expressed by Weibo users, hence providing useful insights about different online communities. The results indicate that the proposed sentiment analysis model outperforms common baseline models in terms of classification metrics and time efficiency, and the clustering analysis has provided valuable insights for in-depth analyses of AI Art.

Grounded Theory Analysis of Online Shopping Reviews Based on Text Clustering

Grounded Theory Analysis of online shopping reviews can improve the quality and accuracy of review data analysis, providing valuable information and suggestions for relevant businesses and consumers. This article focuses on analyzing mobile phone reviews on JD.com as coding objects, effectively employing machine learning algorithms such as Latent Semantic Indexing (LSI), Latent Dirichlet Allocation (LDA), and text clustering in natural language processing to extract thematic keywords from online shopping review texts and perform axial coding. Subsequently, DBSCAN clustering algorithm is applied to categorize the main categories of design, service, and production under the perspective of businesses and consumers as consumption attraction, while categorizing price and brand under consumption intention, forming selective coding. By applying Grounded Theory to the analysis of online shopping reviews, this study accurately identifies consumer needs and feedback, with the aim of helping businesses improve product and service quality, enhance customer satisfaction and loyalty, and engage in personalized marketing and customized services.

Text Classification and Cluster Analysis based on Deep Learning and Natural Language Processing

At present, the commonly used Bag of Words (BOW) expression ignores the semantic information of text and the problems of high dimension and high sparsity of feature extraction. This paper presents a multi-class text representation and classification algorithm. This project is based on the vector expression of keywords and takes the multi-category classification problem as the research object. Then, a hybrid Deep Location network (HDBN) is constructed by combining DBN with Boltzmann (DBM). Then, this paper does a lot of tests on the algorithm and proves the effectiveness of the algorithm. In addition, the 2D visual experiment is carried out with HDBN, and then the high-level text expression based on HDBN is obtained. The expression has strong cohesion and weak coupling.

A comprehensive and analytical review of text clustering techniques

A comprehensive and analytical review of text clustering techniques

Large Language Models Enable Few-Shot Clustering

Abstract Unlike traditional unsupervised clustering, semi-supervised clustering allows users to provide meaningful structure to the data, which helps the clustering algorithm to match the user’s intent. Existing approaches to semi-supervised clustering require a significant amount of feedback from an expert to improve the clusters. In this paper, we ask whether a large language model (LLM) can amplify an expert’s guidance to enable query-efficient, few-shot semi-supervised text clustering. We show that LLMs are surprisingly effective at improving clustering. We explore three stages where LLMs can be incorporated into clustering: before clustering (improving input features), during clustering (by providing constraints to the clusterer), and after clustering (using LLMs post-correction). We find that incorporating LLMs in the first two stages routinely provides significant improvements in cluster quality, and that LLMs enable a user to make trade-offs between cost and accuracy to produce desired clusters. We release our code and LLM prompts for the public to use.1

Unsupervised text feature selection by binary fire hawk optimizer for text clustering

Unsupervised text feature selection by binary fire hawk optimizer for text clustering

MICRank: Multi-information interconstrained keyphrase extraction

Keyphrase Extraction is an automatic task that involves identifying the key words or phrases that capture the main content of an article. It is useful for various downstream tasks, including text search, text clustering, and text classification. Embedding-based methods for keyphrase extraction have shown promising results by utilizing pre-trained language models to represent candidate phrases and documents separately. These methods then rank the candidate phrases based on the cosine similarity between the document and the candidate phrases embeddings. However, there are mainly two shortcomings in such methods: I) Redundancy errors, when there are partial repetitions of candidate keyphrases, the methods tend to use redundant long phrases as keyphrases; II) Low keyphrase coverage, such as some keyphrases used to describe locally important information are ignored. In this paper, we propose an unsupervised keyphrase extraction method called “MICRank”, which evaluates the importance of candidate keyphrases from three perspectives: global information, local information, and attribute information, and solved the aforementioned issues. The experimental results on six benchmarks demonstrate that the proposed MICRank method outperforms the state-of-the-art unsupervised keyphrase extraction methods. In addition, this paper improves the judgment criterion of correct keyphrase extraction and introduces a new evaluation metric called S1@M (M ∈ {5,10,15}) to address the issue of synonyms being considered incorrect predictions.

Beyond content: discriminatory power of function words in text type classification

Abstract Our work aims to evaluate the strength of the association between function words and several text types: novels, poems, academic articles, reviews, and blog posts, and the accuracy of their classification to these categories, through machine-learning and statistical methods. The principal conclusion is that the types of texts are distinguishable based only on the function words, either by vocabulary or vocabulary diversity. Such findings may impact the techniques of authorship attribution based on function words and text clustering techniques since some function words add information about the text types/genres, in addition to content words.

Enhancing accident cause analysis through text classification and accident causation theory: A case study of coal mine gas explosion accidents

Accident analysis is a crucial aspect of accident prevention, and natural language processing (NLP) techniques can efficiently be applied to analyze the causes of accidents. However, existing analysis methods primarily rely on text clustering and lack the application of accident causation theories, leading to a lack of specific accident cause analysis results. Combining text classification techniques with accident causation theories is an effective approach to address this issue. In this study, we integrated text classification techniques with accident causation theories and utilized coal mine gas explosion accidents as an example. We constructed a corpus, trained a BERT model, and evaluated its performance to obtain a text classification model for accident cause analysis. The results indicated that the BERT model-based text classification algorithm had an accuracy and macro-average F1 value of 0.9878 and 0.7792, respectively, significantly outperforming the control model. The application of this approach demonstrated that combining accident causation theories with text classification techniques for accident cause analysis can improve the efficiency of accident analysis while ensuring the richness of details in analyzing accident causes. By efficiently analyzing a large number of accident cases, this approach can provide a data foundation for data-driven accident prevention and technical support for integrated accident prevention.

Predicting likelihood of fraud among financial distressed firms in Malaysia using textual analysis

This research paper aims to analyze and predict fraud patterns among failed companies in Malaysia. The approach involves utilizing textual analysis on the management discussion and analysis (MD&A) section within the annual reports. The dataset is subjected to text clustering to group companies based on similar financial characteristics. This clustering process entails several steps, including data conversion, collation, and summarization into a structured format, followed by text pre-processing to cleanse the dataset. Notably, RapidMiner Studio software was utilized to extract data for the study. Subsequently, the documents are clustered using both the K-means and latent dirichlet allocation (LDA) methods. Upon examining a sample of 22 failed companies in the year 2020, the study reveals that financially distressed companies exhibit prominent financial negativity and utilize litigious financial terms within their MD&A sections. These linguistic traits are found to be closely associated with seven distinct characteristics of fraudulent firms. This preliminary findings provide compelling evidence that financial pressure may serve as a triggering factor for fraudulent activities within companies.