Text Topic Modeling Research Articles

Applying short text topic models to instant messaging communication of software developers

When modeling topics from chat messages of developer instant messaging communication, individual chat messages are short text documents. Our study aims at understanding how short text topic models perform with conversations from developer instant messaging. We applied four models to nine Gitter chat rooms (with sizes ranging from ≈100 to ≈160,000 messages). To assess the quality of topics and identify the best performing models, we compared topics based on four metrics for topic coherence. Furthermore, for a subset of Gitter chat rooms we used two human-based assessments: intrusion tasks with 18 experts analyzing 40 topics each, and topic naming (assigning a name to a topic that summarizes its main concept) with eight additional experts naming 60 topics each. Models performed differently in terms of coherence metrics and human assessment depending on the corpus (small, medium or large chat room). Our findings offer recommendations for the selection and use of short text topic models with developer chat messages based on characteristics of models and their performance with different sizes of corpora, and based on different strategies to assess topic quality.

A semi-supervised approach of short text topic modeling using embedded fuzzy clustering for Twitter hashtag recommendation

Social media stands as a crucial information source across various real-world challenges. Platforms like Twitter, extensively used by news outlets for real-time updates, categorize news via hashtags. These hashtags act as pivotal meta-information for linking tweets to underlying themes, yet many tweets lack them, posing challenges in topic searches. Our contribution addresses this by introducing a novel heuristic for hashtag recommendation. Extracting 20 thousand tweets, 5000 each from distinct categories health, sports, politics, and technology we applied fundamental data cleaning and tokenization techniques. Leveraging Word2Vec, we vectorized tokens, capturing nuanced semantic meanings and mitigating data sparsity issues. The proposed heuristic creates clusters of different topic by combining these embedded features and idea of fuzzy C-Means technique. Develop a rule-based approach that combines both supervised and unsupervised methods to label clusters, indicating their respective topic. The experimental outcomes shows that our proposed techniques achieve better performance metrics in precision, recall, and F1-score compared to specific baseline models.

Short text classification using semantically enriched topic model

Modelling short text is challenging due to the small number of word co-occurrence and insufficient semantic information that affects downstream Natural Language Processing (NLP) tasks, for example, text classification. Gathering information from external sources is expensive and may increase noise. For efficient short text classification without depending on external knowledge sources, we propose Expressive Short text Classification (EStC). EStC consists of a novel document context-aware semantically enriched topic model called the Short text Topic Model (StTM) that captures words, topics and documents semantics in a joint learning framework. In StTM, the probability of predicting a context word involves the topic distribution of word embeddings and the document vector as the global context, which obtains by weighted averaging of word embeddings on the fly simultaneously with the topic distribution of words without requiring an additional inference method for the document embedding. EStC represents documents in an expressive (number of topics × number of word embedding features) embedding space and uses a linear support vector machine (SVM) classifier for their classification. Experimental results demonstrate that EStC outperforms many state-of-the-art language models in short text classification using several publicly available short text data sets.

Application of Latent Semantic Analysis in Accounting Research

ABSTRACT The purpose of this study is to review a text topic modeling methodology, latent semantic analysis (LSA), and provide researchers with the requisite knowledge to allow them to learn and implement their own accounting research study using LSA. The authors first provide a brief literature review of prior business and accounting research studies that have utilized the LSA methodology. Using a provided dataset, the authors present details of how to employ LSA in a research study by replicating the mechanics used in an LSA study conducted by Hutchison, Plummer, and George (2018b). Their intent is to present thorough guidance on data selection, the analysis platform, and the necessary steps needed to conduct LSA research. This article also briefly compares LSA with other topic modeling methodologies, presents several accounting research opportunities where LSA could be utilized, and outlines LSA’s limitations. Data Availability: Data are available from the public sources cited in the text; sample dataset is available for download, see footnote 5.

Topic modeling methods for short texts: A survey

In the present day, online users are incentivized to engage in short text-based communication. These short texts harbor a significant amount of implicit information, including opinions, topics, and emotions, which are of notable value for both exploration and analysis. By alleviating the sparsity in short texts, topic models can be used to discover topics from large collections of short texts. While there is a large body of surveys focused on topic modeling, but only a few of them have focused on the short texts. This paper presents a comprehensive overview of topic modeling methods for short texts from a novel perspective. Firstly, it discusses short text probabilistic topic models and outlines the directions in which they can be improved. Secondly, it explores short text neural topic models, which can be categorized into three groups based on their underlying structures. In addition, this paper provides a detailed investigation of embedding methods in topic modeling. Moreover, various applications and corresponding works are surveyed, with a focus on short texts. The commonly used public corpora and evaluation indicators for topic modeling are also summarized. Finally, the advantages and disadvantages of short text topic modeling are discussed in detail, and future research directions are proposed.

Mapping the sociodemographic distribution and self-reported justifications for non-compliance with COVID-19 guidelines in the United Kingdom.

Which population factors have predisposed people to disregard government safety guidelines during the COVID-19 pandemic and what justifications do they give for this non-compliance? To address these questions, we analyse fixed-choice and free-text responses to survey questions about compliance and government handling of the pandemic, collected from tens of thousands of members of the UK public at three 6-monthly timepoints. We report that sceptical opinions about the government and mainstream-media narrative, especially as pertaining to justification for guidelines, significantly predict non-compliance. However, free text topic modelling shows that such opinions are diverse, spanning from scepticism about government competence and self-interest to full-blown conspiracy theories, and covary in prevalence with sociodemographic variables. These results indicate that attempts to counter non-compliance through argument should account for this diversity in peoples' underlying opinions, and inform conversations aimed at bridging the gap between the general public and bodies of authority accordingly.

A systematic review of the use of topic models for short text social media analysis

Recently, research on short text topic models has addressed the challenges of social media datasets. These models are typically evaluated using automated measures. However, recent work suggests that these evaluation measures do not inform whether the topics produced can yield meaningful insights for those examining social media data. Efforts to address this issue, including gauging the alignment between automated and human evaluation tasks, are hampered by a lack of knowledge about how researchers use topic models. Further problems could arise if researchers do not construct topic models optimally or use them in a way that exceeds the models’ limitations. These scenarios threaten the validity of topic model development and the insights produced by researchers employing topic modelling as a methodology. However, there is currently a lack of information about how and why topic models are used in applied research. As such, we performed a systematic literature review of 189 articles where topic modelling was used for social media analysis to understand how and why topic models are used for social media analysis. Our results suggest that the development of topic models is not aligned with the needs of those who use them for social media analysis. We have found that researchers use topic models sub-optimally. There is a lack of methodological support for researchers to build and interpret topics. We offer a set of recommendations for topic model researchers to address these problems and bridge the gap between development and applied research on short text topic models.

Short text topic modelling using local and global word-context semantic correlation.

Nowadays, people use short text to portray their opinions on platforms of social media such as Twitter, Facebook, and YouTube, as well as on e-commerce websites such as Amazon and Flipkart to share their commercial purchasing experiences. Every day, billions of short texts are created worldwide in tweets, tags, keywords, search queries etc. However, this short text possesses inadequate contextual information, which can be ambiguous, sparse, noisy, remains a major challenge. State-of-the-art strategies of topic modeling such as Latent Dirichlet Allocation and Probabilistic Latent Semantic Analysis are not suitable as it contains a limited number of words in a single document. This work proposes a new model named G_SeaNMF (Gensim_SeaNMF) to improve the word-context semantic relationship by using local and global word embedding techniques. Word embeddings learned from a large corpus provide general semantic and syntactic information about words; it can guide topic modeling for short text collections as supporting information for sparse co-occurrence patterns. In the proposed model, SeaNMF (Semantics-assisted Non-negative Matrix Factorization) is incorporated with word2vec model of Gensim library to strengthen the word's semantic relationship. In this article, a short text topic modeling techniques based on DMM (Dirichlet Multinomial Mixture), self-aggregation and global word co-occurrence were explored. These are evaluated using different measures to gauge cluster coherence on real-world datasets such as Search Snippet, Biomedicine, Pascal Flickr, Tweet and TagMyNews. Empirical evaluation shows that a combination of local and global word embedding provides more appropriate words under each topic with improved outcomes.

Development of domain-specific automatic speech recognition models based on open-source data

Currently, deep neural network architectures are used in the development of automatic speech recognition models for specialized subject areas, which require a large volume of training data. However, these models often prove to be poorly suitable for use in specific information systems due to the inadequate recognition of specialized subject vocabularies. Further training of models to improve their quality in a specific context of recognition encounters difficulties in obtaining sufficient data and the laboriousness of their labeling. Given this, a pertinent task is the creation of methods that allow for a reduction in the effort required to build applied speech recognition models and improve their quality when used in various subject areas. The study used language model-based text topic modeling techniques to adapt open-source data. A deep neural network was used as the pre-trained speech recognition model. For training, data sets from open sources were utilized. A method has been developed for creating automatic speech recognition models for specialized subject areas. This method involves an intermediate step of learning the lexicon of the subject area from data obtained from open sources, selected using thematic sampling. Based on this method, automatic speech recognition models for the energy and healthcare sectors were created and researched, demonstrating higher recognition results than models developed through traditional methods. The validation of the proposed method confirmed its effectiveness. The applied neural network models, developed based on this method, demonstrated their ability to operate in information systems of energy and healthcare facilities in Russian, English, and Italian languages without additional training on closed data.

Short text topic modelling approaches in the context of big data: taxonomy, survey, and analysis.

Social media platforms such as (Twitter, Facebook, and Weibo) are being increasingly embraced by individuals, groups, and organizations as a valuable source of information. This social media generated information comes in the form of tweets or posts, and normally characterized as short text, huge, sparse, and low density. Since many real-world applications need semantic interpretation of such short texts, research in Short Text Topic Modeling (STTM) has recently gained a lot of interest to reveal unique and cohesive latent topics. This article examines the current state of the art in STTM algorithms. It presents a comprehensive survey and taxonomy of STTM algorithms for short text topic modelling. The article also includes a qualitative and quantitative study of the STTM algorithms, as well as analyses of the various strengths and drawbacks of STTM techniques. Moreover, a comparative analysis of the topic quality and performance of representative STTM models is presented. The performance evaluation is conducted on two real-world Twitter datasets: the Real-World Pandemic Twitter (RW-Pand-Twitter) dataset and Real-world Cyberbullying Twitter (RW-CB-Twitter) dataset in terms of several metrics such as topic coherence, purity, NMI, and accuracy. Finally, the open challenges and future research directions in this promising field are discussed to highlight the trends of research in STTM. The work presented in this paper is useful for researchers interested in learning state-of-the-art short text topic modelling and researchers focusing on developing new algorithms for short text topic modelling.

Analysis of social economy research trends in Korea: Comparative analysis of keywords using text network & topic analysis

The purpose of this study is to examine the domestic research trends of social economy and to suggest the direction of future research using text networks and topic modeling analysis techniques. In this study, a total of 3,500 academic papers suitable for the research purpose registered in the Korean Journal Yongin Index(KCI) were set as analysis targets, and core keywords and research topics were compared and analyzed by major period.. As a result of the study, the words that appear in common across time were ‘Responsibility’, ‘Region’, ‘Entrepreneur’, ‘Service’, and ‘Consumer’. Since 2007, new words such as ‘capital’, ‘government’, ‘policy’, ‘system’, ‘management’, and ‘brand’ have emerged in common, which can be interpreted as the result of revitalization of related research as social economy-related legislation progresses.. This study has theoretical implications in that it deepened the knowledge base on domestic social economy research trends by analyzing vast amounts of data processing using text mining techniques.

Tasting the Port wine cellar experience: what features please the most?

ABSTRACT This study addresses wine tasting experienced by tourists in the context of the worldwide renowned Port wine. A total of 824 online reviews posted about four major Port wine cellars in TripAdvisor were collected. Data were analyzed with text mining and topic modeling, and sentiment scores, as well as ratings, were also considered. Overall, results showed that guests were primarily attracted by the experience as a whole, including the tour and the wine culture and not just the wine tasting experience. Furthermore, six topics were found with associated main keywords and comprehending distinct groups of guests with different preferences. Three groups underlined tour and visiting experience, whereas two other groups were composed of enthusiastic wine fans, with one more focused on the wine/winery/cellar experience while the other enjoyed a diversified wine selection. Yet another group aggregated tourists who valued the food experience.

Communication patterns in engineering enterprise social networks: an exploratory analysis using short text topic modelling

Abstract Enterprise social network messaging sites are becoming increasingly popular for team communication in engineering and product design. These digital communication platforms capture detailed messages between members of the design team and are an appealing data set for researchers who seek to better understand communication in design. This exploratory study investigates whether we can use enterprise social network messages to model communication patterns throughout the product design process. We apply short text topic modelling (STTM) to a data set comprising 250,000 messages sent by 32 teams enrolled in a 3-month intensive product design course. Many researchers describe the engineering design process as a series of convergent and divergent thinking stages, such as the popular double diamond model, and we use this theory as a case study in this work. Quantitative and qualitative analysis of STTM results reveals several trends, such as it is indeed possible to see evidence of cyclical convergence and divergence of topics in team communication; within the convergence–divergence pattern, strong teams have fewer topics in their topic models than weaker teams; and teams show characteristics of product, project, course, and other themes within each topic. We provide evidence that the analysis of enterprise social networking messages, with advanced topic modelling techniques, can uncover insights into design processes and can identify the communication patterns of successful teams.

Identifying the Knowledge Structure and Trends of Outreach in Public Health Care: A Text Network Analysis and Topic Modeling.

Outreach programs are considered a key strategy for providing services to underserved populations and play a central role in delivering health-care services. To address this challenge, knowledge relevant to global health outreach programs has recently been expanded. The aims of this study were to analyze the knowledge structure and understand the trends in aspects over time and across regions using text network analysis with NetMiner 4.0. Data analysis by frequency, time and region showed that the central keywords such as patient, care, service and community were found to be highly related to the area, target population, purpose and type of services within the knowledge structure of outreach. As a result of performing topic modeling, knowledge structure in this area consisted of five topics: patient-centered care, HIV care continuum, services related to a specific disease, community-based health-care services and research and education on health programs. Our results newly identified that patient-centered care, specific disease and population have been growing more crucial for all times and countries by the examination of major trends in health-care related outreach research. These findings help health professionals, researchers and policymakers in nursing and public health fields in understanding and developing health-care-related outreach practices and suggest future research direction.

Topic extraction from extremely short texts with variational manifold regularization

With the emerging of massive short texts, e.g., social media posts and question titles from Q&A systems, discovering valuable information from them is increasingly significant for many real-world applications of content analysis. The family of topic modeling can effectively explore the hidden structures of documents through the assumptions of latent topics. However, due to the sparseness of short texts, the existing topic models, e.g., latent Dirichlet allocation, lose effectiveness on them. To this end, an effective solution, namely Dirichlet multinomial mixture (DMM), supposing that each short text is only associated with a single topic, indirectly enriches document-level word co-occurrences. However, DMM is sensitive to noisy words, where it often learns inaccurate topic representations at the document level. To address this problem, we extend DMM to a novel Laplacian Dirichlet Multinomial Mixture (LapDMM) topic model for short texts. The basic idea of LapDMM is to preserve local neighborhood structures of short texts, enabling to spread topical signals among neighboring documents, so as to modify the inaccurate topic representations. This is achieved by incorporating the variational manifold regularization into the variational objective of DMM, constraining the close short texts with similar variational topic representations. To find nearest neighbors of short texts, before model inference, we construct an offline document graph, where the distances of short texts can be computed by the word mover’s distance. We further develop an online version of LapDMM, namely Online LapDMM, to achieve inference speedup on massive short texts. Carrying this implications, we exploit the spirit of stochastic optimization with mini-batches and an up-to-date document graph that can efficiently find approximate nearest neighbors instead. To evaluate our models, we compare against the state-of-the-art short text topic models on several traditional tasks, i.e., topic quality, document clustering and classification. The empirical results demonstrate that our models achieve very significant performance gains over the baseline models.

Improving Semantic Coherence of Gujarati Text Topic Model Using Inflectional Forms Reduction and Single-letter Words Removal

A topic model is one of the best stochastic models for summarizing an extensive collection of text. It has accomplished an inordinate achievement in text analysis as well as text summarization. It can be employed to the set of documents that are represented as a bag-of-words, without considering grammar and order of the words. We modeled the topics for Gujarati news articles corpus. As the Gujarati language has a diverse morphological structure and inflectionally rich, Gujarati text processing finds more complexity. The size of the vocabulary plays an important role in the inference process and quality of topics. As the vocabulary size increases, the inference process becomes slower and topic semantic coherence decreases. If the vocabulary size is diminished, then the topic inference process can be accelerated. It may also improve the quality of topics. In this work, the list of suffixes has been prepared that encounters too frequently with words in Gujarati text. The inflectional forms have been reduced to the root words concerning the suffixes in the list. Moreover, Gujarati single-letter words have been eliminated for faster inference and better quality of topics. Experimentally, it has been proved that if inflectional forms are reduced to their root words, then vocabulary length is shrunk to a significant extent. It also caused the topic formation process quicker. Moreover, the inflectional forms reduction and single-letter word removal enhanced the interpretability of topics. The interpretability of topics has been assessed on semantic coherence, word length, and topic size. The experimental results showed improvements in the topical semantic coherence score. Also, the topic size grew notably as the number of tokens assigned to the topics increased.

Topic prediction and knowledge discovery based on integrated topic modeling and deep neural networks approaches

Understanding the real-world short texts become an essential task in the recent research area. The document deduction analysis and latent coherent topic named as the important aspect of this process. Latent Dirichlet Allocation (LDA) and Probabilistic Latent Semantic Analysis (PLSA) are suggested to model huge information and documents. This type of contexts’ main problem is the information limitation, words relationship, sparsity, and knowledge extraction. The knowledge discovery and machine learning techniques integrated with topic modeling were proposed to overcome this issue. The knowledge discovery was applied based on the hidden information extraction to increase the suitable dataset for further analysis. The integration of machine learning techniques, Artificial Neural Network (ANN) and Long Short-Term (LSTM) are applied to anticipate topic movements. LSTM layers are fed with latent topic distribution learned from the pre-trained Latent Dirichlet Allocation (LDA) model. We demonstrate general information from different techniques applied in short text topic modeling. We proposed three categories based on Dirichlet multinomial mixture, global word co-occurrences, and self-aggregation using representative design and analysis of all categories’ performance in different tasks. Finally, the proposed system evaluates with state-of-art methods on real-world datasets, comprises them with long document topic modeling algorithms, and creates a classification framework that considers further knowledge and represents it in the machine learning pipeline.

대통령 연설문을 통한 규제개혁 정책 경향에 관한 탐색적 연구 : 텍스트 마이닝과 토픽모델링의 활용

대통령 연설문을 통한 규제개혁 정책 경향에 관한 탐색적 연구 : 텍스트 마이닝과 토픽모델링의 활용

A systematic review to identify the effects of tea by integrating an intelligence-based hybrid text mining and topic model

An emerging problem regarding tea and health has been pyramidally addressed as research concerns from limited literature reviews to identify an important and interesting challenge. Although past studies activated for various topics and diverse purposes of tea and health, a gap for using hybrid intelligence-based techniques to discover useful information from literature analysis exists in the exploration of curative effects on tea against fatal diseases other than Western medicine therapies. This study is motivated to bridge this gap by solving this research issue for healthcare applications between tea and health. Thus, this study proposes a hybrid method of an intelligent/objective text mining technique and topic modeling principally by latent Dirichlet allocation with VEM method and Gibbs sampling as along with measurements for three evaluation metrics for model performance from published articles. In the experiment materials, this study sets conditions to collect 2109 journal articles from the Web of Science from 2010 to 2017. We divided this into three datasets that each corresponded to the three periods to differentiate discrepancies in future trends. This study contributes eight beneficial directions as follows: (1) From a technical view, the figure of VEM’s perplexity has a screen plot, but Gibbs sampling is smooth and good; and, interestingly, the greater the number of topics, the lower the perplexity is; (2) in empirical results, the terms for primary topics are tea and tea compounds, and secondary topics are associated with terms for issues regarding tea and health; (3) this study yields five research findings with key empirical evidence that tea has a natural and important preventive impact on treating diseases, especially cancer; (4) as to any managerial implications, early preventions and treatments by greater tea consumption as a valuable healthcare activity with medicinal purposes; (5) regarding the novelty of this research, this study fills the gap in a hybrid knowledge-based objective text mining and topic modeling technique than past researchers have as regards tea and health issues that were only based on traditional content analysis methods; (6) for this study’s strengths, it achieved manpower cost reductions and relative objectivity because the objective LDA method is rarely used for topic modeling when compared to past studies; (7) for the research significance, the proposed method benefits efficient analysis from massive amounts of extant data for exploring latent information, accelerates research processes quickly, improves understanding for new hypotheses, and identifies key questions for further research; and (8) for conclusive research importance, this study offers new rationales for medical application and discovers differentiations and gaps for studying research trends.

텍스트 마이닝과 토픽 모델링을 이용한 성조숙증 관련 인터넷 건강상담 분석

텍스트 마이닝과 토픽 모델링을 이용한 성조숙증 관련 인터넷 건강상담 분석