Topic-word Distribution Research Articles

Text mining of hypertension researches in the west Asia region: a 12-year trend analysis

More than half of the world population lives in Asia and hypertension (HTN) is the most prevalent risk factor found in Asia. There are numerous articles published about HTN in Eastern Mediterranean Region (EMRO) and artificial intelligence (AI) methods can analyze articles and extract top trends in each country. Present analysis uses Latent Dirichlet allocation (LDA) as an algorithm of topic modeling (TM) in text mining, to obtain subjective topic-word distribution from the 2790 studies over the EMRO. The period of checked studied is last 12 years and results of LDA analyses show that HTN researches published in EMRO discuss on changes in BP and the factors affecting it. Among the countries in the region, most of these articles are related to I.R Iran and Egypt, which have an increasing trend from 2017 to 2018 and reached the highest level in 2021. Meanwhile, Iraq and Lebanon have been conducting research since 2010. The EMRO word cloud illustrates ‘BMI’, ‘mortality’, ‘age’, and ‘meal’, which represent important indicators, dangerous outcomes of high BP, and gender of HTN patients in EMRO, respectively.

Discovering significant topics from legal decisions with selective inference.

We propose and evaluate an automated pipeline for discovering significant topics from legal decision texts by passing features synthesized with topic models through penalized regressions and post-selection significance tests. The method identifies case topics significantly correlated with outcomes, topic-word distributions which can be manually interpreted to gain insights about significant topics, and case-topic weights which can be used to identify representative cases for each topic. We demonstrate the method on a new dataset of domain name disputes and a canonical dataset of European Court of Human Rights violation cases. Topic models based on latent semantic analysis as well as language model embeddings are evaluated. We show that topics derived by the pipeline are consistent with legal doctrines in both areas and can be useful in other related legal analysis tasks. This article is part of the theme issue 'A complexity science approach to law and governance'.

Topic Models with Sentiment Priors Based on Distributed Representations

In recent works, topic models for aspect-based opinion mining have been extended to automatically train sentiment priors for topic-word distributions, leading to automated discovery of sentiment words and improved sentiment classification. In this work, we propose an approach where sentiment priors are trained in the space of word embeddings; this allows us to both discover more aspect-related sentiment words and further improve classification. We also present an experimental study that validates our results.

Tracking Brand-Associated Polarity-Bearing Topics in User Reviews

Abstract Monitoring online customer reviews is important for business organizations to measure customer satisfaction and better manage their reputations. In this paper, we propose a novel dynamic Brand-Topic Model (dBTM) which is able to automatically detect and track brand-associated sentiment scores and polarity-bearing topics from product reviews organized in temporally ordered time intervals. dBTM models the evolution of the latent brand polarity scores and the topic-word distributions over time by Gaussian state space models. It also incorporates a meta learning strategy to control the update of the topic-word distribution in each time interval in order to ensure smooth topic transitions and better brand score predictions. It has been evaluated on a dataset constructed from MakeupAlley reviews and a hotel review dataset. Experimental results show that dBTM outperforms a number of competitive baselines in brand ranking, achieving a good balance of topic coherence and uniqueness, and extracting well-separated polarity-bearing topics across time intervals.1

A Novel Framework to Detect Irrelevant Software Requirements Based on MultiPhiLDA as the Topic Model

Noise in requirements has been known to be a defect in software requirements specifications (SRS). Detecting defects at an early stage is crucial in the process of software development. Noise can be in the form of irrelevant requirements that are included within an SRS. A previous study had attempted to detect noise in SRS, in which noise was considered as an outlier. However, the resulting method only demonstrated a moderate reliability due to the overshadowing of unique actor words by unique action words in the topic–word distribution. In this study, we propose a framework to identify irrelevant requirements based on the MultiPhiLDA method. The proposed framework distinguishes the topic–word distribution of actor words and action words as two separate topic–word distributions with two multinomial probability functions. Weights are used to maintain a proportional contribution of actor and action words. We also explore the use of two outlier detection methods, namely percentile-based outlier detection (PBOD) and angle-based outlier detection (ABOD), to distinguish irrelevant requirements from relevant requirements. The experimental results show that the proposed framework was able to exhibit better performance than previous methods. Furthermore, the use of the combination of ABOD as the outlier detection method and topic coherence as the estimation approach to determine the optimal number of topics and iterations in the proposed framework outperformed the other combinations and obtained sensitivity, specificity, F1-score, and G-mean values of 0.59, 0.65, 0.62, and 0.62, respectively.

A semantic analysis-driven customer requirements mining method for product conceptual design

Precise customer requirements acquisition is the primary stage of product conceptual design, which plays a decisive role in product quality and innovation. However, existing customer requirements mining approaches pay attention to the offline or online customer comment feedback and there has been little quantitative analysis of customer requirements in the analogical reasoning environment. Latent and innovative customer requirements can be expressed by analogical inspiration distinctly. In response, this paper proposes a semantic analysis-driven customer requirements mining method for product conceptual design based on deep transfer learning and improved latent Dirichlet allocation (ILDA). Initially, an analogy-inspired verbal protocol analysis experiment is implemented to obtain detailed customer requirements descriptions of elevator. Then, full connection layers and a softmax layer are added to the output-end of Chinese bidirectional encoder representations from Transformers (BERT) pre-training language model. The above deep transfer model is utilized to realize the customer requirements classification among functional domain, behavioral domain and structural domain in the customer requirement descriptions of elevator by fine-tuning training. Moreover, the ILDA is adopted to mine the functional customer requirements that can represent customer intention maximally. Finally, an effective accuracy of customer requirements classification is acquired by using the BERT deep transfer model. Meanwhile, five kinds of customer requirements of elevator and corresponding keywords as well as their weight coefficients in the topic-word distribution are extracted. This work can provide a novel research perspective on customer requirements mining for product conceptual design through natural language processing.

Knowledge discovery using an enhanced latent Dirichlet allocation-based clustering method for solving on-site assembly problems

• We propose a text mining approach combining topic modeling, text clustering , and association rule identification algorithms for knowledge discovery. • We propose an enhanced latent Dirichlet allocation scheme to explore document-topic and topic-word distributions. • We introduce a refined DBSCAN method for text clustering. • The results demonstrate the capability and effectiveness of the proposed approach. Prompt responses to problems/faults arising in an assembly workshop are crucial in terms of production reliability and efficiency. However, human-dependent tasks are time-consuming and prone to error. In this paper, we propose a knowledge discovery approach. We extract the patterns of associations between texts in problem-solving records to generate appropriate solutions automatically. First, we use an enhanced latent Dirichlet allocation (EnLDA) technique to explore the document-topic and topic-word distributions of a text corpus recording assembly problems, causes, and solutions. To increase accuracy, we adjust the elements of the document-term matrix, and we assign term frequency-inverse document frequencies. Second, we use the Refining Density-based Spatial Clustering of Application with Noise (Rf-DBSCAN) algorithm for text clustering. This refines the distances among topic distribution vectors and incorporates noise objects into clustering. This clusters textual documents with similar semantic information, maximizing information retention. Third, we use the Apriori algorithm to identify pattern associations among document clusters that represent the problems, causes, and solutions. We perform a case study using field data from an automobile assembly workshop. The results show that the method retrieves hidden but valuable information from textual records. The decision support knowledge facilitates assembly problem-solving.

Discovering Computer Science Research Topic Trends using Latent Dirichlet Allocation

Before conducting a research project, researchers must find the trends and state of the art in their research field. However, that is not necessarily an easy job for researchers, partly due to the lack of specific tools to filter the required information by time range. This study aims to provide a solution to that problem by performing a topic modeling approach to the scraped data from Google Scholar between 2010 and 2019. We utilized Latent Dirichlet Allocation (LDA) combined with Term Frequency-Indexed Document Frequency (TF-IDF) to build topic models and employed the coherence score method to determine how many different topics there are for each year’s data. We also provided a visualization of the topic interpretation and word distribution for each topic as well as its relevance using word cloud and PyLDAvis. In the future, we expect to add more features to show the relevance and interconnections between each topic to make it even easier for researchers to use this tool in their research projects.

An Embedding-Based Topic Model for Document Classification

Topic modeling is an unsupervised learning task that discovers the hidden topics in a collection of documents. In turn, the discovered topics can be used for summarizing, organizing, and understanding the documents in the collection. Most of the existing techniques for topic modeling are derivatives of the Latent Dirichlet Allocation which uses a bag-of-word assumption for the documents. However, bag-of-words models completely dismiss the relationships between the words. For this reason, this article presents a two-stage algorithm for topic modelling that leverages word embeddings and word co-occurrence. In the first stage, we determine the topic-word distributions by soft-clustering a random set of embedded n -grams from the documents. In the second stage, we determine the document-topic distributions by sampling the topics of each document from the topic-word distributions. This approach leverages the distributional properties of word embeddings instead of using the bag-of-words assumption. Experimental results on various data sets from an Australian compensation organization show the remarkable comparative effectiveness of the proposed algorithm in a task of document classification.

Semantic-based topic representation using frequent semantic patterns

Topic modeling discovers the hidden topics in a document collection. Most of the existing topic models focus only on word usage and generate the topics based on the word frequency and co-occurrence without considering the meaning of the text. In this paper, we propose a novel approach to generate a semantic pattern-based topic representation based on the meaning of the text to represent the topics in a document collection. The proposed approach considers both the semantics and co-occurrence of words to generate a set of frequent semantic patterns to represent each topic. The semantics are captured by matching the words in each topic with concepts in the Probase ontology. A set of frequent semantic patterns in each topic is generated based on the co-occurrence of the matched words to represent the topic. Hence, our approach differs from traditional topic models because of the meaningful frequent semantic patterns generated based on the ontology. The proposed topic representation was evaluated in terms of topic quality and information filtering performance against a set of state-of-the-art systems. Perplexity, coherence, and topic word distribution were examined in the topic quality evaluation. The generated frequent semantic patterns were used as features for the information filtering evaluation. Our topic representation outperformed in all the evaluations.

Evaluating On-Line Courses via Reviews Mining

Because the participants are not limited by age-, gender-, race-, or geography-related barriers, recently, massive open online courses (MOOC) have witnessed remarkable growth in number of online self-learners, courses providers and online platforms. MOOC learners usually share some learning experiences and release millions of course-related comments in discussion forum. On the one hand, these comments could reflect the learners’ attitudes toward the online courses. On the other hand, semantic knowledge hidden in these comments would assist other learners to choose the appropriate courses and help instructors to improve their courses’ attraction. Recently, few research works focus on evaluating the courses through reviews mining. Thus, this paper constructs a curriculum evaluation system based on MOOC reviews, which quantifies the curriculum from different topics. Firstly, we employ latent dirichlet allocation (LDA) to mine the reviews generated by students, and obtain a topic-word distribution matrix and a comment-topic distribution matrix which can describe the topics of the course comments. Next, the emotion values of the comments in each topic are calculated by the auto-encoder and Bi-LSTM text classification model. We utilize the emotions and the quantified scores of the courses on different topics to establish a comprehensive curriculum evaluation system. The experimental results show that there are five main indicators abstracted from students’ reviews, which are instructor, course content, course assessment, MOOC platform, and hot courses. Moreover, comment texts of each course under different evaluation indicators are objectively and accurately converted into numerical marks, which can provide the students and educators with reliable references.

Monolingual and multilingual topic analysis using LDA and BERT embeddings

Analyzing research topics offers potential insights into the direction of scientific development. In particular, analyzing multilingual research topics can help researchers grasp the evolution of topics globally, revealing topic similarity among scientific publications written in different languages. Most studies to date on topic analysis have been based on English-language publications and have relied heavily on citation-based topic evolution analysis. However, since it can be challenging for English publications to cite non-English sources and since many languages do not offer English translations of abstracts, citation-based methodologies are not suitable for analyzing multilingual research topic relations. Since multilingual sentence embeddings can effectively preserve word semantics in multilingual translation tasks, a topic model based on multilingual sentence embeddings could potentially generate topic–word distributions for publications in multilingual analysis. In this paper, which is situated in the field of library and information science, we use multilingual pretrained Bidirectional Encoder Representations from Transformers (BERT) embeddings and the Latent Dirichlet Allocation (LDA) topic model to analyze topic evolution in monolingual and multilingual topic similarity settings. For each topic, we multiply its LDA probability value by the averaged tensor similarity of BERT embeddings to explore the evolution of the topic in scientific publications. As our proposed method does not rely on a machine translator or the author's subjective translation, it avoids confusion and misusages caused by either machine error or the author's subjectively chosen English keywords. Our results show that the proposed approach is well-suited to analyzing the scientific evolutions in monolingual and scientific multilingual topic similarity relations.

Topic representation model based on microblogging behavior analysis

With the development of microblogging, it has become an important way for people to obtain information, express opinions, and make suggestions. Identifying new topics quickly and accurately from the massive microblogging data plays a crucial role for recommending information and controlling public opinion. The topic representation model provides a basis for topic detection. In this paper, we propose a topic representation model based on user behavior analysis, i.e., microblogging behavior analysis-latent Dirichlet allocation (MBA-LDA) model, for microblogging datasets. Topic-word distribution is acquired by the LDA model which considers information on user behaviors (such as posting, forwarding and commenting) and word distribution among documents within one topic and among different topics. The model also re-assesses the importance of words in topic representation. The basic idea is that the distribution of words within a topic or among different topics has a great influence on the selection of topic expression words. If a word is evenly distributed among all documents of a certain topic, it indicates that the word is the common word of all documents in the topic, and it is more suitable to represent this topic. If a word is more evenly distributed among various topics, it indicates that the word is the common word of all topics, and it can’t achieve the purpose of distinguishing topics, so it is less suitable to represent any topic. By experiments with Sina Microblogging’s actual data set, the topic model based on the MBA-LDA algorithm makes the representative words more important and increases the differentiation of topic words, which effectively improves the accuracy of subsequent topic detection and evolutionary analysis.

Smart teaching evaluation model using weighted naive bayes algorithm

There is a certain subjectivity in the teaching evaluation process, which leads to a low accuracy of the intelligent scoring system. In order to promote the intelligent development of teaching evaluation, based on machine learning, this study briefly introduces the background and current status of teaching evaluation, and describes in detail the relevant algorithm principles of data analysis and modeling using data mining technology and machine learning methods. Moreover, this study describes the establishment process of the traditional classroom teaching evaluation system and uses the classification algorithm in machine learning in the construction of evaluation models to further improve the scientificity and feasibility of teaching evaluation. In addition, in this study, empirical algorithm is used as the basic algorithm to evaluate teaching quality, and the topic word distribution obtained by joint model training is used as the original knowledge. Finally, this research analyzes the performance of this research system through a control experiment. The research results show that the scores of the research model are close to the standard manual scores and can provide a theoretical reference for subsequent related research.

Topic model with incremental vocabulary based on Belief Propagation

Most of the LDA algorithms make the same limiting assumption based on a fixed vocabulary. When these algorithms process data streams in real time, the non-existent words in the vocabulary are discounted. Unexpected words that appear in the streams are incapable to be processed, as the atoms in the Dirichlet distribution are fixed. In order to address the drawbacks as mentioned above, ivLDA with topic–word distribution stemming from the Dirichlet process that has infinite atoms instead of Dirichlet distribution is proposed. ivLDA involves an incremental vocabulary that enables the topic models to process data streams. Besides, two methods are presented to manage the indices of the words, namely, ivLDA-Perp and ivLDA-PMI. ivLDA-Perp is capable of achieving high accuracy and ivLDA-PMI is able to identify the most valuable words to represent the topic. As indicated by experiments, ivLDA-Perp and ivLDA-PMI can achieve superior performance to infvoc-LDA and other state-of-the-art algorithms with fixed vocabulary.

Document Informed Neural Autoregressive Topic Models with Distributional Prior

We address two challenges in topic models: (1) Context information around words helps in determining their actual meaning, e.g., “networks” used in the contexts artificial neural networks vs. biological neuron networks. Generative topic models infer topic-word distributions, taking no or only little context into account. Here, we extend a neural autoregressive topic model to exploit the full context information around words in a document in a language modeling fashion. The proposed model is named as iDocNADE. (2) Due to the small number of word occurrences (i.e., lack of context) in short text and data sparsity in a corpus of few documents, the application of topic models is challenging on such texts. Therefore, we propose a simple and efficient way of incorporating external knowledge into neural autoregressive topic models: we use embeddings as a distributional prior. The proposed variants are named as DocNADEe and iDocNADEe. We present novel neural autoregressive topic model variants that consistently outperform state-of-the-art generative topic models in terms of generalization, interpretability (topic coherence) and applicability (retrieval and classification) over 7 long-text and 8 short-text datasets from diverse domains.

A nonparametric model for online topic discovery with word embeddings

With the explosive growth of short documents generated from streaming textual sources (e.g., Twitter), latent topic discovery has become a critical task for short text stream clustering. However, most online clustering models determine the probability of producing a new topic by manually setting some hyper-parameter/threshold, which becomes barrier to achieve better topic discovery results. Moreover, topics generated by using existing models often involve a wide coverage of the vocabulary which is not suitable for online social media analysis. Therefore, we propose a nonparametric model (NPMM) which exploits auxiliary word embeddings to infer the topic number and employs a “spike and slab” function to alleviate the sparsity problem of topic-word distributions in online short text analyses. NPMM can automatically decide whether a given document belongs to existing topics, measured by the squared Mahalanobis distance. Hence, the proposed model is free from tuning the hyper-parameter to obtain the probability of generating new topics. Additionally, we propose a nonparametric sampling strategy to discover representative terms for each topic. To perform inference, we introduce a one-pass Gibbs sampling algorithm based on Cholesky decomposition of covariance matrices, which can further be sped up using a Metropolis-Hastings step. Our experiments demonstrate that NPMM significantly outperforms the state-of-the-art algorithms.

Trending topics and themes in offsite construction(OSC) research

PurposeThe purpose of this study is to discover the distribution and trends of existing Offsite construction (OSC) literature with an intention to highlight research niches and propose the future outline.Design/methodology/approachThe paper adopted literature reviews methodology involving 1,057 relevant documents published in 2008-2017 from 15 journals. The selected documents were empirically analyzed through a topic-modeling technique. A latent Dirichlet allocation model was applied to each document to infer 50 key topics. A machine learning for language toolkit was used to get topic posterior word distribution and word composition.FindingsThis is an exploratory study, which identifies the distribution of topics and themes; the trend of topics and themes; journal distribution trends; and comparative topic, themes and journal distribution trend. The distribution and trends show an increase in researcher’s interest and the journal’s priority on OSC research. Nevertheless, OSC existing literature is faced with; under-researched topics such as building information modeling, smart construction and marketing. The under-researched themes include organizational management, supply chain and context. The authors also found an overload of similar information in prefabrication and concrete topics. Furthermore, the innovative methods and constraints themes were found to be overloaded with similar information.Research limitations/implicationsThe naming of the themes was based on our own interpretation; hence, the research results may lack generalizability. Therefore, a comparative study using different data processing is proposed. The study also provides future research outline as follows: studying OSC topics from dynamic evolution perspective and identifying the new emerging topics; searching for effective strategies to enhance OSC research; identifying the contribution of countries, affiliation and funding agency; and studying the impact of these themes to the adoption of OSC.Practical implicationsThis study is of values to the scholars, as it could stimulate research to under-researched areas.Originality/valueThis paper justifies a need to have a broad understanding of the nature and structure of existing OSC literature.

GDTM: A Gaussian Dynamic Topic Model for Forwarding Prediction Under Complex Mechanisms

Forwarding is the keyway for information propagation in social networks that affected by complex factors. Due to the fact that the forwarding mechanisms are essentially unclear, this paper focuses on the formation and evolution of the external as well as internal driving mechanisms and develops a Gaussian dynamic topic model for forwarding prediction by incorporating all the information of nodes and edges. First, based on the diversity of communities each user located in, latent Dirichlet allocation (LDA) traditional text modeling method is applied to user following relationships modeling that leads to the initial formation of communities, and user interacting relationships modeling that leads to the evolution of communities. Taking the advantage of LDA topic model in dealing with the problem of polysemy and synonym, we can mine user latent distribution over communities and analyze the external community driving effects. Second, considering the differences in individual habits, time factor is introduced and the dynamic topic model is proposed to model user behavioral attributes. Meanwhile, in regard to continuous user attributes modeling, the parameter of topic-word distribution in the topic model is replaced by multivariate Gaussian distributions, which shown to be effective at capturing the regularities of individual behavioral habits and analyzing the internal individual driving effects. Finally, combining with external and internal factors, a probabilistic graph model is used to modeling forwarding behavior. We propose methods based on Gibbs sampling and an expectation–maximization algorithm to estimate model parameters and fit our model to predict user forwarding actions. Experimental results indicate that the model can not only detect the latent communities but also can improve the performance of forwarding prediction effectively.

ILDA: An interactive latent Dirichlet allocation model to improve topic quality

User-generated content has been an increasingly important data source for analysing user interests in both industries and academic research. Since the proposal of the basic latent Dirichlet allocation (LDA) model, plenty of LDA variants have been developed to learn knowledge from unstructured user-generated contents. An intractable limitation for LDA and its variants is that low-quality topics whose meanings are confusing may be generated. To handle this problem, this article proposes an interactive strategy to generate high-quality topics with clear meanings by integrating subjective knowledge derived from human experts and objective knowledge learned by LDA. The proposed interactive latent Dirichlet allocation (iLDA) model develops deterministic and stochastic approaches to obtain subjective topic-word distribution from human experts, combines the subjective and objective topic-word distributions by a linear weighted-sum method, and provides the inference process to draw topics and words from a comprehensive topic-word distribution. The proposed model is a significant effort to integrate human knowledge with LDA-based models by interactive strategy. The experiments on two real-world corpora show that the proposed iLDA model can draw high-quality topics with the assistance of subjective knowledge from human experts. It is robust under various conditions and offers fundamental supports for the applications of LDA-based topic modelling.